Intelligent garbage recycling and classifying device and control method thereof

Abstract

The invention discloses an intelligent garbage recycling and classifying device which comprises a conveying module for conveying to-be-recognized garbage and dispersing the garbage in the conveying process; the image acquisition module acquires image information of to-be-identified garbage; the main control module performs garbage identification after receiving the image information to obtain an identification result, sends a control instruction to the transmission module, analyzes and controls the rotation angle of the rotation steering engine and the rotation direction of the rotatable baffle plate according to the identification result, and sends a stop instruction to the rotation steering engine; the rotating steering engine controls the rotating direction of the rotatable baffle according to the control instruction, and garbage is thrown into the two-dimensional holder tray or the recyclable garbage can. The two-dimensional holder tray is rotated according to the recognition result to throw the garbage into the corresponding garbage can; the main control module is further used for starting the conveying module to continue to convey the to-be-recognized garbage after throwing is completed. The device not only can realize an automatic single-classification function, but also can realize a multi-classification function, does not need manual classification, and is high in efficiency, labor-saving and high in practicability.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage sorting devices, and in particular to an intelligent garbage recycling and sorting device and a control method thereof. Background Art

[0002] With the acceleration of urbanization, garbage disposal has become an important issue in urban management. People's existing household trash cans basically do not have the function of garbage sorting, which leads to people mixing recyclable garbage with non-recyclable garbage when using them, making it difficult to separate. Manual garbage sorting not only takes up more personal time, but also has low efficiency. Most modern smart trash cans can only achieve single classification, which requires garbage to be put in one by one to be correctly classified successfully. This not only wastes people's time, but also has low efficiency.

[0003] In recent years, with the rapid development of society and the continuous improvement of social economy, the sales volume of goods has gradually increased. However, the public health service personnel are insufficient, resulting in untimely garbage production. The garbage overflowing from the barrels is scattered all over the ground, which is easy to breed bacteria. Coupled with the hot weather, the odor can easily spread quickly, affecting public health. Summary of the invention

[0004] In view of the defects in the prior art, the present invention provides an intelligent garbage recycling and classification device and a control method thereof, which can not only realize the automatic single classification function, but also realize the multi-classification function, without the need for manual classification, with high efficiency, labor saving and high practicality.

[0005] In a first aspect, the present invention provides an intelligent garbage recycling and sorting device, comprising: a transmission module, an image acquisition module, a main control module, a rotating steering gear, a rotatable baffle, a two-dimensional pan-tilt tray and a communication module;

[0006] The conveying module is used to convey the garbage to be identified when it is detected that there is garbage on the conveyor, convey the garbage to be identified, and disperse the garbage during the transmission process, and stop working in the image acquisition area;

[0007] The image acquisition module is used to collect image information of the garbage to be identified and transmit the image information to the main control module;

[0008] The main control module is used to perform garbage recognition after receiving the image information, obtain the recognition result, and send a stop command to the transmission module, analyze and control the rotation angle of the rotating steering gear and the rotation direction of the rotatable baffle according to the recognition result, and send a control command to the rotating steering gear;

[0009] The rotating steering gear controls the rotating direction of the rotatable baffle according to the control command to put the garbage into the two-dimensional pan-tilt tray or the recyclable garbage bin;

[0010] The two-dimensional pan-tilt tray rotates the tray according to the recognition result to put the garbage into the corresponding garbage bin;

[0011] The main control module is also used to start the transmission module to continue transmitting the garbage to be identified after the delivery is completed;

[0012] The communication module is used to transmit the data and positioning information processed by the main control module to the cloud platform.

[0013] In a second aspect, another embodiment of the present invention provides a control method for an intelligent garbage recycling and sorting device, comprising the following steps:

[0014] The conveying module conveys the garbage to be identified when it detects that there is garbage on the conveyor belt, disperses the garbage during the transmission process, and stops working in the image acquisition area;

[0015] The image acquisition module acquires image information of the garbage to be identified and transmits the image information to the main control module;

[0016] After receiving the image information, the main control module performs garbage recognition, obtains the recognition result, and sends a stop command to the transmission module. According to the recognition result, the main control module analyzes and controls the rotation angle of the rotating steering gear and the rotation direction of the rotatable baffle, and sends a control command to the rotating steering gear;

[0017] The servo drives control the rotation direction of the rotatable baffle according to the control command, and puts the garbage into the two-dimensional pan-tilt tray or the recyclable garbage bin;

[0018] The two-dimensional pan-tilt tray rotates the tray according to the recognition result to put the garbage into the corresponding trash can;

[0019] After the two-dimensional pan-tilt tray is placed, the main control module starts the transmission module to continue transmitting the garbage to be identified.

[0020] Beneficial effects of the present invention:

[0021] An intelligent garbage recycling and classification device provided by an embodiment of the present invention can not only realize an automatic single classification function, but also realize a multi-classification function, without the need for manual classification, with high efficiency, labor saving, and high practicality.

[0022] The intelligent garbage recycling and sorting device can also compress recyclable garbage by setting up a compression module, reducing the volume of garbage, facilitating storage and transportation, and improving the efficiency of resource recycling.

[0023] The intelligent garbage recycling and classification device sets up a full load detection module to detect whether the garbage capacity is full and sends the detection signal to the main control module. It can monitor the status of the garbage bin in real time and process it in time when it is full.

[0024] The intelligent recycling and sorting garbage device is equipped with a solar power supply module, which can absorb solar energy for charging on sunny days and power itself through the built-in energy storage system at night or on cloudy days, thus avoiding the high energy consumption of artificial power generation. The solar panels are used for charging and energy storage, and no external power supply is required during use. It can also continue to work in long rainy weather, thereby improving the energy saving effect and work efficiency of the intelligent garbage can.

[0025] The main control module of the intelligent recycling and sorting garbage device adopts the YOLOv8 target detection algorithm, which can accurately classify different types of garbage, and adds the EMA mechanism to the recognition result to enhance the data set, which improves the model performance and makes the classification results more accurate and stable.

[0026] A control method for an intelligent garbage recycling and sorting device provided in another embodiment of the present invention is based on the same inventive concept as the intelligent garbage recycling and sorting device and has the same beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0028] Figure 1 A schematic structural diagram of an intelligent garbage recycling and sorting device provided by a first embodiment of the present invention is shown.

[0029] Figure 2 A flowchart of a target detection algorithm based on YOLOv8 in an embodiment of the present invention is shown;

[0030] Figure 3 The figure shows the accuracy results of predicting different types of garbage based on the target detection algorithm of YOLOv8 in an embodiment of the present invention;

[0031] Figure 4 The figure shows the accuracy results of predicting different types of garbage based on the target detection algorithm of YOLOv5 in an embodiment of the present invention;

[0032] Figure 5 The figure shows the recall rate result of predicting different types of garbage based on the object detection algorithm of YOLOv8 in an embodiment of the present invention;

[0033] Figure 6 The figure shows the recall rate result of predicting different types of garbage based on the object detection algorithm of YOLOv5 in the embodiment of the present invention;

[0034] Figure 7 A schematic diagram showing a loss function introduced into a model in an embodiment of the present invention is shown;

[0035] Figure 8 The figure shows a Loss curve diagram based on the YOLOv5 target detection algorithm in an embodiment of the present invention;

[0036] Fig. 9 The figure shows a Loss curve diagram based on the YOLOv8 target detection algorithm in an embodiment of the present invention;

[0037] Fig.10 A display screen showing a plurality of test results;

[0038] Fig.11 A flow chart of a control method of an intelligent garbage recycling and sorting device provided by another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0041] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0042] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0043] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0044] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0045] like Figure 1 As shown, an intelligent garbage recycling and classification device provided by the first embodiment of the present invention includes an outer barrel, and a transmission module, an image acquisition module, a main control module, a rotating servo, a rotatable baffle, a two-dimensional pan-tilt tray and a communication module are arranged in the outer barrel. Among them, the outer barrel serves as the carrier of the entire product, and multiple garbage bins for storing garbage are arranged in the outer barrel. Two delivery ports are arranged on the top of the outer barrel, one is a single-classification delivery port, and the other is a multi-classification delivery port. Users can choose multi-classification and single-classification according to different needs. The image acquisition module uses a camera. The control chip used in the main control module is the STM32F103VET6 chip, which is designed based on the ARM Cortex M3 core and has 100 pins. The control board is equipped with a motor drive module interface, a six-way servo interface, an extended sensor interface, and a serial port. It is mainly responsible for communication with the cloud platform, buzzer alarm, motor drive, and the rotation angle of the servo. The two control boards communicate data through the serial port. The two-dimensional pan-tilt uses multiple MG996R servos as power, has two degrees of freedom, and can carry out sorting of multiple garbage on the tray. The MG996R servo has three connecting wires, the red wire is connected to the 5V DC power supply, the brown wire is connected to GND, and the yellow wire is the data control wire, which is connected to the GPIO port of the microcontroller. The main control module analyzes the rotation angle of the rotating servo according to the image recognition results, and sends a PWM (pulse width modulation) signal to the rotating servo. The duty cycle of the PWM signal determines the rotation angle of the rotating servo. The rotating servo drives the internal motor to rotate according to the rotation angle, and gives power to the swing arm through the reduction gear. At the same time, the potentiometer returns the current position signal to determine whether the set position has been reached. The normal working voltage of the servo is 4.8V~6V, the rotation angle is 180°, the pulse period is 20ms, and the pulse width of 0.5ms~2.5ms corresponds to an angle of -90°~+90°.

[0046] When performing single classification, when the garbage to be identified passes through the single classification delivery port, the image acquisition module collects the image information of the garbage to be identified and transmits the image information to the main control module for identification. After receiving the image information, the main control module identifies the garbage and obtains the identification result. The main control module analyzes the rotation direction of the rotatable baffle and the rotation angle of the rotating servo in the identification area according to the identification result. The rotatable baffle is controlled by the rotating servo to automatically adjust the rotation angle so that the garbage falls accurately into the two-dimensional pan-tilt tray, and then the two-dimensional pan-tilt controls the tray to make the garbage fall into the corresponding trash can. In order to prevent the garbage from being too heavy, a load-bearing pin is set under the baffle.

[0047] When multi-classification delivery is carried out, the transmission module plays a key role. The transmission module mainly includes a conveyor belt with multiple transmission gears embedded. The speed of each conveyor belt is different. The motor drives the driving wheel to drive the conveyor belt. The number of teeth on the gear is 54. The garbage is directly put into the conveyor belt through the multi-classification delivery port. A photoresistor is set to detect whether there is garbage to be identified on the conveyor belt. When garbage is detected, the motor is started. The garbage is transmitted through multiple conveyor belts. The speed of each conveyor belt is different, so that it is basically dispersed during the transmission process. The garbage is then transmitted to the image acquisition area. The image acquisition module collects the image information of the garbage to be identified and transmits it to the main control module. The conveyor belt stops immediately. The main control module receives the image information and identifies it to obtain the identification result. The identification result is analyzed to control the rotation angle of the rotating servo and the rotation direction of the rotatable baffle. The control command is sent to the rotating servo. The main control module sends a stop command to the transmission module. The rotating servo controls the rotation direction of the rotatable baffle according to the control command to put the garbage into the two-dimensional pan-tilt tray. The two-dimensional pan-tilt tray rotates the tray according to the identification result to put the garbage into the corresponding trash can. After completing the first garbage classification, the main control module starts the transmission module to continue transmitting the garbage to be identified. The above process is repeated until all the garbage is identified and classified.

[0048] The device also includes a communication module, a display module, and a GPS positioning module. The display module is used to display the detection image and data related to garbage classification, and the GPS positioning module is used for positioning. The main control module transmits the processed data and positioning information to the cloud platform through the communication module. The communication module adopts the NB-IoT communication module, and the cloud platform adopts the Alibaba Cloud platform to ensure its compatibility with the Alibaba Cloud IoT platform. Create devices and products on the Alibaba Cloud IoT platform, and associate the NB-IOT module with the devices and products. In this way, the device can be managed and controlled on the Alibaba Cloud platform, and the data sent by the device can be received. It can also locate its own position, collect the quantity data of each type of garbage through the camera, and transmit the collected data to the Alibaba Cloud platform through the NB-IOT module. Create an appropriate database on the Alibaba Cloud platform to store the garbage quantity data, and use Alibaba Cloud's data visualization tools (such as data visualization large screen, Quick BI, etc.) to create dashboards or reports to display the quantity of different types of garbage to ensure that the data visualization interface can update the data in real time. The display module adopts an HDM display, which refers to a display device that supports an HDMI interface, which can receive HDMI signals and display them as high-definition images and sounds. HDMI displays usually have the characteristics of high resolution, wide color gamut, and high image quality. The page is written in Python and can display the amount and type of garbage, whether the classification is successful, real-time monitoring images and garbage promotional videos on the screen in real time.

[0049] In another embodiment of the present invention, the device also includes a compression module, which is used to compress recyclable garbage. The compression module includes a telescopic motor, a compression partition and a photocoupler isolation relay module. When the servo is turned to put the garbage into the recyclable garbage bin, the telescopic motor pushes the compression partition to compress the recyclable garbage under the control of the photocoupler isolation relay module. In this embodiment, an XC759 electric push rod DC 12V reciprocating telescopic motor is used, and a multi-channel photocoupler isolation relay module is used to drive the telescopic motor. The working voltage of the photocoupler isolation relay module is 12V. The high level or low level trigger is set by the jumper, and the relay and the patch photocoupler are combined to achieve isolation between the low voltage circuit and the high voltage circuit. The signal in the low voltage circuit emits light through the LED at the input end of the photocoupler, and then the light is transmitted to the photosensitive diode at the output end through the isolation material, thereby generating an electrical signal corresponding to the input signal to control the switching state of the relay. The contacts of the relay can be switched in the high voltage circuit, and there is no direct electrical connection between the low voltage circuit and the high voltage circuit, so as to achieve electrical isolation and protection of the low voltage circuit. The power of the motor should be selected according to the power required by the production machinery, and the motor should be operated under rated load as much as possible.

[0050] In another embodiment of the present invention, the device also includes a full load detection module, which is used to detect whether the capacity of each trash can reaches the set trash capacity threshold, and sends the detection information to the main control module. When the main control module receives a signal that the trash can capacity reaches the set trash can capacity threshold, it sends information showing that the corresponding trash can is full to the display module, and sends full load information to the cloud platform. Specifically, the full load detection module uses a laser sensor module. Inside each trash can, at a height of 75% from its bottom, laser head sensor modules KY-008 are installed on opposite sides. This small laser module is suitable for 5 volt DC power supply and can emit point lasers. In order to receive the laser signal, we installed a photoresistor sensor module at the corresponding position on the other side of the trash can, which is also powered by 5 volt DC. The DO terminal of the module outputs a TTL switch signal. Under normal circumstances, that is, when the amount of garbage does not reach 75% of the capacity of the trash can, the DO terminal outputs a low level. However, once the amount of garbage exceeds 75%, the garbage will block the laser, causing the photoresistor sensor to be unable to receive enough light to reach the preset threshold, and the DO terminal will output a high level. In order to monitor the status of the trash can in real time, we connect the DO terminals of the four photoresistors to the general input / output ports of the STM32VET6 chip and set these ports to input mode. In this way, the TTL digital signal status of each DO port can be detected in real time.

[0051] In another embodiment of the present invention, the intelligent garbage recycling and sorting device is provided with a solar power supply module for converting solar energy into electrical energy to power the intelligent garbage recycling and sorting device. The solar power supply module can absorb solar energy for charging on sunny days, and supply power to itself through a built-in energy storage system at night or on cloudy days, thereby avoiding the high energy consumption of artificial power generation. The solar power supply module has the following functions: Energy storage function: The intelligent garbage recycling and sorting device does not rely on an external power supply during operation, but is powered by solar energy. Therefore, a large-capacity energy storage device is required to ensure normal operation even when it cannot be charged for a long time. Power supply function: Solar panels are used for charging and energy storage. No external power supply is required during use, and it can continue to work even in long periods of rainy weather, thereby improving the energy-saving effect and work efficiency of the device.

[0052] In another embodiment of the present invention, the main control module includes a model construction unit, which uses the YOLOv8 target detection algorithm to construct garbage classification recognition. The model garbage classification recognition model includes a backbone feature network, a feature extraction network and a prediction network. The backbone feature network extracts preliminary features of each type of garbage through a convolutional layer, and the feature extraction network is used to extract effective features from the preliminary features. The prediction network is used to predict and output classification results based on effective features. It also includes an image recognition unit, which uses a garbage classification recognition model to perform garbage classification recognition on the collected image information and recognize different types of garbage. The main control module also includes a recognition result processing unit, which uses a sliding window mechanism and an exponential sliding average to process the recognition result to obtain a final classification result.

[0053] Specifically, YOLOv8 is a target detection algorithm based on deep learning, which can be used for garbage classification tasks. It has four parts: Backbone, Neck, Head, and Loss. The deep learning convolutional neural network (CNN) architecture, Darknet-53, was first adopted. Its core is the feature extraction network as the Backbone. Neck is responsible for further processing and fusing the features extracted by the Backbone to better perform multi-scale object detection. Common Neck structures include FPN (Feature Pyramid Network) and so on. Head is responsible for the final category and bounding box prediction. Loss is used to calculate the loss of categories, bounding boxes, and masks. The flow chart of the target detection algorithm based on YOLOv8 is as follows: Figure 2 shown.

[0054] An image with a length, width and height of 640×640×3 is input into the network. The Conv (convolutional layer) extracts the image features of each type of garbage. The C2f (Cross Stage Partial with Fusion) divides the features into two parts, and then performs convolution operations on each part, and then fuses the results to enhance the feature expression ability. Finally, the SPPF (Spatial PyramidPooling Fast) spatial pyramid pooling layer performs pooling operations on features at multiple scales to capture a wider range of image information. After the FPN transformation, it is the result of a small size at the top and a large number of convolutions, which is rich in semantic information, and a large size at the bottom and a small number of convolutions, which passes down the strong semantic features of the high-level, enhances the entire pyramid, and enhances the semantic information.

[0055] The feature map after FPN is divided into two parts, one of which is directly transferred to the subsequent layer, and the other part is processed after complex convolution operations to obtain the processed feature map, and then the processed feature map is spliced ​​with the directly transferred part to obtain the spliced ​​feature map, and the spliced ​​feature map is further convolved to obtain feature maps of different scales, and then the feature maps of different scales are upsampled (Upsample) or downsampled (Downsample), and then fused (for example, by addition or splicing operations). Finally, a multi-scale fused feature map is generated for the prediction of the Head part.

[0056] Head will receive some feature images of Neck, and first fork into two branches. Each branch will pass through Conv to extract features, Batch Normalization to standardize the output of the convolution layer, stabilize the training process, and speed up the convergence. Then, nonlinear Activation (activation function) will be introduced. The two feature maps after forking will pass through a Conv2d convolution layer respectively. The function of this layer is to map the high-dimensional features to the final output channel in preparation for classification and regression prediction.

[0057] First, calculate the Classification Loss (classification loss) through a fully connected layer or 1x1 convolution layer to output the category prediction, and then use the Cross-Entropy Loss (cross entropy loss function) to calculate the difference between the predicted category and the actual category. Finally, calculate the Bbox Loss (bounding box regression loss) through a fully connected layer or 1x1 convolution) to output the bounding box prediction, and then use the regression loss function such as Smooth L1 Loss or GIoU Loss to calculate the difference between the predicted bounding box and the actual bounding box. Then output the predicted image.

[0058] Data collection and annotation:

[0059] A large amount of image data containing different types of garbage is collected and annotated. Each garbage object is framed and its category is marked. The data set comes from 1240 images of each type of garbage collected by USB cameras in real life. The collected images are divided into four categories and put into the training set and validation set respectively. There are 200 images of recyclable garbage, 271 images of hazardous garbage, 156 images of kitchen waste, and 365 images of other garbage. The specific classification and quantity of each type of garbage are shown in Table 1. Among all the data sets, the training samples account for 80%, that is, 992 images, and the validation set samples account for 20%, that is, 248 images.

[0060] Table 1

[0061]

[0062] Model training and deployment:

[0063] Use the collected and annotated data to input them into the YOLOv8 and YOLOv5 models for training. During the training process, the model will automatically learn to identify different categories of garbage objects and generate corresponding detection results. In this embodiment, the trained model is deployed on the domestic NVIDIA Jetson nano. The YOLOv8 and YOLOv5 deployed on Jetson nano are compared, where the YOLOv8 model of Jetson nano is converted to TensorRT and then built using the official DeepStream framework.

[0064] Environment configuration:

[0065] The experiment in this embodiment was carried out under the Windows 10 operating system, with a GeForce RTX 3060 GPU, an R7-5800 CPU, 16GB of memory, and a CUDA version of 12.0. The development environment software used was Pycharm, the language was Python 3.10, and the deep learning framework was Facebook's open source Pytorch. The iteration parameter epochs of the experiment was set to 100, the batch size was 16, the stochastic gradient descent method was used, and the weight decay was 0.0005.

[0066] Precision-Confidence:

[0067] The Precision-Confidence Curve is a curve used to show the change in the accuracy of the model under different confidence thresholds. Confidence indicates the confidence of the model in a certain prediction result, usually between 0 and 1. By adjusting the confidence threshold, you can observe the accuracy of the model at different confidence levels. The calculation formula is as follows:

[0068]

[0069] Among them, TP (True Positives) is the true positive example, that is, the number of samples correctly predicted as positive, and FP (False Positives) is the false positive example, that is, the number of samples incorrectly predicted as positive.

[0070] YOLOv8 training results:

[0071] from Figure 3 As can be seen in , the accuracy of all categories increases with the increase of confidence. This is in line with expectations, as higher confidence means that the model is more confident in the prediction, thus reducing the probability of false detection.

[0072] The thick blue line represents the overall performance of all categories. It can be seen that when the confidence level is 0.917, the average accuracy of all categories reaches 1.00. This shows that at this confidence threshold, the model's prediction results are very reliable. Batteries, Pills, Cans and other categories: The curves of these categories have an accuracy close to 1.0 when the confidence level is low, indicating that the model predictions of these categories are very accurate.

[0073] Categories such as Waterbox, Potato, and Bairob: The curves for these categories have slightly lower precision when the confidence is low, but as the confidence increases, the precision increases rapidly to close to 1.0.

[0074] Cipian, Redrob, Stone, Brick and other categories: The curves of these categories have lower precision when the confidence level is low, but as the confidence level increases, the precision gradually increases and eventually approaches 1.0.

[0075] When the confidence level is lower than 0.2, the accuracy of some categories drops significantly, indicating that at low confidence levels, the model may produce more false detections.

[0076] When the confidence level is higher than 0.8, the accuracy of all categories approaches 1.0, indicating that the model has a high confidence level in target detection and the prediction results are very reliable.

[0077] YOLOv5 training results:

[0078] from Figure 4 It can be seen that at a confidence level of 0.929, the average precision of all categories reached 1.00. This shows that at this confidence threshold, the model's prediction results are very reliable.

[0079] Battery, Pills, Cans and other categories: The curves of these categories have an accuracy close to 1.0 when the confidence level is low, indicating that the model predictions of these categories are very accurate.

[0080] Categories such as Waterbox, Potato, and Bairob: The curves for these categories have slightly lower precision when the confidence is low, but as the confidence increases, the precision increases rapidly to close to 1.0.

[0081] Redrob, Cipian, Stone, Brick and other categories: The curves of these categories have low precision when the confidence level is low, but as the confidence level increases, the precision gradually increases and eventually approaches 1.0.

[0082] Recall-Confidence

[0083] Recall-Confidence Curve is a curve used to show the change of recall rate of the model under different confidence thresholds. Confidence indicates the confidence of the model in a certain prediction result, usually between 0 and 1. By adjusting the confidence threshold, you can observe the recall performance of the model under different confidence levels. The calculation formula is as follows

[0084]

[0085] Among them, TP (True Positives): True positives, that is, the number of samples correctly predicted as positive. FN (False Negatives): False negatives, that is, the number of samples that are actually positive but are wrongly predicted as negative.

[0086] YOLOv8 training results:

[0087] from Figure 5 It can be seen that the thick blue line in the figure represents the comprehensive performance of all categories, that is, "all classes". This line shows that as the confidence increases from 0 to 1, the recall rate begins to gradually decrease from 1, especially when the confidence is close to 1.0, the decrease is significantly larger. This shows that the model reduces false alarms (false positives) at high confidence thresholds, but also misses some real targets (true positives).

[0088] The thin lines of various colors represent the recall-confidence curves of different categories, such as "battery", "pills", "cans", etc. In the range of lower confidence values ​​(0 to 0.8), the recall rates of all categories are close to 1.0, indicating that the model can detect most target objects in this range. In the range of high confidence values ​​(0.8 to 1.0), the recall rate curves of various categories are clearly differentiated, and the recall rates of some categories (such as "battery") decrease slowly, while the recall rates of other categories (such as "brick") decrease rapidly, showing the detection uncertainty of the model on these categories.

[0089] YOLOv5 training results:

[0090] from Figure 6 It can be seen that the curves of categories such as battery, pills, and waterbox still maintain a high recall rate at a higher confidence threshold, indicating that the model has a strong detection ability for these categories.

[0091] Categories such as cans, potato, and redrob: The curves of these categories drop rapidly at lower confidence thresholds, indicating that the model's detection ability for these categories is relatively weak and is prone to missed detection.

[0092] The curves of some categories (such as bairob, stone, brick, etc.) are steeper, indicating that the recall rates of these categories drop faster at high confidence thresholds, reflecting that these categories are more difficult to detect.

[0093] The curves of other categories (such as battery, pills, etc.) are relatively flat, indicating that these categories maintain a high recall rate over a wider range of confidence thresholds and the model has a good detection effect on these categories.

[0094] The Loss curve is an important tool for evaluating model performance during deep learning training. It shows the changing trend of the loss function value during training and verification. The lower the Loss value, the higher the model's detection accuracy for the target in the test set. The following two loss functions are used.

[0095] (1) Classification loss:

[0096] VFL (Varifocal Loss) introduces focal loss by combining the ideas of classification loss and focal loss. By adjusting the weight of the samples, the samples that are difficult to classify (i.e., those close to the decision boundary) occupy a larger weight in the loss function, so that the model pays more attention to these difficult samples. VFL further introduces a weighted mechanism for the sample confidence distribution, weighting the loss of each sample, so that samples with high confidence occupy a larger proportion in the loss calculation. The calculation formula is as follows

[0097]

[0098] (2) Target recognition loss:

[0099] DFL (Distribution Focal Loss) is a loss function designed specifically for target detection tasks. Through distribution weighting, DFL can dynamically adjust the loss value according to the difficulty of the sample. This means that samples that are difficult to classify will be given higher weights, prompting the model to pay more attention to these difficult samples. The introduction of Focal Loss helps the model deal with the problem of imbalance between positive and negative samples. Focal Loss reduces the weight of simple samples, allowing the model to pay more attention to samples that are difficult to classify. The schematic diagram is as follows Figure 7 As shown, the calculation formula is as follows:

[0100] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1))

[0101] The position of the box is modeled as a General distribution in the figure, allowing the network to quickly focus on the distribution of positions close to the target position.

[0102] The YOLOv5 and YOLOv8 Loss curves are as follows Figure 8 , Fig. 9 shown.

[0103] 1. Loss function:

[0104] Similarities:

[0105] (1) Train / Box Loss:

[0106] The loss of bounding box regression for YOLOv5 shows a clear downward trend, from around 0.08 at the beginning to below 0.02, while YOLOv8 drops from 1.4 to 0.4, a large drop, indicating that the model gradually improves the prediction accuracy of bounding boxes during training.

[0107] (2) Train / Cls Loss:

[0108] The classification loss of YOLOv5 dropped from about 0.06 to close to 0, while that of YOLOv8 dropped from 3.5 to about 1.0, indicating a significant reduction in classification loss. The performance of the model in object classification gradually improved.

[0109] (3)Val / Box Loss, Val / Obj Loss, Val / Cls Loss:

[0110] The loss on the YOLOv5 validation set also shows a similar downward trend, while the YOLOv8 loss downward trend is consistent with the training set, but with greater fluctuations, which may be due to the complexity of the validation set or insufficient data.

[0111] Differences:

[0112] (4) Train / Obj Loss:

[0113] The loss of YOLOv5 target confidence dropped from about 0.025 to close to 0.005, indicating that the model's confidence prediction of the target has become more accurate.

[0114] (5)Train / DFL Loss (Distribution Focal Loss):

[0115] YOLOv8 drops from 1.6 to 1.0, indicating that the loss on the target distribution is also decreasing.

[0116] 2. Precision and Recall

[0117] (1)Precision:

[0118] The accuracy of YOLOv5 gradually increased and stabilized, and finally approached 1.0, indicating high recognition accuracy. The accuracy of YOLOv8 increased from the initial 0.75 and stabilized, and finally approached 1.0, indicating high recognition accuracy.

[0119] (2)Recall:

[0120] The recall rate of YOLOv5 gradually increased and stabilized, eventually approaching 1.0, while the recall rate of YOLOv8 also gradually increased and eventually approached 1.0, indicating that the model can recognize most targets.

[0121] 3. Average Precision (mAP)

[0122] MAP (Mean Average Precision) is a commonly used evaluation indicator in target detection, which is used to measure the performance of the model on a set of test data. The area under the precision-recall curve is calculated using an interpolation method. Commonly used methods are the 11-point interpolation method or the direct integration method. The 11-point interpolation method takes a point every 0.1 in the recall range of 0 to 1, calculates the maximum precision value of the point, and adds the AP values ​​of all categories. The calculation formula is as follows:

[0123]

[0124] The average precision is obtained by dividing the sum of AP by the total number of categories N. The calculation formula is as follows:

[0125]

[0126] The YOLOv8 training results are shown in Table 2, and the YOLOv5 training results are shown in Table 3:

[0127] Table 2

[0128]

[0129] Table 3

[0130] Class Images Istance Box Right mAP50 battery 247 31 0.995 1 0.995 pills 247 31 0.993 0.968 0.97 cans 247 30 0.968 0.995 0.982 waterbox 247 20 1 0.954 0.995 potato 247 10 0.978 1 0.995 bairob 247 13 0.986 1 0.995 redrob 247 16 0.934 1 0.991 cipian 247 14 0.982 1 0.995 stone 247 72 0.998 1 0.995 brick 247 10 0.973 1 0.995 all 247 247 0.986 0.989 0.99

[0131] (1) mAP@0.5:

[0132] The average precision of YOLOv5 when the IoU threshold is 0.5 gradually increases and stabilizes, and finally approaches 1.0, while the average precision of YOLOv8 when the IoU threshold is 0.5 gradually increases and stabilizes, and finally approaches 1.0, showing a high detection accuracy.

[0133] (2) mAP@0.5:0.95:

[0134] The average precision of YOLOv5 at different IoU thresholds increases from about 0.4 to above 0.6, while that of YOLOv8 increases from about 0.4 to about 0.65, indicating that the model has good detection performance under a variety of IoU conditions.

[0135] Comprehensive analysis:

[0136] (1) Loss function:

[0137] The initial loss value of YOLOv8 is significantly higher than that of YOLOv5, but the decline is also greater, indicating that YOLOv8 has a larger error in the early stage of training, but it can be significantly improved after training. The loss on the validation set fluctuates greatly, which may be due to the complexity of the data or the generalization ability of the model on the validation set needs further optimization.

[0138] (2) Precision and recall:

[0139] Both perform well in terms of precision and recall, and both end up close to 1.0, showing high recognition accuracy and comprehensiveness.

[0140] (3) Average accuracy:

[0141] YOLOv8 performs slightly better than YOLOv5 in mAP@0.5 and mAP@0.5:0.95, especially in the different garbage cases.

[0142] Since computer vision uses cameras as sensors to collect images, different light intensities may have different recognition effects on the same item, and during the recognition process, light and the moment when garbage falls will cause recognition errors, but when the garbage tends to be stable, the recognition rate is the highest. Therefore, this embodiment uses a sliding window mechanism combined with EMA (Exponential Moving Average). EMA is used to reduce fluctuations in time series data and reveal data trends. EMA gives higher weights to recent data points, so it can respond to changes in the latest data more quickly. A fixed-size window is initialized at the starting position of the data sequence. The data points in the window slide forward over time, and an exponentially weighted average is performed on the window. The output value is considered to be the true value, and the calculation formula is:

[0143] EMA t =α·x t +(1-α) EMA t-1

[0144] Among them, EMA tis the EMA value at time t, x t is the data value at time t, α is the smoothing coefficient, the value range is 0<α≤1, EMA t-1 It is the EMA value at time t-1.

[0145] In summary, this paper consists of two different recognition algorithms, different lighting conditions, and whether there is EMA. The eight test sets contain 1240 images of each type of garbage as shown in the following table. The models with data augmentation and without data augmentation are tested respectively. The test results are shown in Table 4 and Table 5.

[0146] Table 4

[0147]

[0148] Table 5

[0149]

[0150] As shown in Tables 4 and 5, YOLOv8 shows higher average precision than YOLOv5 regardless of whether EMA is used (without data augmentation: 89.92 vs 87.59; with data augmentation: 89.96 vs 88.17). This shows that YOLOv8 has made significant improvements in model optimization and performance improvement compared to YOLOv5. The test datasets using EMA generally show higher precision, especially in YOLOv5 (for example, test datasets 1 and 3 compared to 2 and 4). In YOLOv8, the impact of EMA is relatively small, but a certain degree of performance improvement can still be observed. Data augmentation significantly improves the stability and accuracy of the test results. For example, in the test dataset 5 of YOLOv8, the average value with data augmentation is 90.43, while the average value without data augmentation is 89.97. This shows that data augmentation plays an important role in improving the generalization ability and robustness of the model. Natural light and lighting conditions do not have a significant impact on the test results, but in certain cases (such as test datasets 2 and 4), the accuracy under lighting conditions is slightly improved. This may be related to the characteristics of the dataset and the model's ability to adapt to different lighting conditions.

[0151] In summary, YOLOv8 is superior to YOLOv5 in all aspects. Both EMA and data enhancement have a significant effect on improving model performance. The results under different lighting conditions show that the model has good adaptability. Future improvements can focus on further optimizing the model structure and data enhancement strategy to obtain higher accuracy and stability.

[0152] Deploy the trained model on Jetson nano and start the corresponding program. The screen will display the PYQT5 page and the image captured by the camera. In this embodiment, 10 types of garbage are put into the test respectively. 9 types of garbage include water bottles, cans, batteries, pills, white radish, carrots, potatoes, stones, and porcelain pieces. At the same time, these 9 types of garbage are divided into four categories: recyclable garbage, hazardous garbage, other garbage, and kitchen waste. The final test results are as follows: Fig.10 shown.

[0153] The intelligent garbage recycling and classification device provided by the embodiment of the present invention can not only realize the automatic single classification function of garbage, but also realize the multi-classification function. The quantity and type of garbage put in can be displayed in real time on the screen of the garbage bin, and can also be uploaded to the cloud platform through NB-IOT for back-end big data analysis. In addition, a garbage full load module and a compression module are installed in the bin. When the garbage is recyclable, the compression device will automatically compress the garbage. The full load detection module can detect whether the garbage bin is full. When it is full, the garbage full load signal and location information will be automatically uploaded to the cloud platform, and the relevant information will be fed back to the management personnel. A solar power supply module is set up, which can absorb solar energy for charging on sunny days, and supply power through the built-in energy storage system at night or on cloudy days, thereby avoiding the high energy consumption of artificial power generation. The YOLOv8 target detection algorithm is adopted to accurately classify different types of garbage, and the EMA mechanism is added to the recognition result to enhance the data set. Even if the first recognition error is made, it can be automatically corrected by the algorithm, which improves the model performance and makes the classification result have higher accuracy and stability.

[0154] In the first embodiment described above, an intelligent garbage recycling and classification device is provided. Correspondingly, the present application also provides a control method for an intelligent garbage recycling and classification device. Fig.11 , which is a flow chart of a control method of an intelligent garbage recycling and classification device provided by another embodiment of the present invention. Since the method embodiment is basically similar to the device embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the device embodiment. The method embodiment described below is only illustrative.

[0155] like Fig.11 As shown, another embodiment of the present invention provides a control method for an intelligent garbage recycling and classification device, the intelligent garbage recycling and classification device adopts the device described in the above embodiment, and the control method includes the following steps:

[0156] The conveying module conveys the garbage to be identified when it detects that there is garbage on the conveyor belt, disperses the garbage during the transmission process, and stops working in the image acquisition area;

[0157] The image acquisition module acquires image information of the garbage to be identified and transmits the image information to the main control module;

[0158] After receiving the image information, the main control module performs garbage recognition, obtains the recognition result, and sends a stop command to the transmission module. According to the recognition result, the main control module analyzes and controls the rotation angle of the rotating steering gear and the rotation direction of the rotatable baffle, and sends a control command to the rotating steering gear;

[0159] The servo drives control the rotation direction of the rotatable baffle according to the control command, and puts the garbage into the two-dimensional pan-tilt tray or the recyclable garbage bin;

[0160] The two-dimensional pan-tilt tray rotates the tray according to the recognition result to put the garbage into the corresponding trash can;

[0161] After the two-dimensional pan-tilt tray is placed, the main control module starts the transmission module to continue transmitting the garbage to be identified.

[0162] The control method of the intelligent garbage recycling and sorting device and the intelligent garbage recycling and sorting device provided in the embodiment of the present invention are based on the same inventive concept and have the same beneficial effects, which will not be described in detail here.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An intelligent garbage recycling and sorting device, characterized in that: include: Transmission module, image acquisition module, main control module, rotating servo, rotatable baffle, two-dimensional pan-tilt tray and communication module; The conveying module starts conveying the garbage to be identified when detecting that there is garbage on the conveyor, disperses the garbage during the transmission process, and stops working in the image acquisition area; The image acquisition module is arranged in the image acquisition area, and is used to acquire image information of the garbage to be identified, and transmit the image information to the main control module; The main control module is used to perform garbage recognition after receiving the image information, obtain the recognition result, and send a stop command to the transmission module, analyze and control the rotation angle of the rotating steering gear and the rotation direction of the rotatable baffle according to the recognition result, and send a control command to the rotating steering gear; The rotating steering gear controls the rotating direction of the rotatable baffle according to the control command to put the garbage into the two-dimensional pan-tilt tray; The two-dimensional pan-tilt tray rotates the tray according to the recognition result to put the garbage into the corresponding garbage bin; The main control module is also used to start the transmission module to continue transmitting the garbage to be identified after the two-dimensional pan-tilt tray is placed; The communication module is used to transmit the data and positioning information processed by the main control module to the cloud platform.

2. The device according to claim 1, characterized in that The device also includes a compression module, which is used to compress recyclable garbage. The compression module includes a telescopic motor, a compression partition and a photocoupler isolation relay module. When the servo is turned to put the garbage into the recyclable garbage bin, the telescopic motor pushes the compression partition to compress the recyclable garbage under the control of the photocoupler isolation relay module.

3. The device according to claim 1, characterized in that The device also includes a display module, which is used to display the detection image and data related to garbage classification.

4. The device according to claim 3, characterized in that The device also includes a full load detection module, which is used to detect whether the capacity of each garbage bin reaches a set garbage capacity threshold, and send the detection information to the main control module. When the main control module receives a signal that the garbage bin capacity reaches the set garbage bin capacity threshold, it sends information showing that the corresponding garbage bin is full to the display module and sends the full load information to the cloud platform.

5. The device according to claim 4, characterized in that The device also includes a solar power supply module, which is used to convert solar energy into electrical energy to power the intelligent garbage recycling and sorting device.

6. The device according to claim 1, characterized in that The main control module includes a model building unit, which is used to build a garbage classification and recognition model using the YOLOv8 target detection algorithm. The garbage classification and recognition model includes a backbone feature network, a feature extraction network and a prediction network. The backbone feature network extracts preliminary features of each type of garbage through a convolutional layer, the feature extraction network is used to extract effective features from the preliminary features, and the prediction network is used to predict and output classification results based on the effective features.

7. The device according to claim 6, characterized in that The main control module also includes an image recognition unit, which uses a garbage classification recognition model to perform garbage classification recognition on the collected image information to identify different types of garbage.

8. The device according to claim 7, characterized in that The main control module also includes a recognition result processing unit, which processes the recognition result by combining a sliding window mechanism with an exponential sliding average to obtain a final classification result.

9. A control method for an intelligent garbage recycling and sorting device, applicable to the intelligent garbage recycling and sorting device according to any one of claims 1 to 8, characterized in that: The following steps are involved: The conveying module conveys the garbage to be identified when it detects that there is garbage on the conveyor belt, disperses the garbage during the transmission process, and stops working in the image acquisition area; The image acquisition module acquires image information of the garbage to be identified and transmits the image information to the main control module; After receiving the image information, the main control module performs garbage recognition, obtains the recognition result, and sends a stop command to the transmission module. According to the recognition result, the main control module analyzes and controls the rotation angle of the rotating steering gear and the rotation direction of the rotatable baffle, and sends a control command to the rotating steering gear; The servo drives control the rotation direction of the rotatable baffle according to the control command, and puts the garbage into the two-dimensional pan-tilt tray or the recyclable garbage bin; The two-dimensional pan-tilt tray rotates the tray according to the recognition result to put the garbage into the corresponding trash can; After the two-dimensional pan-tilt tray is placed, the main control module starts the transmission module to continue transmitting the garbage to be identified.

Images