A garbage classification visual recognition method and system based on an improved YOLOv5s model

Abstract

This invention discloses a visual recognition method and system for waste sorting based on an improved YOLOv5s model, relating to the fields of intelligent waste sorting and machine vision target detection. It aims to solve the problems of existing YOLOv5s models when deployed on embedded devices, such as the difficulty in balancing lightweight design and recognition accuracy, poor waste target recognition performance in complex scenes, and high detection latency. This invention replaces the original YOLOv5s backbone network with a lightweight MobileNetV3 network, embeds a parallel hybrid domain ESA attention mechanism module at key nodes of the backbone network, optimizes bounding box regression using the CIoU loss function, and achieves waste sorting recognition in conjunction with a triggered image stream detection mode. This invention significantly improves recognition accuracy and inference speed while reducing model computation, enabling real-time, high-precision waste sorting recognition on low-cost embedded devices and adapting to the application of intelligent waste sorting equipment.

Description

Technical Field

[0001] This invention relates to the field of intelligent waste sorting and machine vision target detection technology, and in particular to a waste sorting visual recognition method and system based on an improved YOLOv5s model. Background Technology

[0002] With the acceleration of urbanization in my country, the amount of household waste generated continues to rise. Traditional manual waste sorting suffers from pain points such as low efficiency, high cost, and poor working environment. Intelligent waste sorting equipment based on deep learning has become the core development direction in the field of solid waste treatment.

[0003] Currently, the YOLO series of object detection algorithms are the mainstream solution for waste sorting and identification, but existing technologies have significant shortcomings in practical applications. On the one hand, the standard YOLOv5s model has a large number of parameters and computational load, making it difficult to achieve real-time and efficient deployment on low-cost embedded development boards such as Jetson Nano; on the other hand, existing lightweight modification solutions generally sacrifice recognition accuracy, resulting in serious false detection and false negative problems for waste targets with large differences in shape, overlapping placement, and small size, and insufficient accuracy in identifying easily confused categories.

[0004] On the other hand, the serial attention mechanism used in existing models is prone to feature loss and overfitting, resulting in weak generalization ability; traditional video stream detection modes are prone to frame drops and delays, and single-item garbage identification takes a long time, which cannot meet the real-time requirements of cruise sorting equipment; at the same time, the general garbage classification dataset has incomplete category coverage and insufficient annotation accuracy, which further limits the actual implementation performance of the model. Summary of the Invention

[0005] To address these issues, this invention provides a visual recognition method and system for waste sorting based on an improved YOLOv5s model. This system solves several technical problems encountered when deploying the YOLOv5s model in embedded systems for intelligent waste sorting scenarios. These problems include high parameter and computational load, making efficient real-time operation on low-cost edge devices; insufficient feature extraction capabilities after lightweight modifications leading to low accuracy and recall for small, overlapping, and easily confused waste targets; weak generalization ability; and high latency and insufficient sorting efficiency in traditional video stream detection modes.

[0006] To address the aforementioned technical problems, this invention provides a visual recognition method for waste sorting based on an improved YOLOv5s model. The method includes the steps of constructing a target detection model, training the target detection model, and using the trained target detection model to identify the waste image to be detected. The target detection model is an improved YOLOv5s model, which includes a backbone network, a neck network, and a prediction head connected in sequence. The backbone network adopts the MobileNetV3 lightweight network, replacing the CSPDarknet53 backbone network of the original YOLOv5s model; the backbone network includes sequentially cascaded initial convolutional modules and multiple sets of MobileNetV3_block feature extraction modules, and parallel hybrid domain ESA attention mechanism modules are embedded in the mid-level feature output end and the deep feature output end of the backbone network respectively. The neck network adopts an FPN+PAN bidirectional feature fusion architecture, which receives three sets of feature maps of different scales output by the backbone network to complete the bidirectional fusion of deep semantic features and shallow detail features. The prediction head adopts a three-branch multi-scale detection architecture, which receives three sets of fused feature maps output by the neck network, and outputs the classification result, confidence score and bounding box position information of the garbage to be detected.

[0007] Preferably, the MobileNetV3_block feature extraction module is an inverted residual structure, which is arranged sequentially along the feature transmission direction with a 1×1 up-dimensional convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 down-dimensional convolutional layer; the inverted residual structure uses the Hard-Swish function as the non-linear activation function, and sets the shortcut direct connection branch only when the number of input and output channels is the same and the stride is 1.

[0008] Preferably, the parallel hybrid domain ESA attention mechanism module includes a channel attention branch (CAM) and a spatial attention branch (SAM) configured in parallel. The channel attention branch CAM uses global average pooling and global max pooling in parallel to extract channel features, captures local cross-channel features through one-dimensional convolution, does not perform channel dimensionality reduction processing throughout the process, and finally generates channel attention weights through the Sigmoid function. The spatial attention branch SAM performs global max pooling and global average pooling on the input feature map along the channel dimension, and then concatenates the two pooling results to generate spatial attention weights through convolution. The input feature map is simultaneously input to the channel attention branch (CAM) and the spatial attention branch (SAM) to complete the weight calculation. The outputs of the two branches are then multiplied element-wise with the original input feature map, and finally fused at the residual level to obtain the output feature map.

[0009] Preferably, the FPN+PAN bidirectional feature fusion architecture of the neck network includes a top-down upsampling fusion branch and a bottom-up downsampling fusion branch; The upsampling fusion branch performs convolutional dimensionality reduction and upsampling on the 20×20 deep features output by the backbone network, and then concatenates and fuses them with the 40×40 mid-level features output by the backbone network. The fused features are then enhanced by the C3 module and subjected to convolutional dimensionality reduction and upsampling again. They are then concatenated and fused with the 80×80 shallow features output by the backbone network, and finally enhanced by the C3 module before being output to the prediction head. The downsampling fusion branch performs convolutional downsampling on the 80×80 features output by the upsampling fusion branch, then concatenates and fuses them with the 40×40 intermediate features output by the upsampling fusion branch. After enhancement by the C3 module, the fusion is output to the prediction head. Simultaneously, it performs convolutional downsampling again, concatenates and fuses them with the 20×20 deep features output by the backbone network, and after enhancement by the C3 module, the fusion is output to the prediction head.

[0010] Preferably, the improved YOLOv5s model uses the CIoU loss function as the bounding box regression loss function. The CIoU loss function comprehensively considers three geometric parameters: the overlap area between the predicted box and the ground truth box, the Euclidean distance between the center points, and the consistency of the aspect ratio. Its expression is as follows: ; in, Let CIoU be the loss function. The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box is the Euclidean distance. The length of the diagonal of the smallest closure region containing two boxes. For weight adjustment parameters, This is a parameter for aspect ratio consistency.

[0011] Preferably, the input image resolution of the improved YOLOv5s model is set to 640×640; when identifying the garbage to be detected, a triggered multi-frame image acquisition method is adopted to continuously acquire a set number of static images of the garbage to be detected, and reasoning is performed on each frame image separately. The result with the highest confidence among the multi-frame recognition results is taken as the final classification result.

[0012] Preferably, the improved YOLOv5s model is trained based on a self-made EWID waste classification dataset; the EWID dataset covers four major categories of household waste, including recyclables, kitchen waste, hazardous waste, and other waste, totaling 18 subcategories, and contains 2,647 labeled training set images and 302 labeled test set images.

[0013] This invention also provides a visual recognition system for waste sorting based on an improved YOLOv5s model, including an image acquisition module, an embedded inference module, a serial communication module, and an improved YOLOv5s model stored in the embedded inference module. The image acquisition module is used to acquire static images of the waste to be detected and transmit them to the embedded inference module; The embedded inference module is used to run the improved YOLOv5s model to complete image recognition and output garbage classification results. The improved YOLOv5s model includes a backbone network, a neck network, and a prediction head connected in sequence. The backbone network adopts a lightweight MobileNetV3 network, and parallel hybrid domain ESA attention mechanism modules are embedded in the middle and deep feature output ends of the backbone network, respectively. The neck network adopts an FPN+PAN bidirectional feature fusion architecture. The prediction head adopts a three-branch multi-scale detection architecture. The serial communication module is used to receive trigger signals from the lower-level machine and output the corresponding number signal for the garbage sorting result.

[0014] Preferably, the embedded inference module is equipped with a Jetson Nano development board, the image acquisition module uses a USB industrial camera, the USB industrial camera is connected to the Jetson Nano development board via signal, and the serial communication module uses a TTL serial port to communicate with the lower-level sorting actuator.

[0015] Preferably, the serial communication module works in conjunction with the lower-level sorting execution mechanism: when the waste to be detected falls onto the sorting rudder, the serial communication module receives the arrival trigger signal sent by the lower-level machine, triggers the image acquisition module to acquire the image and the embedded inference module to complete the recognition, and sends the corresponding waste classification number signal to the lower-level machine after the recognition is completed. If no valid target is recognized, the set number is sent.

[0016] As can be seen from the above technical solutions, this invention application has the following beneficial effects: First, it significantly improves the adaptability of lightweight deployment and reduces the cost of application implementation. This invention replaces the original YOLOv5s CSPDarknet53 backbone network with the MobileNetV3 lightweight network. Through the combined design of inverted residual structure, depthwise separable convolution and Hard-Swish activation function, the model size is greatly compressed while retaining the core feature extraction capability. The model's GFLOPs are reduced to 7.5, the number of parameters is only about 3.1 million, and the inference FPS is increased to 229.53. It can achieve smooth real-time inference on low-cost embedded development boards such as Jetson Nano, solving the pain points of traditional models with large number of parameters and difficulty in deployment at the edge, and significantly reducing the hardware cost and deployment threshold of intelligent waste sorting equipment.

[0017] Secondly, it significantly improves recognition accuracy and model generalization ability in complex scenarios. This invention embeds a parallel hybrid domain ESA attention mechanism module into key nodes of the backbone network, simultaneously calibrating feature weights in the channel and spatial dimensions. This avoids the feature loss problem of traditional serial attention mechanisms. Combined with CIoU loss function to optimize bounding box regression, it effectively solves the problems of false detection and false negative detection of small-sized, overlapping, morphologically different, and easily confused garbage targets. The model achieves an F1 score of 0.9947 across all categories and an mAP50 of 0.9950. The false recognition rate of easily confused categories is reduced from 3.2% to 0.8%, and the recall rate of garbage with low sample size is increased to 99.2%. It has extremely strong stability and generalization performance in real garbage classification scenarios.

[0018] Third, the optimized recognition execution mode significantly improves the overall sorting efficiency and engineering practicality. This invention abandons the traditional continuous video stream detection mode and adopts a trigger-based multi-frame image stream recognition scheme. It only acquires images and completes inference when a garbage arrival trigger signal is received. The recognition time for a single piece of garbage is reduced from 5-6 seconds in the traditional scheme to 2-3 seconds, doubling the recognition efficiency. At the same time, it avoids the problems of frame drops, latency, and excessive power consumption in video stream detection. With the serial port linkage mechanism, it can realize the fully automated linkage of automatic triggering, recognition, and classification signal output for garbage arrival. The full-process recognition time for a single piece of garbage is less than 1 second, and the recognition success rate is over 99%. It is perfectly adapted to the real-time sorting requirements of cruise-type garbage sorting machines and has extremely strong engineering application value. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Referring to the drawings will make the features and advantages of the present invention clearer. The drawings are illustrative and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 A schematic diagram of the improved YOLOv5s model; Figure 2 This is a schematic diagram of the inverted residual structure; Figure 3 This is a schematic diagram of the SE channel attention layer; Figure 4 This is a schematic diagram of the parallel hybrid domain ESA attention mechanism module; Figure 5 This is a schematic diagram of the channel attention branch CAM structure; Figure 6 This is a schematic diagram of the spatial attention branch (SAM). Figure 7This is a schematic diagram of the F1-confidence curve of the original YOLOv5s model. Figure 8 A schematic diagram of the F1-confidence curve for improving the YOLOv5s model; Figure 9 A schematic diagram of the confusion matrix of the original YOLOv5s model; Figure 10 A schematic diagram of the confusion matrix for improving the YOLOv5s model; Figure 11 This is a schematic diagram of the loss curve of the original YOLOv5s model; Figure 12 A schematic diagram of the loss curve for improving the YOLOv5s model; Figure 13 A block diagram of a waste sorting visual recognition system based on an improved YOLOv5s model provided by the present invention; Figure 14 A schematic diagram of a real-world test case to improve the YOLOv5s model for waste sorting. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides a visual recognition method and system for waste sorting based on an improved YOLOv5s model. It addresses the technical problems of existing YOLOv5s models when deployed in embedded devices for intelligent waste sorting scenarios. These problems include high parameter and computational load, making it difficult to run efficiently and in real-time on low-cost edge devices; insufficient feature extraction capabilities after lightweight modification leading to low accuracy and recall in the recognition of small, overlapping, and easily confused waste targets; weak generalization ability; and high latency and insufficient sorting efficiency in traditional video stream detection modes.

[0022] The core of the overall technical solution of this invention lies in the dual optimization of the original YOLOv5s model in terms of lightweighting and feature extraction capabilities. The original backbone network is replaced with a lightweight MobileNetV3 network, and a parallel hybrid domain ESA attention mechanism module is embedded in the key nodes of the backbone network. Combined with CIoU loss function optimization and trigger-based image stream recognition mode, the model can be deployed with high precision and low latency on the Jetson Nano embedded development board, which is suitable for the actual application needs of cruise-type garbage sorting machines.

[0023] Example 1: This embodiment provides a visual recognition method for garbage classification based on an improved YOLOv5s model, including the core steps of constructing a target detection model, training the target detection model, and recognizing the garbage image to be detected using the trained target detection model. The following is a detailed description of each step.

[0024] I. Improved YOLOv5s model construction The target detection model is an improved YOLOv5s model, see [link / reference] Figure 1 The improved YOLOv5s model includes a backbone network, a neck network, and a prediction head connected in sequence. It adopts the three-segment architecture of the YOLO series of object detection algorithms, while structural optimizations are made for garbage classification scenarios and embedded deployment requirements.

[0025] 1. Backbone Network Design The backbone network adopts the MobileNetV3 lightweight network, replacing the CSPDarknet53 backbone network of the original YOLOv5s model. The backbone network includes cascaded initial convolutional modules and multiple sets of MobileNetV3_block feature extraction modules. Parallel hybrid domain ESA attention mechanism modules are embedded in the mid-level feature output and deep feature output of the backbone network, respectively.

[0026] In this embodiment, the initial convolutional module is the conv_bn_hswish module, which integrates 2D convolutional layers, batch normalization (BN) layers, and Hard-Swish nonlinear activation functions. The model input is a 640×640×3 RGB three-channel image to be detected. After the initial feature extraction and dimension transformation are completed by the initial convolutional module, the input image is input to the subsequent cascaded MobileNetV3_block feature extraction module.

[0027] The MobileNetV3_block feature extraction module is an inverted residual structure, see [link / reference]. Figure 2 Along the feature transfer direction, a 1×1 up-dimensional convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 down-dimensional convolutional layer are sequentially arranged. The inverse residual structure uses the Hard-Swish function as the non-linear activation function, and only sets the shortcut direct connection branch when the number of input and output channels are the same and the stride is 1. This inverse residual structure adopts the operation logic of "first widening, then processing, then narrowing". First, the number of channels is increased through the 1×1 up-dimensional convolutional layer. In a richer feature space, the 3×3 depthwise separable convolution is used to complete feature extraction. Finally, the number of channels is compressed through the 1×1 down-dimensional convolutional layer. More effective features are retained in the depth non-linear transformation, while the depthwise separable convolution significantly reduces the number of model parameters and computation.

[0028] The SE channel attention layer is referred to in [reference]. Figure 3 The algorithm first performs global average pooling on the features of each channel to obtain the channel description vector. Then, it learns the importance weights of each channel through two fully connected layers. The number of nodes in the first fully connected layer is equal to 1 / 4 of the number of input channels, and the number of nodes in the second fully connected layer is consistent with the number of input channels. The second fully connected layer uses the Hard-Sigmoid activation function. Finally, the learned weights are used to recalibrate the original feature map, enhancing important feature channels and suppressing unimportant feature channels to improve the model's ability to perceive key features.

[0029] In this embodiment, MobileNetV3 uses the Hard-Swish activation function to replace the traditional ReLU6 activation function and the Swish activation function. The expressions for each activation function are as follows: Sigmoid activation function: ; Swish activation function: ; ReLU6 activation function: ; h-sigmoid activation function: ; Hard-Swish activation function: .

[0030] The Hard-Swish activation function maintains nonlinear expressive power while simplifying computation and differentiation, making it more quantization-friendly and significantly improving model inference speed on embedded devices.

[0031] In this embodiment, the network configuration of MobileNetV3 is shown in Table 1 below. Here, SE indicates whether the attention mechanism is used, NL indicates the non-linear activation function currently used, the value following Bneck is the kernel size of the DW convolution, and NBN indicates that the convolution of the classifier part does not use the BN layer.

[0032] Table 1 Network Configuration of MobileNetV3

[0033] Based on the above configuration, the image feature processing flow of MobileNetV3 is as follows: After the image is input, it first undergoes a preprocessing step of normalization and size adjustment to adapt it to the network input requirements; then, feature extraction is completed through multiple cascaded MobileNetV3_block feature extraction modules. During the feature extraction process, the feature map is enhanced through the SE channel attention layer to strengthen the perception of key features; in the final stage of feature extraction, the spatial dimension of the feature map is reduced to a fixed size through global average pooling, and finally, multiple sets of feature maps of different scales are output to the neck network.

[0034] By replacing the MobileNetV3 lightweight backbone network as described above, the number of model parameters and computational cost are significantly reduced. The performance comparison of different models is shown in Table 2 below.

[0035] Table 2 Performance Comparison of Different Models

[0036] To compensate for the loss of feature extraction capability caused by the lightweight modification, this embodiment embeds parallel hybrid domain ESA attention mechanism modules at the mid-layer feature output end and the deep-layer feature output end of the backbone network, respectively. The structure of the parallel hybrid domain ESA attention mechanism module is described in [reference needed]. Figure 4 Its embedding position is right after the output of the corresponding MobileNetV3_block feature extraction module in the backbone network. After the backbone network completes the initial feature extraction, it immediately performs spatial focusing and channel calibration on the feature map, filters background noise earlier, and significantly improves the recognition accuracy of small garbage and low-contrast targets.

[0037] The parallel hybrid domain ESA attention mechanism module includes a parallel channel attention branch (CAM) and a spatial attention branch (SAM). Channel and spatial attention are given the same priority, avoiding feature loss and overfitting problems caused by traditional serial attention mechanisms. The input feature map is simultaneously input to both the channel attention branch (CAM) and the spatial attention branch (SAM) for weight calculation. The outputs of the two branches are then element-wise multiplied with the original input feature map, followed by residual-level fusion to obtain the output feature map. The fusion formula is as follows: ; The structure of the channel attention branch CAM is shown below. Figure 5This method employs parallel global average pooling and global max pooling to extract channel features, capturing local cross-channel features through one-dimensional convolution without any channel dimensionality reduction. Finally, channel attention weights are generated using the Sigmoid function. The specific processing flow is as follows: For the input W×H×C feature map, parallel global max pooling and global average pooling are used to compress the feature map into a 1×1×C channel feature vector; the results of both pooling operations are input into the same one-dimensional convolutional layer to capture local cross-channel feature information, avoiding the channel attention reduction problem caused by dimensionality reduction; the extracted local cross-channel information is summed element-wise and superimposed, then non-linearly transformed using the Sigmoid function to generate a channel weight vector. This vector is multiplied by the input feature map to strengthen channels containing core information, completing channel attention calibration.

[0038] In this embodiment, the kernel size of the one-dimensional convolution is... Through adaptive The nearest neighbor method is used to determine the method, and the calculation formula is as follows: ; in, Indicates distance The most recent odd number, , , The number of channels in the input feature map.

[0039] The structure of the spatial attention branch (SAM) is described in [reference]. Figure 6 The input feature map undergoes global max pooling and global average pooling along the channel dimension. The results of the two pooling operations are concatenated and then convolutional to generate spatial attention weights. Specifically, the process involves: performing global max pooling and global average pooling along the channel dimension on the input W×H×C feature map to extract pixel information at the same location, resulting in two W×H×1 single-channel feature maps; merging these two single-channel feature maps into a W×H×2 feature map through a stacking operation; performing feature fusion on this feature map through a convolutional layer to transform it into a W×H×1 feature map; and finally, performing a non-linear transformation using the Sigmoid activation function to generate spatial attention weights, thus completing the feature redistribution in the spatial domain.

[0040] In this embodiment, the internal processing of the parallel hybrid domain ESA attention mechanism module is divided into four key steps: ① Channel compression: using a 1×1 convolutional layer to compress the input feature map. Perform channel dimensionality reduction, reducing the number of channels from Compress to , for Alternatively, the value can be fixed at 16 to reduce the number of parameters and computational complexity in subsequent operations; ② Multi-scale spatial focusing: Dependencies of different ranges are captured through two parallel processing paths. The downsampling layer uses a 7×7 large kernel for stride downsampling to obtain the global view, and the refined convolutional group captures local textures through multiple sets of 3×3 convolutions; ③ Spatial restoration and feature alignment: The downsampled low-resolution feature map is restored to the original H×W size through bilinear interpolation, and the restored features are fused across channels through 1×1 convolution; ④ Adaptive weight mapping: The Sigmoid activation function is used for nonlinear transformation to generate a spatial attention mask. The weights are applied to the input feature map through element-wise multiplication, and the final output is completed with the help of shortcut connections.

[0041] 2. Neck network design The neck network adopts an FPN+PAN bidirectional feature fusion architecture, which receives three sets of feature maps of different scales (80×80, 40×40, and 20×20) output by the backbone network, and completes the bidirectional fusion of deep semantic features and shallow detail features to solve the feature loss problem of small-sized, overlapping garbage targets.

[0042] The FPN+PAN bidirectional feature fusion architecture of the neck network includes a top-down upsampling fusion branch and a bottom-up downsampling fusion branch: The upsampling fusion branch performs convolutional dimensionality reduction and upsampling on the 20×20 deep features output by the backbone network, and then merges them with the 40×40 mid-level features output by the backbone network through the Concat concatenation module to achieve channel dimension fusion. After fusion, the features are enhanced by the C3 module and then subjected to convolutional dimensionality reduction and upsampling again. They are then merged with the 80×80 shallow features output by the backbone network through Concat concatenation, enhanced by the C3 module, and output to the prediction head. The downsampling fusion branch performs convolutional downsampling on the 80×80 features output by the upsampling fusion branch, then concatenates and fuses them with the 40×40 intermediate features output by the upsampling fusion branch. After enhancement by the C3 module, the fusion is output to the prediction head. Simultaneously, it performs convolutional downsampling again, concatenates and fuses it with the 20×20 deep features output by the backbone network, then enhances it by the C3 module and outputs it to the prediction head.

[0043] 3. Prediction Head Design The prediction head adopts a three-branch multi-scale detection architecture, receiving three sets of fused feature maps output from the neck network, and outputting the classification result, confidence score, and bounding box location information of the garbage to be detected, specifically including: P3 detection branch: Input shallow features of size 80×80, corresponding to the detection convolution (Conv Detect) module, responsible for detecting small-sized garbage targets; P4 detection branch: Inputting mid-layer features of size 40×40, corresponding to the detection convolution (Conv Detect) module, responsible for detecting mid-sized garbage targets; P5 detection branch: Input deep features of size 20×20, corresponding to the detection convolution (Conv Detect) module, responsible for detecting large-sized garbage targets.

[0044] Each detection branch completes the target classification confidence calculation and bounding box coordinate regression through convolution operation, and finally outputs the category, confidence and location information of the garbage target, realizing multi-scale accurate detection of household waste of different sizes and types.

[0045] 4. Loss Function Optimization The improved YOLOv5s model adopts the CIoU loss function as the bounding box regression loss function. The CIoU loss function comprehensively considers three geometric parameters: the overlap area between the predicted and ground truth boxes, the Euclidean distance between the center points, and the consistency of aspect ratio. This addresses the shortcomings of traditional IoU loss, such as gradient vanishing when boxes do not overlap and slow convergence speed of GIoU loss. Its expression is: ; in, Let CIoU be the loss function. The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box is the Euclidean distance. The length of the diagonal of the smallest closure region containing two boxes. For weight adjustment parameters, This is a parameter for aspect ratio consistency.

[0046] Aspect Ratio Consistency Parameters The expression is: ; in, , For the width and height of the actual bounding box, and The width and height of the prediction box; weight adjustment parameters. exist The time-attenuated effect causes the network to prioritize fitting the center distance in the early stages of training.

[0047] By applying the CIoU loss function, the optimized model can achieve more accurate and stable bounding box regression when dealing with garbage targets that have overlaps, deformations, and dense arrangements.

[0048] II. Improving the Training of YOLOv5s Models In this embodiment, the improved YOLOv5s model is trained based on a self-made EWID garbage classification dataset. The specific training process is as follows: 1. Dataset Construction: The EWID dataset is a waste classification dataset that was independently collected, photographed, and labeled. It covers four major categories of waste commonly found in daily life: recyclables, kitchen waste, hazardous waste, and other waste, comprising 18 subcategories. Image data collection and target location labeling were completed for all categories of waste. The EWID dataset contains 2647 training images and 302 test images, which can adapt to the actual needs of household waste classification scenarios and improve the accuracy and generalization ability of model recognition and classification.

[0049] 2. Training parameter settings: The resolution of the input image to the model is set to 640×640 to ensure image clarity while achieving fast inference speed; the confidence threshold is set to 0.70 to ensure high recognition accuracy; the IOU threshold is set to 0.65 to retain more overlapping bounding boxes, adapting to the scenario of detecting overlapping garbage objects; the training iteration rounds are set to 100 rounds. After completing the iterative training, the training results are repeatedly tested and optimized to finally obtain the model weights adapted to the garbage classification scenario.

[0050] 3. Training Performance Validation: After training, the model performance is evaluated using the F1-confidence curve, confusion matrix, and loss curve. See [link to relevant documentation]. Figure 7 , Figure 9 , Figure 11 The original YOLOv5s model achieved an F1 score of 0.99 with a confidence level of 0.65 across all categories, and achieved a 100% validation rate for identifying over 95% of the garbage types in the dataset; see [link / reference]. Figure 8 , Figure 10 , Figure 12 The improved YOLOv5s model consistently achieved F1 scores above 0.95 for all 17 waste categories, with an average F1 score of 1.00 across all categories at a confidence level of 0.797. The accuracy in identifying easily confused waste was significantly improved, with the false positive rate for plastic and radish slices decreasing from 3.2% to 0.8%. The recall rate for waste with low sample sizes increased from 97.3% to 99.2%. During training, the improved model exhibited faster convergence in bounding box loss, category loss, and distribution focus loss, lower validation set loss, and better model training stability and fitting performance, with less susceptibility to overfitting.

[0051] The improved model's inference performance is shown in Table 3 below.

[0052] Table 3. Inference performance of the improved model

[0053] Through the above optimizations, the computational load of the model is reduced by 8.5% compared to the original model, the inference time is shortened from 0.0051s to 0.004357s, and the inference FPS is increased to 229.53. This fully meets the design requirement that the machine's single garbage identification latency is ≤0.5 seconds, the identification success rate for different types of garbage is over 99%, the identification time for a single piece of garbage is less than 1 second, and it can still maintain a smooth response in batch garbage processing scenarios.

[0054] III. Steps for Recognizing the Garbage Image to be Detected When the improved YOLOv5s model, after training, identifies the garbage to be detected, it adopts a triggered image stream acquisition method, abandoning the traditional video stream detection mode, thus avoiding frame drops and latency issues. The specific steps are as follows: 1. Trigger signal reception: The system receives the trigger signal from the lower-level machine when the waste to be detected is in place via the serial communication module. When the waste to be detected falls onto the sorting rudder, the lower-level machine immediately sends a trigger signal to the execution end of this method. 2. Triggered image acquisition: Upon receiving a trigger signal, five static images of the waste to be inspected are immediately and continuously acquired through an industrial camera. After image acquisition is completed, the images are transmitted to the embedded inference module. 3. Model Inference and Recognition: The trained improved YOLOv5s model is called to detect and recognize each of the collected static images, and the classification results, confidence scores and location information of the garbage targets in each image are obtained. 4. Result Mapping and Output: The result with the highest confidence among the 5 image recognition results is taken as the final recognition result. The final recognition result is mapped to the four preset categories of waste: recyclables, kitchen waste, hazardous waste, and other waste. The corresponding category numbers 1 to 4 are sent to the lower-level machine. If no valid target is recognized, number 0 is sent. At the same time, the recognition result is saved in a txt document for subsequent data statistics and model optimization.

[0055] By using the aforementioned trigger-based image stream recognition method, the recognition time for a single piece of waste is reduced from an average of 5-6 seconds in the traditional video stream mode to an average of 2-3 seconds, doubling the recognition efficiency and significantly improving the overall operating efficiency of waste sorting equipment.

[0056] Example 2: This embodiment provides a waste sorting visual recognition system based on an improved YOLOv5s model. This system is used to implement the waste sorting visual recognition method described in Embodiment 1. (See also...) Figure 13 The system includes an image acquisition module, an embedded inference module, a serial communication module, and an improved YOLOv5s model stored in the embedded inference module.

[0057] The image acquisition module is used to acquire static images of the waste to be detected and transmit them to the embedded inference module. In this embodiment, the image acquisition module uses a USB industrial camera, which is signal-connected to the embedded inference module and installed directly above the sorting rudder. The camera's field of view covers the entire sorting rudder, enabling it to clearly capture images of the waste to be detected that has fallen onto the rudder.

[0058] The embedded inference module is used to run the improved YOLOv5s model to complete image recognition and output garbage classification results. The structure and training method of the improved YOLOv5s model are completely consistent with those in Embodiment 1, and will not be repeated here. In this embodiment, the embedded inference module is equipped with a Jetson Nano development board, which can efficiently handle complex image processing and deep learning tasks, adapt to the real-time inference requirements of the improved lightweight model, and realize efficient deployment of low-cost embedded terminals.

[0059] The serial communication module is used to receive the trigger signal from the lower-level sorting execution mechanism and to output the number signal corresponding to the garbage classification result to the lower-level sorting execution mechanism. In this embodiment, the serial communication module uses a TTL serial port to communicate with the lower-level sorting execution mechanism to realize linkage and cooperation with the lower-level machine.

[0060] In this embodiment, the linkage workflow between the serial communication module and the lower-level sorting execution mechanism is as follows: When the waste to be detected falls onto the sorting rudder, the lower-level sorting execution mechanism immediately sends a position trigger signal to the serial communication module; after receiving the trigger signal, the serial communication module synchronously triggers the image acquisition module to acquire an image of the waste to be detected, and triggers the embedded inference module to start model inference; after the embedded inference module completes the identification and obtains the final waste classification result, it sends the corresponding waste classification number signal to the lower-level sorting execution mechanism through the serial communication module, and sends the set number 0 when no valid target is identified; after receiving the number signal, the lower-level sorting execution mechanism drives the rudder to rotate to the corresponding classification area to complete the automatic sorting of waste.

[0061] The system in this embodiment, through improvements to the lightweight and high-precision optimization of the YOLOv5s model, combined with trigger-based image stream acquisition and serial port linkage control, achieves a complete process of automatic identification, automatic transmission of category information, and automatic sorting after waste falls onto the steering wheel. It boasts high identification accuracy, fast response speed, and low deployment cost, demonstrating strong practical value and application capability. Figure 14 This is a real-world example of how the model can identify various types of waste.

[0062] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A visual recognition method for waste sorting based on an improved YOLOv5s model, comprising the steps of constructing an object detection model, training the object detection model, and recognizing the waste image to be detected using the trained object detection model, characterized in that, The target detection model is an improved YOLOv5s model, which includes a backbone network, a neck network, and a prediction head connected in sequence. The backbone network adopts the MobileNetV3 lightweight network, replacing the CSPDarknet53 backbone network of the original YOLOv5s model; the backbone network includes sequentially cascaded initial convolutional modules and multiple sets of MobileNetV3_block feature extraction modules, and parallel hybrid domain ESA attention mechanism modules are embedded in the mid-level feature output end and the deep feature output end of the backbone network respectively. The neck network adopts an FPN+PAN bidirectional feature fusion architecture, which receives three sets of feature maps of different scales output by the backbone network to complete the bidirectional fusion of deep semantic features and shallow detail features. The prediction head adopts a three-branch multi-scale detection architecture, which receives three sets of fused feature maps output by the neck network, and outputs the classification result, confidence score and bounding box position information of the garbage to be detected.

2. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The MobileNetV3_block feature extraction module is an inverted residual structure, which consists of a 1×1 up-dimensional convolutional layer, a 3×3 depthwise separable convolutional layer, an SE channel attention layer, and a 1×1 down-dimensional convolutional layer along the feature transmission direction. The inverted residual structure uses the Hard-Swish function as a non-linear activation function, and a shortcut direct connection branch is set only when the number of input and output channels is the same and the stride is 1.

3. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The parallel hybrid domain ESA attention mechanism module includes a parallel channel attention branch (CAM) and a spatial attention branch (SAM). The channel attention branch CAM uses global average pooling and global max pooling in parallel to extract channel features, captures local cross-channel features through one-dimensional convolution, does not perform channel dimensionality reduction processing throughout the process, and finally generates channel attention weights through the Sigmoid function. The spatial attention branch SAM performs global max pooling and global average pooling on the input feature map along the channel dimension, and then concatenates the two pooling results to generate spatial attention weights through convolution. The input feature map is simultaneously input to the channel attention branch (CAM) and the spatial attention branch (SAM) to complete the weight calculation. The outputs of the two branches are then multiplied element-wise with the original input feature map, and finally fused at the residual level to obtain the output feature map.

4. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The FPN+PAN bidirectional feature fusion architecture of the neck network includes a top-down upsampling fusion branch and a bottom-up downsampling fusion branch; The upsampling fusion branch performs convolutional dimensionality reduction and upsampling on the 20×20 deep features output by the backbone network, and then concatenates and fuses them with the 40×40 mid-level features output by the backbone network. The fused features are then enhanced by the C3 module and subjected to convolutional dimensionality reduction and upsampling again. They are then concatenated and fused with the 80×80 shallow features output by the backbone network, and finally enhanced by the C3 module before being output to the prediction head. The downsampling fusion branch performs convolutional downsampling on the 80×80 features output by the upsampling fusion branch, then concatenates and fuses them with the 40×40 intermediate features output by the upsampling fusion branch. After enhancement by the C3 module, the fusion is output to the prediction head. Simultaneously, it performs convolutional downsampling again, concatenates and fuses them with the 20×20 deep features output by the backbone network, and after enhancement by the C3 module, the fusion is output to the prediction head.

5. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The improved YOLOv5s model uses the CIoU loss function as the bounding box regression loss function. The CIoU loss function comprehensively considers three geometric parameters: the overlap area between the predicted and ground truth boxes, the Euclidean distance between the center points, and the consistency of aspect ratio. Its expression is: ； in, Let CIoU be the loss function. The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box is the Euclidean distance. The length of the diagonal of the smallest closure region containing two boxes. For weight adjustment parameters, This is a parameter for aspect ratio consistency.

6. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The input image resolution of the improved YOLOv5s model is set to 640×640. When identifying the garbage to be detected, a triggered multi-frame image acquisition method is adopted to continuously acquire a set number of static images of the garbage to be detected. Reasoning is performed on each frame image separately, and the result with the highest confidence among the multi-frame recognition results is taken as the final classification result.

7. The visual recognition method for waste sorting based on the improved YOLOv5s model according to claim 1, characterized in that, The improved YOLOv5s model was trained on a self-made EWID waste classification dataset. The EWID dataset covers four major categories of household waste: recyclables, kitchen waste, hazardous waste, and other waste, totaling 18 subcategories. It includes 2,647 labeled training set images and 302 labeled test set images.

8. A visual recognition system for waste sorting based on an improved YOLOv5s model, characterized in that, It includes an image acquisition module, an embedded inference module, a serial communication module, and an improved YOLOv5s model stored in the embedded inference module; The image acquisition module is used to acquire static images of the waste to be detected and transmit them to the embedded inference module; The embedded inference module is used to run the improved YOLOv5s model to complete image recognition and output garbage classification results. The improved YOLOv5s model includes a backbone network, a neck network, and a prediction head connected in sequence. The backbone network adopts a lightweight MobileNetV3 network, and parallel hybrid domain ESA attention mechanism modules are embedded in the middle and deep feature output ends of the backbone network, respectively. The neck network adopts an FPN+PAN bidirectional feature fusion architecture. The prediction head adopts a three-branch multi-scale detection architecture. The serial communication module is used to receive trigger signals from the lower-level machine and output the corresponding number signal for the garbage sorting result.

9. The waste sorting visual recognition system based on the improved YOLOv5s model according to claim 8, characterized in that, The embedded inference module is equipped with a Jetson Nano development board, the image acquisition module uses a USB industrial camera, the USB industrial camera is connected to the Jetson Nano development board via signal, and the serial communication module uses a TTL serial port to communicate with the lower-level sorting actuator.

10. The waste sorting visual recognition system based on the improved YOLOv5s model according to claim 8, characterized in that, The serial communication module works in conjunction with the lower-level sorting execution mechanism: when the waste to be detected falls onto the sorting rudder, the serial communication module receives the arrival trigger signal sent by the lower-level machine, triggering the image acquisition module to acquire the image and the embedded inference module to complete the recognition. After the recognition is completed, the corresponding waste classification number signal is sent to the lower-level machine. If no valid target is recognized, the set number is sent.

Images