A garbage classification model training method, device and equipment

Abstract

The application discloses a garbage classification model training method, device and equipment. A set proportion of garbage images is selected from garbage images, and image properties of regions where each target is located in the selected garbage images are adjusted to obtain a first data set. The first data set includes garbage images whose image properties are adjusted and garbage images whose image properties are not adjusted in original garbage images. The garbage images have been labeled with categories of the targets and regions where the targets are located. Garbage images including multiple targets that overlap with each other in the first data set are copied, and the copied garbage images are added to the first data set to obtain a second data set. The copied garbage images are labeled with one of the multiple targets that overlap with each other. A target detection model is trained based on the second data set. The application not only improves the efficiency of garbage classification model training, but also further improves the accuracy and detection speed of garbage classification performed by the garbage classification model.

Description

Technical Field

[0001] This application belongs to the field of target detection technology, and in particular relates to a training method, apparatus and equipment for a garbage classification model. Background Technology

[0002] Artificial intelligence algorithms are developing rapidly and have made breakthrough progress in the field of object detection. Object detection models extract feature information from images, classify and locate regions of interest in the images, and finally output the location and category of target objects in the images.

[0003] Currently, when using object detection models for garbage classification, two problems arise. First, the diverse shapes and high overlap of garbage itself make it difficult for models to quickly extract features, resulting in long detection times and high detection difficulty. Second, for images captured in complex scenarios such as changing lighting, complex backgrounds, and object occlusion, existing garbage classification object detection models extract rather messy features, leading to poor detection results. Furthermore, existing garbage classification object detection models typically employ a two-stage detection algorithm. This method uses ordinary convolutional layers and Feature Pyramid Networks (FPNs), which have weak representational capabilities, large semantic differences between layers, and require many parameters for training, resulting in high computational cost and poor detection performance.

[0004] Therefore, there is currently a lack of target detection models that are highly adaptable and efficient for waste sorting scenarios. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a training method, apparatus, and equipment for a waste sorting model, thereby improving the training efficiency and detection effectiveness of the waste sorting model.

[0006] Firstly, this application provides a method for training a waste sorting model, the method comprising:

[0007] A set proportion of garbage images are selected from the garbage images, and the image attributes of each target region in the selected garbage images are adjusted to obtain a first dataset. The first dataset includes garbage images with adjusted image attributes and garbage images in the original garbage images whose image attributes have not been adjusted. The garbage images have been labeled with the target category and the target region.

[0008] Copy the junk image from the first dataset that includes multiple overlapping targets, and add the copied junk image to the first dataset to obtain a second dataset, wherein the copied junk image is labeled with one of the multiple overlapping targets;

[0009] The object detection model is trained based on the second dataset.

[0010] In one possible implementation, the source of the junk images is as follows:

[0011] Images were collected from waste sorting scenarios, obtained from existing image datasets related to waste sorting scenarios, and processed using image generation models.

[0012] The image generation model generates junk images by learning features from junk images obtained from two other sources. The category to which the target in the junk image generated by the image generation model belongs is the category in which the number of targets in the junk images obtained from the other two sources is less than a preset threshold.

[0013] In one possible implementation, training the object detection model based on the second dataset includes:

[0014] Based on the source of the garbage images in the second dataset, different learning rates are set for garbage images from different sources. The learning rate represents the magnitude of parameter adjustment of the target detection model during each weight update.

[0015] The target detection model is trained in multiple rounds based on the second dataset. In each round of training, garbage images from the second dataset are input in batches, and each batch of garbage images comes from the same source. The learning rate of each batch of garbage images input into the target detection model during training is equal to the learning rate set for each batch of garbage images.

[0016] In one possible implementation, adjusting the image attributes of the regions where each target is located in the selected garbage image to obtain the first dataset includes:

[0017] According to a preset image attribute adjustment method, the image attributes of each target region in the selected garbage image are adjusted to obtain a garbage image with adjusted image attributes, wherein the image attributes include at least one of hue, brightness and contrast;

[0018] The selected garbage images are replaced with the garbage images whose image attributes have been adjusted from the original garbage images to obtain the first dataset.

[0019] In one possible implementation, the target detection model includes a feature extraction network, a feature fusion network, and a detection network;

[0020] The feature extraction network is a deep residual network CSPDarknet, and the C3 module in CSPDarknet is replaced with a C2F module, and the standard convolution Conv in CSPDarknet is replaced with a dynamic convolution.

[0021] The feature fusion network is a lightweight asymptotic pyramid network Slim AFPN, and a lightweight convolution GSConv is used to replace the standard convolution Conv as the upsampling component in the Slim AFPN;

[0022] The detection network is an Autoassign Head.

[0023] In one possible implementation, after obtaining the second dataset and before training the object detection model based on the second dataset, the method further includes:

[0024] Determine the categories of the labeled targets in the second dataset that meet the unbalanced conditions. The unbalanced conditions are that the proportion of the unbalanced targets corresponding to the categories that meet the unbalanced conditions in all the labeled targets in the second dataset is less than a preset proportion threshold.

[0025] Copy any garbage image in the second dataset that includes the target to be balanced, and add the copied garbage image copy to the second dataset. Each copied garbage image copy retains only the annotation of the target to be balanced.

[0026] Repeat the above process until there are no categories in the second dataset that meet the balance condition for the labeled targets.

[0027] In one possible implementation, a pre-trained image segmentation model is used to determine the regions where each target is located in the selected garbage image.

[0028] Secondly, this application provides a training device for a waste sorting model, the device comprising:

[0029] The image attribute adjustment module is used to select a set proportion of garbage images from the garbage images and adjust the image attributes of the regions where each target is located in the selected garbage images to obtain a first dataset. The first dataset includes the garbage images with adjusted image attributes and the original garbage images with unadjusted image attributes. The garbage images have been labeled with the category to which the target belongs and the region where the target is located.

[0030] An overlapping target processing module is used to copy a junk image from the first dataset that includes multiple overlapping targets, and add the copied junk image to the first dataset to obtain a second dataset, wherein the copied junk image is labeled with one of the multiple overlapping targets;

[0031] The model training module is used to train the object detection model based on the second dataset.

[0032] In one possible implementation, the source of the junk images is as follows:

[0033] Images were collected from waste sorting scenarios, obtained from existing image datasets related to waste sorting scenarios, and processed using image generation models.

[0034] The image generation model generates junk images by learning features from junk images obtained from two other sources. The category to which the target in the junk image generated by the image generation model belongs is the category in which the number of targets in the junk images obtained from the other two sources is less than a preset threshold.

[0035] In one possible implementation, the model training module is specifically used for:

[0036] Based on the source of the garbage images in the second dataset, different learning rates are set for garbage images from different sources. The learning rate represents the magnitude of parameter adjustment of the target detection model during each weight update.

[0037] The target detection model is trained in multiple rounds based on the second dataset. In each round of training, garbage images from the second dataset are input in batches, and each batch of garbage images comes from the same source. The learning rate of each batch of garbage images input into the target detection model during training is equal to the learning rate set for each batch of garbage images.

[0038] In one possible implementation, the image attribute adjustment module is specifically used for:

[0039] According to a preset image attribute adjustment method, the image attributes of each target region in the selected garbage image are adjusted to obtain a garbage image with adjusted image attributes, wherein the image attributes include at least one of hue, brightness and contrast;

[0040] The selected garbage images are replaced with the garbage images whose image attributes have been adjusted from the original garbage images to obtain the first dataset.

[0041] In one possible implementation, the target detection model includes a feature extraction network, a feature fusion network, and a detection network;

[0042] The feature extraction network is a deep residual network CSPDarknet, and the C3 module in CSPDarknet is replaced with a C2F module, and the standard convolution Conv in CSPDarknet is replaced with a dynamic convolution.

[0043] The feature fusion network is a lightweight asymptotic pyramid network Slim AFPN, and a lightweight convolution GSConv is used to replace the standard convolution Conv as the upsampling component in the Slim AFPN;

[0044] The detection network is an Autoassign Head.

[0045] In one possible implementation, the apparatus further includes a target balancing module. After obtaining the second dataset, and before training the target detection model based on the second dataset, the target balancing module is specifically used for:

[0046] Determine the categories of the labeled targets in the second dataset that meet the unbalanced conditions. The unbalanced conditions are that the proportion of the unbalanced targets corresponding to the categories that meet the unbalanced conditions in all the labeled targets in the second dataset is less than a preset proportion threshold.

[0047] Copy any garbage image in the second dataset that includes the target to be balanced, and add the copied garbage image copy to the second dataset. Each copied garbage image copy retains only the annotation of the target to be balanced.

[0048] Repeat the above process until there are no categories in the second dataset that meet the balance condition for the labeled targets.

[0049] In one possible implementation, the image attribute adjustment module is specifically used for:

[0050] The regions where each target is located in the selected garbage image are determined using a pre-trained image segmentation model.

[0051] Thirdly, embodiments of this application provide an apparatus including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a training method for a garbage classification model as provided in any of the first aspects of this application.

[0052] Fourthly, embodiments of this application also provide a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by the processor of a terminal device, enables the terminal device to execute a training method for a garbage classification model as described in any of the claims in the first aspect of this application.

[0053] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0054] This application provides a training method, apparatus, and device for a waste sorting model. Addressing common issues in waste sorting scenarios such as lighting variations, complex backgrounds, and object occlusion, the method first adjusts the image attributes of some waste images in the original waste image dataset to obtain a first dataset. Then, waste images containing multiple overlapping targets are copied from the first dataset and added to the first dataset to obtain a second dataset. The copied waste images are labeled with one of the overlapping targets. Finally, the target detection model is trained based on the second dataset. This not only improves the efficiency of model training but also further enhances the accuracy and detection speed of the waste sorting model. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0057] Figure 2 A flowchart illustrating a training method for a waste sorting model provided in this application embodiment;

[0058] Figure 3 This is a schematic diagram of the model training process provided in the embodiments of this application;

[0059] Figure 4 This is a schematic diagram of the target detection model architecture provided in the embodiments of this application;

[0060] Figure 5 A schematic diagram of a training device for a waste sorting model provided in an embodiment of this application;

[0061] Figure 6 This is a schematic diagram of a training device for a waste sorting model provided in an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0063] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0064] (1) An object detection model typically consists of three parts: a feature extraction network (Backbone), a feature fusion network (Neck), and a detection network (Head). In the object detection model, the backbone's main task is to extract image features, typically including a standard convolutional module (Conv), a C3 module for extracting and fusing features from different dimensions, and a sppf module for fusing features at different scales to ensure a uniform output feature vector. The neck's task is to further integrate and enhance the features extracted by the backbone, so that the subsequent detection head can better perform object detection.

[0065] (2) The C2F module stands for "Convolution to Fully-connected," and it is typically used to convert the output of a convolutional layer into the input of a fully connected layer. In a convolutional neural network, convolutional layers are used to extract features from the input data, while fully connected layers are used to map these features to the final output. The role of the C2F module is to flatten the output of the convolutional layer and use the flattened feature vector as the input to the fully connected layer. This preserves the spatial relationships between features and reduces the number of parameters in the fully connected layer.

[0066] (3)GSConv (Ghost Shifted Convolution) is a specific convolution operation that divides a standard convolutional layer into multiple smaller convolutional layers. Each smaller convolutional layer is responsible for generating a portion of the output feature map, and then these feature maps are merged to form the final output.

[0067] (4) Dynamic Convolution is a convolution operation that adaptively adjusts the weights of the convolution kernel according to the characteristics of the input feature map during the convolution process. This adjustment is usually achieved through the attention mechanism, which enables the model to assign different importance to different spatial locations or channels, thereby improving the model's adaptability and performance.

[0068] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0069] The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0070] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0071] Artificial intelligence algorithms are developing rapidly and have made breakthrough progress in the field of object detection. Object detection models extract feature information from images, classify and locate regions of interest in the images, and finally output the location and category of target objects in the images.

[0072] Currently, when using object detection models for garbage classification, two problems arise. First, the diverse shapes and high overlap of garbage itself make it difficult for models to quickly extract features, resulting in long detection times and high detection difficulty. Second, for images captured in complex scenarios such as changing lighting, complex backgrounds, and object occlusion, existing garbage classification object detection models extract rather messy features, leading to poor detection results. Furthermore, existing garbage classification object detection models typically employ a two-stage detection algorithm. This method uses ordinary convolutional layers and Feature Pyramid Networks (FPNs), which have weak representational capabilities, large semantic differences between layers, and require many parameters for training, resulting in high computational cost and poor detection performance.

[0073] Therefore, there is currently a lack of target detection models that are highly adaptable and efficient for waste sorting scenarios.

[0074] In view of the above problems, this application provides a training method, apparatus and equipment for a waste sorting model, which relates to the fields of target recognition and data processing. It can be applied to target detection scenarios where waste sorting or other targets are interfered with by factors such as lighting and background or where there is overlap between targets. It is used to address common problems in waste sorting scenarios such as lighting changes, complex backgrounds and object occlusion. Through a series of data processing and enhancement methods, it improves the training efficiency of the waste sorting model and further improves the efficiency of the waste sorting model in target detection.

[0075] In this embodiment, firstly, a first dataset is obtained by adjusting the image attributes of some garbage images in the original garbage images. Then, garbage images containing multiple overlapping targets are copied from the first dataset and added to the first dataset to obtain a second dataset. The copied garbage images are labeled with one of the multiple overlapping targets. Finally, the target detection model is trained based on the second dataset. This not only improves the efficiency of model training but also further enhances the accuracy and detection speed of the garbage classification model.

[0076] The specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0077] It should be noted that the following scenarios are for illustrative purposes only and are not intended to limit the scope of this application. In actual implementation, the technical solutions provided in the embodiments of this application can be flexibly applied according to actual needs.

[0078] like Figure 1 The diagram shown is an application scenario illustration provided by an embodiment of this application. The application scenario includes a waste sorting device 10, a monitoring area 20, and a waste bin 30. The waste sorting device 10 is equipped with a trained waste sorting model.

[0079] In one possible scenario, the waste sorting device 10 is installed directly above the waste bin 30. After the waste sorting device 10 is started, it divides the monitoring area 20 to obtain the target detection area where the waste bin 30 is located. Then, it collects the image of the target detection area where the waste bin 30 is located and outputs the waste sorting result based on the collected image.

[0080] Optionally, the aforementioned waste sorting device 10 can also execute corresponding alarm logic based on the waste sorting results, and the specific collection frequency can be set according to the actual scenario and needs.

[0081] It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the implementation of this application is not limited in any way. Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0082] To further illustrate the technical solutions provided in the embodiments of this application, the training method of the garbage classification model provided in the exemplary embodiments of this application will be described below with reference to the accompanying drawings.

[0083] like Figure 2 The diagram shown is a flowchart of a training method for a garbage classification model provided in an embodiment of this application. The method includes the following steps S201-S203.

[0084] Step S201: Select a set proportion of garbage images from the garbage images, and adjust the image attributes of the regions where each target is located in the selected garbage images to obtain the first dataset;

[0085] The first dataset includes junk images with adjusted image attributes and junk images from the original junk images whose image attributes have not been adjusted. The junk images have been labeled with the category to which the target belongs and the region where the target is located.

[0086] In the embodiments of this application, during the training process of the garbage classification model, the initial garbage images come from the following three sources: Source 1 to Source 3.

[0087] Source 1: Data collected in waste sorting scenarios.

[0088] Specifically, the garbage images belonging to Source 1 are collected from real garbage sorting scenarios. Furthermore, this application utilizes image acquisition devices installed directly above each garbage bin to collect garbage images.

[0089] Source 2: Images related to garbage sorting scenarios were obtained from existing image datasets.

[0090] Specifically, the garbage images belonging to Source 2 are images related to garbage sorting scenarios selected from other labeled datasets, which include objects that may appear in garbage bins. For example, datasets containing common types of garbage, or datasets containing materials from urban management departments.

[0091] Source 3: Obtained through image generation model processing.

[0092] Specifically, the spam images belonging to source three are generated by an image generation model that learns features from spam images obtained from the other two sources. The targets in the spam images generated by the image generation model belong to categories where the number of targets in the spam images obtained from the other two sources is less than a preset threshold. In other words, this application uses an image generation model to generate sample images containing certain targets with a small number of samples, constructing a generated image set.

[0093] In one or more embodiments, before acquiring garbage images in a garbage sorting scenario, this application divides the monitoring screen of the image acquisition device and only acquires images of the area where the garbage bin is located. This avoids other irrelevant background areas from affecting the subsequent model training and detection results. At the same time, during the implementation process after the model is trained, when acquiring garbage images for garbage sorting, the monitoring screen of the image acquisition device is also divided and only the area where the garbage bin is located is acquired. This can avoid affecting the garbage sorting results if the garbage bin's position or shape changes.

[0094] In this embodiment of the application, after labeling the target category and target location in the garbage images from the above three sources, the labeled garbage images are divided into a training set and a validation set. Then, in order to improve the accuracy and speed of the target detection model, a series of data augmentation processes are required on the training set used for model training.

[0095] Specifically, in addition to the data augmentation methods commonly used when training object detection models, the data augmentation processing methods in this application also include adjusting image attributes, cleaning up overlapping objects, balancing the proportion of various objects, and setting different learning rates for garbage images from different sources during the model training process.

[0096] The data augmentation processing method for adjusting image attributes in this application is described below.

[0097] As a feasible implementation method, the image attributes of the regions where each target is located in the selected garbage images are adjusted to obtain the first dataset, including the following steps S201a-S201b:

[0098] Step S201a: Adjust the image attributes of each target region in the selected garbage image according to the preset image attribute adjustment method to obtain the garbage image after adjusting the image attributes.

[0099] The aforementioned image attributes include at least one of hue, brightness, and contrast.

[0100] Step S201b: Replace the selected garbage image with the garbage image whose image attributes have been adjusted in the original garbage image to obtain the first dataset.

[0101] For example, this application selects 30% of the garbage images from the training set and adjusts the image attributes of the regions where each target is located in these 30% of garbage images to obtain garbage images with adjusted image attributes.

[0102] Then, the first dataset consisted of 30% of the garbage images whose image attributes had been adjusted and 70% of the original garbage images whose image attributes had not been adjusted.

[0103] In some embodiments, a pre-trained image segmentation model is used to determine the regions where each target is located in the selected garbage image.

[0104] Optionally, a deep learning model containing a self-attention module (SAM) can be used to segment the regions where each target is located in the selected garbage images. After training, the deep learning model can identify and segment the targets in the garbage images and generate a mask (i.e., a binary image), in which the regions where the targets are located are marked as 1 (or 255) and the background regions are marked as 0.

[0105] After obtaining the target mask, image attributes of the area covered by the mask can be adjusted, such as changing hue, brightness, and contrast. By adjusting the hue, brightness, and contrast of the areas where each target is located in the selected garbage image, more training data can be created, increasing the diversity of garbage image data, thereby improving the robustness of the target detection model to image conditions under different garbage classification scenarios and the model's generalization ability.

[0106] Optionally, the above image attribute adjustment method can be to adjust the image attributes of each target region in a random manner after segmenting the regions where each target is located.

[0107] Optionally, the above image attribute adjustment methods can be to use the same image attribute adjustment method for each target area, or to use different image attribute adjustment methods for each target area. Alternatively, the image attribute adjustment method can be set according to the actual scenario and needs to increase the diversity of junk image data.

[0108] Step S202: Copy the garbage images containing multiple overlapping targets from the first dataset, and add the copied garbage images to the first dataset to obtain the second dataset;

[0109] The copied garbage image is labeled with one of the multiple overlapping targets.

[0110] Because garbage images collected in garbage sorting scenarios often contain objects that overlap, the difficulty of object detection models to identify them increases.

[0111] Therefore, this application copies the garbage images with high target overlap in the first dataset. By copying the image and ignoring some overlapping targets, the copied garbage image is labeled with one of the multiple overlapping targets, thereby increasing the number of training samples with different overlap conditions and improving the model's ability to recognize overlapping targets.

[0112] Specifically, for a junk image, the intersection-union ratio (IUGR) between targets is calculated, and when the IUGR exceeds a preset value, overlapping targets in the junk image need to be cleaned up. The IUGR is calculated as (intersection area) / (union area).

[0113] For example, for a garbage image Q containing 5 targets, the intersection-union ratio between targets a and b exceeds a preset value, and the other 3 targets c, d, and e do not overlap with other targets, then garbage images Q1 and Q2 will be obtained. Garbage images Q1 and Q2, like garbage image Q, all contain the 5 targets a, b, c, d, and e. However, the label of b in garbage image Q1 is ignored, while the labels of targets a, c, d, and e are retained. The same applies to garbage image Q2.

[0114] Step S203: Train the object detection model based on the second dataset.

[0115] As a feasible implementation method, the object detection model is trained based on the second dataset, including the following steps S203a and S203b:

[0116] Step S203a: Based on the source of the garbage images in the second dataset, set different learning rates for garbage images from different sources;

[0117] Specifically, the learning rate is an important hyperparameter in machine learning. In particular, the learning rate controls the magnitude of parameter adjustment in the object detection model with each weight update. In the embodiments of this application, a larger learning rate means that the model parameters will change more with each update, which helps the model explore the parameter space more quickly and find better solutions.

[0118] like Figure 3 The diagram shown is a schematic representation of the model training process provided in an embodiment of this application.

[0119] As can be seen from the foregoing embodiments, the three sources of garbage images in this application, namely Source 1 to Source 3, have different data distributions and noise levels, reflecting different actual scenarios or conditions.

[0120] Therefore, by setting different learning rates for different data sources, the object detection model can learn the characteristics of each scene better and learn the features of garbage images more effectively, thereby improving model performance, enabling the object detection model to converge to the optimal solution faster, and improving the model's generalization ability in different garbage classification scenarios.

[0121] Optionally, this application sets corresponding learning rates a, b, and c for the garbage images from sources one to three, where a is the largest. That is, in some embodiments, this application focuses on learning features from garbage images collected from real-world garbage sorting scenarios. Garbage images collected from real-world scenarios typically better reflect the challenges faced by the model in practical applications. By setting a larger learning rate for real-world scenario data, the model can be more influenced by this data during training, making it easier to learn features that perform well in these real-world scenarios, thus enabling it to quickly adapt to real-world applications.

[0122] In addition, if the object detection model uses too much synthetic or simulated data from source 3 for training, it may overfit to these data. Therefore, in order to avoid the model overfitting to synthetic or simulated data and to improve the model's performance in practical applications, the learning rate c corresponding to source 3 is usually set to be the minimum.

[0123] Step S203b: Train the object detection model multiple times based on the second dataset.

[0124] It should be noted that, Figure 3 Each source corresponds to one batch for illustration only. The junk images under each source can be further divided into a set number of batches. The specific number of batches and the order of batch input depend on the actual scenario and needs. This application embodiment does not impose too many limitations on this.

[0125] Continue to refer to Figure 3 In each training round, junk images from the second dataset are input in batches. The sources of junk images in different batches may be the same or different, while junk images in the same batch may have the same source. The learning rate of a batch of junk images input into the object detection model during training depends on the source of that batch of junk images.

[0126] In other words, this application uses the same second dataset to train the object detection model in multiple rounds. During each round of training, garbage images from various sources in the second dataset are input into the object detection model in batches, and the learning rate of each batch of data depends on the source.

[0127] like Figure 4 The diagram shown is a schematic representation of the target detection model architecture provided in an embodiment of this application. (Refer to...) Figure 4 The target detection model includes a feature extraction network, a feature fusion network, and a detection network.

[0128] In some embodiments, the feature extraction network employs the deep residual network CSPDarknet, and the C3 module in CSPDarknet is replaced with the C2F module, and the standard convolution Conv in CSPDarknet is replaced with a dynamic convolution.

[0129] Specifically, replacing the C3 module with the C2F module reduces computation while maintaining sufficient feature diversity, thus improving detection accuracy and speed. Replacing the standard convolution (Conv) with dynamic convolution is because dynamic convolution requires two additional computations compared to standard convolution:

[0130] Attention weight calculation and dynamic weight fusion.

[0131] In this model, the attention weights are not fixed but change with the input. This gives dynamic convolution a stronger feature representation capability. By reducing redundant information and improving feature quality, the model can more flexibly adapt to different input features, thereby improving the model's ability to recognize garbage images in various scenarios and the accuracy of detecting various types of garbage.

[0132] In some embodiments, the feature fusion network is a lightweight asymptotic pyramid network, Slim AFPN, and a lightweight convolutional network, GSConv, replaces the standard convolutional network, Conv, as the upsampling component in the Slim AFPN. The detection network is an Autoassign Head.

[0133] Specifically, refer to Figure 4 In the feature fusion network, upward arrows represent upsampling, and downward arrows represent downsampling. GSConv first downsamples the input using a regular convolution to obtain a convolution result, then performs a depthwise convolution using depthwise separable convolution (DWConv) to obtain another convolution result. The two convolution results are then concatenated, and finally a shuffle operation is performed to connect the channel counts of the two previous convolution results. GSConv typically has fewer parameters and lower computational cost than a standard convolution; replacing standard convolutions with GSConv can reduce model size and computational complexity.

[0134] It should be noted that 1×1 convolution and bilinear interpolation are used to upsample the features, and different convolution kernels and strides are used to perform downsampling depending on the required downsampling rate.

[0135] Furthermore, this application uses a Slim AFPN network as the feature fusion network, which progressively integrates low-level features (Low), mid-level features (Mid), and high-level features (High) during the bottom-up feature extraction process of the feature extraction network. In this process, Slim AFPN initially fuses the low-level feature (Low), then the mid-level feature (Mid), and finally the high-level feature (High). The structure of Slim AFPN helps capture information at different scales, and through multi-level feature fusion, the model can better handle targets of different sizes and shapes. This structure enhances the model's adaptability to different input variations when handling target detection tasks in complex scenes.

[0136] In the feature fusion network of this application embodiment, GSConv can be used to quickly extract key features, while Slim AFPN can ensure that these features are effectively utilized at different levels.

[0137] Therefore, by combining GSConv and Slim AFPN structures to form a feature fusion network, we can not only reduce the parameters of model training, reduce the amount of computation, and improve the training efficiency of the model, making it easier for the model to converge to a good solution, but also progressively fuse the features of each output layer of the feature extraction network. This can also improve the problem of weak detection performance for large objects and avoid large semantic gaps between non-adjacent layers.

[0138] Furthermore, compared to the two-stage object detection model that uses ordinary convolutional layers and FPN, which has weak representation capabilities, large semantic differences between layers, and relatively high computational cost, the single-stage object detection model provided in this application, which has undergone the model improvement measures in the aforementioned embodiments, not only improves the model training efficiency by using training samples processed based on the data augmentation method provided in this application, but also improves the model's accuracy in addressing common problems in garbage classification scenarios such as lighting changes, complex backgrounds, and object occlusion.

[0139] In some embodiments, after obtaining the second dataset, it is necessary to balance the proportion of various objectives in the second dataset before training the model based on the balanced second dataset.

[0140] As a feasible implementation method, after obtaining the second dataset, before training the object detection model based on the second dataset, the following steps S202a-S202c are used to balance the proportion of various objects in the second dataset:

[0141] Step S202a: Determine the categories of the labeled targets in the second dataset that meet the conditions for balancing.

[0142] The aforementioned unbalanced condition is that the proportion of the unbalanced target corresponding to the category that meets the unbalanced condition among all targets labeled in the second dataset is less than a preset proportion threshold.

[0143] For the garbage images in the second dataset, this application needs to iterate through the annotations of all targets and count the number of targets in each category. Then, the categories with a target count ratio less than a preset ratio threshold are identified as the categories that meet the balancing conditions, and the targets corresponding to the categories that need to be balanced are the targets to be balanced.

[0144] Step S202b: Copy any garbage image in the second dataset that includes the target to be balanced, and add the copied garbage image copy to the second dataset. Each copied garbage image copy retains only the annotation of the target to be balanced.

[0145] Step S202c: Repeat the above process until there are no categories in the second dataset that meet the balance condition among the categories to which the labeled targets belong.

[0146] In this embodiment of the application, the number of targets corresponding to the category to be balanced is increased by copying the garbage image containing the target to be balanced, so as to balance the number of targets in each category and make the number of target samples in each category meet the minimum proportion requirement.

[0147] Specifically, in this embodiment of the application, when copying garbage images, the targets that need to be balanced are retained, while the labels of targets that do not need to be balanced are ignored (the targets ignored in the copied image will not participate in training later), and the above process is repeated until the number of targets of the category to be balanced meets the minimum proportion requirement.

[0148] The training method for the waste classification model provided in this application involves, firstly, adjusting the image attributes of some waste images in the original waste images to obtain a first dataset. Then, copying waste images from the first dataset that include multiple overlapping targets, and adding the copied waste images to the first dataset to obtain a second dataset. The copied waste images are labeled with one of the multiple overlapping targets. Finally, the target detection model is trained based on the second dataset. This not only improves the efficiency of model training but also further enhances the accuracy and detection speed of the waste classification model.

[0149] Based on the same inventive concept, embodiments of this application also provide a training device for a waste sorting model, such as... Figure 5 As shown, the device includes:

[0150] Image attribute adjustment module 501 is used to select a set proportion of garbage images from the garbage images and adjust the image attributes of the regions where each target is located in the selected garbage images to obtain a first dataset. The first dataset includes the garbage images after the image attributes have been adjusted and the original garbage images whose image attributes have not been adjusted. The garbage images have been labeled with the category to which the target belongs and the region where the target is located.

[0151] The overlapping target processing module 502 is used to copy the garbage image that includes multiple overlapping targets in the first dataset, and add the copied garbage image to the first dataset to obtain a second dataset, wherein the copied garbage image is labeled with one of the multiple overlapping targets;

[0152] The model training module 503 is used to train the object detection model based on the second dataset.

[0153] In one possible implementation, the source of the junk images is as follows:

[0154] Images were collected from waste sorting scenarios, obtained from existing image datasets related to waste sorting scenarios, and processed using image generation models.

[0155] The image generation model generates junk images by learning features from junk images obtained from two other sources. The category to which the target in the junk image generated by the image generation model belongs is the category in which the number of targets in the junk images obtained from the other two sources is less than a preset threshold.

[0156] In one possible implementation, the model training module 503 is specifically used for:

[0157] Based on the source of the garbage images in the second dataset, different learning rates are set for garbage images from different sources. The learning rate represents the magnitude of parameter adjustment of the target detection model during each weight update.

[0158] The target detection model is trained in multiple rounds based on the second dataset. In each round of training, garbage images from the second dataset are input in batches, and each batch of garbage images comes from the same source. The learning rate of each batch of garbage images input into the target detection model during training is equal to the learning rate set for each batch of garbage images.

[0159] In one possible implementation, the image attribute adjustment module 501 is specifically used for:

[0160] According to a preset image attribute adjustment method, the image attributes of each target region in the selected garbage image are adjusted to obtain a garbage image with adjusted image attributes, wherein the image attributes include at least one of hue, brightness and contrast;

[0161] The selected garbage images are replaced with the garbage images whose image attributes have been adjusted from the original garbage images to obtain the first dataset.

[0162] In one possible implementation, the target detection model includes a feature extraction network, a feature fusion network, and a detection network;

[0163] The feature extraction network is a deep residual network CSPDarknet, and the C3 module in CSPDarknet is replaced with a C2F module, and the standard convolution Conv in CSPDarknet is replaced with a dynamic convolution.

[0164] The feature fusion network is a lightweight asymptotic pyramid network Slim AFPN, and a lightweight convolution GSConv is used to replace the standard convolution Conv as the upsampling component in the Slim AFPN;

[0165] The detection network is an Autoassign Head.

[0166] In one possible implementation, the apparatus further includes a target balancing module. After obtaining the second dataset, and before training the target detection model based on the second dataset, the target balancing module is specifically used for:

[0167] Determine the categories of the labeled targets in the second dataset that meet the unbalanced conditions. The unbalanced conditions are that the proportion of the unbalanced targets corresponding to the categories that meet the unbalanced conditions in all the labeled targets in the second dataset is less than a preset proportion threshold.

[0168] Copy any garbage image in the second dataset that includes the target to be balanced, and add the copied garbage image copy to the second dataset. Each copied garbage image copy retains only the annotation of the target to be balanced.

[0169] Repeat the above process until there are no categories in the second dataset that meet the balance condition for the labeled targets.

[0170] In one possible implementation, the image attribute adjustment module 501 is specifically used for:

[0171] The regions where each target is located in the selected garbage image are determined using a pre-trained image segmentation model.

[0172] Based on the same inventive concept, this application also provides a training device 600 for a waste sorting model, such as... Figure 6 As shown, it includes at least one processor 602; and a memory 601 communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the training method of the above-described garbage classification model.

[0173] Memory 601 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions. Memory 601 may be volatile memory, such as random-access memory (RAM); it may also be non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may be any one or a combination of the above-mentioned volatile and non-volatile memory types.

[0174] Processor 602 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0175] This invention also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to execute the training method for the garbage classification model provided in the above embodiments.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0178] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0180] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0181] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0182] The technical solutions provided in this application have been described in detail above. Specific examples have been used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0183] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0187] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A training method for a waste sorting model, characterized in that, include: A set proportion of garbage images are selected from the garbage images, and the image attributes of each target region in the selected garbage images are adjusted to obtain a first dataset. The first dataset includes garbage images with adjusted image attributes and garbage images in the original garbage images whose image attributes have not been adjusted. The garbage images have been labeled with the target category and the target region. Copy the junk image from the first dataset that includes multiple overlapping targets, and add the copied junk image to the first dataset to obtain a second dataset, wherein the copied junk image is labeled with one of the multiple overlapping targets; The object detection model is trained based on the second dataset.

2. The method according to claim 1, characterized in that, The sources of the spam images are as follows: Images were collected from waste sorting scenarios, obtained from existing image datasets related to waste sorting scenarios, and processed using image generation models. The image generation model generates junk images by learning features from junk images obtained from two other sources. The category to which the target in the junk image generated by the image generation model belongs is the category in which the number of targets in the junk images obtained from the other two sources is less than a preset threshold.

3. The method according to claim 2, characterized in that, The step of training the object detection model based on the second dataset includes: Based on the source of the garbage images in the second dataset, different learning rates are set for garbage images from different sources. The learning rate represents the magnitude of parameter adjustment of the target detection model during each weight update. The target detection model is trained in multiple rounds based on the second dataset. In each round of training, garbage images from the second dataset are input in batches, and each batch of garbage images comes from the same source. The learning rate of each batch of garbage images input into the target detection model during training is equal to the learning rate set for each batch of garbage images.

4. The method according to claim 1, characterized in that, The first dataset is obtained by adjusting the image attributes of the regions where each target is located in the selected garbage images, including: According to a preset image attribute adjustment method, the image attributes of each target region in the selected garbage image are adjusted to obtain a garbage image with adjusted image attributes, wherein the image attributes include at least one of hue, brightness and contrast; The selected garbage images are replaced with the garbage images whose image attributes have been adjusted from the original garbage images to obtain the first dataset.

5. The method according to claim 1, characterized in that, The target detection model includes a feature extraction network, a feature fusion network, and a detection network; The feature extraction network is a deep residual network CSPDarknet, and the C3 module in CSPDarknet is replaced with a C2F module, and the standard convolution Conv in CSPDarknet is replaced with a dynamic convolution. The feature fusion network is a lightweight asymptotic pyramid network Slim AFPN, and a lightweight convolution GSConv is used to replace the standard convolution Conv as the upsampling component in the Slim AFPN; The detection network is an Autoassign Head.

6. The method according to claim 1, characterized in that, After obtaining the second dataset, before training the object detection model based on the second dataset, the method further includes: Determine the categories of the labeled targets in the second dataset that meet the unbalanced conditions. The unbalanced conditions are that the proportion of the unbalanced targets corresponding to the categories that meet the unbalanced conditions in all the labeled targets in the second dataset is less than a preset proportion threshold. Copy any garbage image in the second dataset that includes the target to be balanced, and add the copied garbage image copy to the second dataset. Each copied garbage image copy retains only the annotation of the target to be balanced. Repeat the above process until there are no categories in the second dataset that meet the balance condition for the labeled targets.

7. The method according to claim 4, characterized in that, The regions where each target is located in the selected garbage image are determined using a pre-trained image segmentation model.

8. A training device for a waste sorting model, characterized in that, include: The image attribute adjustment module is used to select a set proportion of garbage images from the garbage images and adjust the image attributes of the regions where each target is located in the selected garbage images to obtain a first dataset. The first dataset includes the garbage images with adjusted image attributes and the original garbage images with unadjusted image attributes. The garbage images have been labeled with the category to which the target belongs and the region where the target is located. An overlapping target processing module is used to copy a junk image from the first dataset that includes multiple overlapping targets, and add the copied junk image to the first dataset to obtain a second dataset, wherein the copied junk image is labeled with one of the multiple overlapping targets; The model training module is used to train the object detection model based on the second dataset.

9. A training device for a waste sorting model, characterized in that, It includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the training method of the garbage classification model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method for the garbage classification model as described in any one of claims 1-7.

Images