A deep learning-based garbage classification method and system

Abstract

The application discloses a kind of based on deep learning's garbage classification method and system, it is related to garbage recycling technical field, including obtaining historical garbage classification data and constructing training sample set, utilize the deep learning model including classification branch and morphological feature extraction branch to carry out feature extraction and classification training to garbage picture, obtain garbage classification model;Obtain garbage picture to be classified, and output garbage classification result and extract morphological feature vector by garbage classification model;Based on classification label, morphological feature vector and acquisition time information, construct garbage classification time sequence characteristic data, monitor the change of the morphological feature distribution of each kind of garbage with acquisition time, generate morphological drift early warning information according to the test result of current morphological feature and historical sample range;Update garbage classification model in combination with new training sample.The application can perceive garbage appearance morphological change, and improve the adaptability of long-term operation of garbage classification model.

Description

Technical Field

[0001] This invention relates to the field of waste recycling technology, specifically to a waste sorting method and system based on deep learning. Background Technology

[0002] With the development of deep learning and computer vision technologies, image recognition-based automatic waste sorting technology is gradually being applied to the field of intelligent waste management. Existing waste sorting systems typically collect image data of different categories of waste and use deep learning models to learn visual features such as color, texture, contour, and spatial structure in the waste images to achieve automatic identification of different waste categories. Currently, deep learning-based waste sorting models can, to some extent, replace manual waste classification, improving the automation level of waste sorting and processing.

[0003] In actual waste disposal, the same type of waste may exhibit different appearances due to factors such as storage environment, degree of damage, compression, pollution, and differences in processing procedures. This results in variations in the visual features contained in waste images. Furthermore, as the number of waste samples continues to increase, the actual waste image data collected may differ from the data used in the initial training phase. Therefore, how to improve the adaptability of image recognition-based waste sorting technology in complex application environments, enabling it to meet the intelligent sorting needs during long-term operation, has become a continuous concern in the field of intelligent waste disposal. Summary of the Invention

[0004] To address the aforementioned technical problems, this paper provides a waste sorting method and system based on deep learning. This solution addresses the issue raised in the background that, in actual waste processing, the same type of waste may exhibit different appearances due to factors such as storage environment, degree of damage, compression, pollution, and processing procedures. This results in variations in the visual features contained in waste images. Furthermore, as the number of waste samples increases, the actual collected waste image data may differ from the data used in the initial training phase. Therefore, improving the adaptability of image recognition-based waste sorting technology in complex application environments, enabling it to meet the intelligent sorting needs during long-term operation, has become a persistent concern in the field of intelligent waste management.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A deep learning-based waste sorting method includes:

[0007] S1. Obtain historical waste sorting data and waste sorting standards, extract historical waste images corresponding to different waste sorting categories from the historical waste sorting data, and construct classification labels for the historical waste images according to the waste sorting standards to form a training sample set;

[0008] S2. Build a deep learning model. Input historical garbage images from the training sample set into the deep learning model. The deep learning model extracts the feature information of the historical garbage images and trains the deep learning model based on the classification labels to obtain a garbage classification model.

[0009] S3. Obtain images of waste to be classified, input them into the waste classification model, and have the waste classification model output waste classification results and extract the corresponding morphological feature vectors. Verify the waste classification results based on the actual waste classification results, use the verified waste classification results as training labels, and incorporate the images of waste to be classified, training labels, and corresponding morphological feature vectors into the training sample set.

[0010] S4. Based on the training sample set and the garbage classification results, garbage classification time series feature data is generated. The distribution changes of morphological features of garbage categories in different collection periods are analyzed. When the current morphological features exceed the range of historical samples, morphological drift warning information is generated.

[0011] S5. Based on the verified waste classification results, the newly added training sample set, and the morphological drift warning information, determine the model parameter update requirements of the waste classification model. Depending on the degree of morphological drift, select incremental training or full retraining to update the waste classification model and obtain the iteratively optimized waste classification model.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0013] In this invention, a deep learning model containing classification branches and morphological feature extraction branches is constructed. While outputting the waste classification results, morphological feature vectors that can characterize the changes in the appearance of waste are extracted. Combined with the collection time information, waste classification time-series feature data is formed. Thus, the waste classification model can not only identify the current waste category, but also perceive the appearance changes of different categories of waste over time, providing a data foundation for subsequent morphological change analysis.

[0014] In this invention, based on the classification labels, morphological feature vectors, and collection time information in historical waste classification samples, time-series data of morphological features corresponding to different waste classification types are established. By analyzing the changes in the distribution of morphological features between different time stages, the change pattern of waste appearance is detected, and the model is optimized and updated based on the detection results.

[0015] In this invention, by combining deep learning image recognition with temporal morphological change analysis, dynamic adjustments can be made for actual application scenarios such as waste damage, compression, pollution, and morphological changes, so that the waste sorting system can maintain good recognition reliability and adaptability during long-term operation. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the overall process of the waste sorting method in this invention.

[0017] Figure 2 This is a flowchart of the early warning information acquisition steps in the waste sorting method of this invention;

[0018] Figure 3 This is a flowchart of the model iteration steps of the waste sorting method in this invention;

[0019] Figure 4 This is a logical framework diagram of the waste sorting system in this invention. Detailed Implementation

[0020] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0021] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] Reference Figures 1-4 As shown, a deep learning-based garbage classification method specifically includes:

[0024] S1. Obtain historical waste sorting data and waste sorting standards, extract historical waste images corresponding to different waste sorting categories from the historical waste sorting data, and construct classification labels for the historical waste images according to the waste sorting standards to form a training sample set;

[0025] S2. Build a deep learning model. Input historical garbage images from the training sample set into the deep learning model. The deep learning model extracts the feature information of the historical garbage images and trains the deep learning model based on the classification labels to obtain a garbage classification model.

[0026] S3. Obtain images of waste to be classified, input them into the waste classification model, and have the waste classification model output waste classification results and extract the corresponding morphological feature vectors. Verify the waste classification results based on the actual waste classification results, use the verified waste classification results as training labels, and incorporate the images of waste to be classified, training labels, and corresponding morphological feature vectors into the training sample set.

[0027] S4. Based on the training sample set and the garbage classification results, garbage classification time series feature data is generated. The distribution changes of morphological features of garbage categories in different collection periods are analyzed. When the current morphological features exceed the range of historical samples, morphological drift warning information is generated.

[0028] S5. Based on the verified waste classification results, the newly added training sample set, and the morphological drift warning information, determine the model parameter update requirements of the waste classification model. Depending on the degree of morphological drift, select incremental training or full retraining to update the waste classification model and obtain the iteratively optimized waste classification model.

[0029] Optionally, step S1 specifically includes:

[0030] Acquire historical waste sorting data, extract images of different types of historical waste from the historical waste sorting data, and remove historical waste images with incomplete main bodies;

[0031] Perform at least one of the following preprocessing operations on the retained historical junk images: size unification, pixel value scaling, random cropping, or color adjustment;

[0032] Based on the waste classification standards, the classification labels corresponding to each historical waste image are determined, and the classification labels are associated with the corresponding historical waste images to form training samples.

[0033] Obtain the collection time information corresponding to each historical waste image, and associate the collection time information with the corresponding training samples to form a training sample set containing historical waste images, classification labels and collection time information;

[0034] In an optional embodiment, historical waste sorting data is first obtained, which includes historical waste images corresponding to different waste sorting categories and the collection time information corresponding to the historical waste images;

[0035] Since historical garbage images may contain issues such as the garbage being obscured, the garbage not appearing completely within the image area, or the garbage outline being unidentifiable, manual inspection can be used to screen historical garbage images for completeness. Historical garbage images with obscured garbage, missing garbage areas, or unidentifiable garbage can be removed to reduce the impact of abnormal samples on subsequent model training.

[0036] Subsequently, image preprocessing was performed on the preserved historical garbage images. The purpose of image preprocessing is to unify the data format of historical garbage images from different acquisition devices and shooting environments, so that each historical garbage image has a consistent data scale, thereby improving the stability of deep learning models when extracting garbage features.

[0037] Image preprocessing may include one or more of the following operations: resizing, pixel scaling, random cropping, and color adjustment.

[0038] After image preprocessing, the waste classification category of each historical waste image is determined according to the waste classification standards, and the association between the historical waste images and the corresponding classification labels is established to form training samples;

[0039] Next, the collection time information corresponding to each historical garbage image is extracted, and the collection time information is associated with the corresponding training samples and stored to form a training sample set containing historical garbage images, classification labels and collection time information;

[0040] Since the training sample set simultaneously stores waste category information, image information, and collection time information, it can not only be used for training the waste classification model in subsequent steps, but also provide basic data support for constructing waste classification time-series feature data in subsequent steps, enabling the model to analyze the changes in the distribution of waste appearance features with collection time.

[0041] For example, obtain historical waste sorting data collected over three consecutive months, including historical waste images corresponding to multiple waste sorting categories such as plastic bottles, beverage cans, cardboard boxes, fruit peels, and waste textiles;

[0042] First, the integrity of each historical garbage image is checked. If the main body of the garbage is obscured by other objects, resulting in the loss of part of the garbage body, or if the main body of the garbage exceeds the image boundary, resulting in incomplete outline information, or if the outline of the main body of the garbage cannot be effectively identified, the corresponding historical garbage image will be removed.

[0043] Subsequently, the retained historical junk images were uniformly adjusted to 224×224 pixels, and pixel value scaling and random cropping were performed to reduce the image differences caused by different acquisition devices.

[0044] Then, according to the waste classification standards, corresponding classification labels were assigned to "recyclable waste", "kitchen waste", "hazardous waste" and "other waste", and the collection time corresponding to each historical waste image was recorded, such as January 1, 2026, January 3, 2026 and January 4, 2026;

[0045] Finally, a training sample set is formed, consisting of historical garbage images, classification labels, and collection time information. This training sample set is then input into the subsequent deep learning model for training and serves as the data source for constructing subsequent garbage classification time-series feature data.

[0046] Optionally, step S2 specifically includes:

[0047] The input layer, feature extraction layer, classification branch, and morphological feature extraction branch of a deep learning model are established based on at least one of convolutional neural networks, attention-based neural networks, or hybrid neural networks that combine convolutional neural networks and attention mechanisms.

[0048] Historical garbage images from the training sample set are fed into the input layer, where the feature extraction layer extracts features to obtain a garbage feature map.

[0049] The garbage feature map is input into the classification branch and the morphological feature extraction branch respectively. The classification branch outputs the predicted classification result, and the morphological feature extraction branch extracts the morphological feature vector representing the appearance of the garbage. The morphological feature vector is then associated with the collection time information of the corresponding historical garbage images.

[0050] The classification loss of the predicted classification result is calculated using the classification labels in the training sample set, and the deep learning model is iteratively trained based on the classification loss to update the model parameters of the deep learning model and obtain the garbage classification model.

[0051] In an optional embodiment, after obtaining the training sample set, a deep learning model is first established based on a convolutional neural network, an attention-based neural network, or a hybrid neural network combining convolutional neural networks and attention mechanisms.

[0052] A deep learning model includes at least an input layer, a feature extraction layer, a classification branch, and a morphological feature extraction branch. The input layer is used to receive historical garbage images, the feature extraction layer is used to extract garbage appearance information such as texture features, contour features, and color features from the garbage images, the classification branch is used to learn garbage classification features, and the morphological feature extraction branch is used to extract morphological feature vectors that can characterize changes in the appearance of garbage.

[0053] Subsequently, historical garbage images from the training sample set are fed into the input layer, where the feature extraction layer extracts features from the historical garbage images layer by layer to generate a garbage feature map.

[0054] Since the garbage feature map retains the main appearance features of the garbage image, it can be used as a shared input for the classification branch and the morphological feature extraction branch.

[0055] The waste feature map is input into the classification branch and the morphological feature extraction branch respectively. The classification branch outputs the corresponding predicted classification result. The morphological feature extraction branch extracts the appearance structure information from the waste feature map to obtain a morphological feature vector containing waste outline features, size ratio features, texture features and surface morphological change features. The morphological feature vector is then associated with the collection time information of the corresponding historical waste image.

[0056] Since the collection time information can reflect the collection order of garbage images, subsequent steps can construct garbage classification time-series feature data based on the morphological feature vector and collection time information, and analyze the changes in the distribution of garbage appearance morphological features with collection time.

[0057] Next, based on the classification labels corresponding to each historical garbage image in the training sample set, the cross-entropy classification loss between the predicted classification probability output by the model and the classification label is calculated. The classification label is converted into a one-hot encoded vector, and the probability value of the category position corresponding to the classification label in the predicted classification probability is obtained. The negative natural logarithm of the probability value is taken to obtain the classification loss corresponding to a single sample. The classification losses corresponding to multiple samples in a batch are arithmetically averaged to obtain the classification loss of that batch. Based on the classification loss, the model parameters of the deep learning model are updated through backpropagation to obtain the garbage classification model.

[0058] The classification loss corresponding to the predicted classification result is calculated using the classification labels in the training sample set, and the deep learning model is iteratively updated based on the classification loss. During the iterative update process, when the classification loss generated by subsequent iterations no longer decreases compared to the previous iteration, and the predicted classification result output by the model remains stable, the training of the garbage classification model is completed. The garbage classification model can not only output the garbage classification result, but also simultaneously extract the morphological feature vector corresponding to the appearance of the garbage, providing basic data for subsequent garbage classification, morphological drift analysis and model updates.

[0059] In one alternative embodiment, the deep learning model employs a shared feature extraction and multi-branch output structure, including an input layer, a shared feature extraction layer, a classification branch, and a morphological feature extraction branch.

[0060] The input layer receives historical garbage images from the training sample set and adjusts them to a uniform input size. The shared feature extraction layer consists of multiple stacked convolutional layers, with pooling layers used for feature dimensionality reduction between each convolutional layer. This layer is used to extract features from the input historical garbage images step by step. Through convolution operations, it extracts color, texture, edge contours, and local structural information from the garbage images, forming a shared feature map containing garbage appearance information.

[0061] After the shared feature map is output from the shared feature extraction layer, it is input into the classification branch and the morphological feature extraction branch respectively. The classification branch converts the shared feature map into a feature vector through global average pooling and then feeds it into a fully connected layer. The number of neurons in the fully connected layer is equal to the number of waste classification categories defined in the waste classification standard. It outputs the predicted probability of each category and outputs the waste classification result according to the category with the highest probability. The morphological feature extraction branch converts the shared feature map into a feature vector through global average pooling and then compresses and encodes the features through an additional fully connected layer. It outputs a fixed-dimensional morphological feature vector, which is used to characterize the shape changes, surface texture changes, and structural changes of waste.

[0062] By sharing feature extraction and a dual-branch output structure, the waste classification model can simultaneously obtain morphological feature vectors that reflect changes in the appearance of waste while completing waste category identification. This provides a data foundation for subsequent construction of waste classification time-series feature data based on collection time information and for morphological drift analysis.

[0063] For example, a hybrid neural network combining convolutional neural networks and attention mechanisms can be used as a deep learning model. The convolutional neural network is responsible for extracting local texture features and contour features from historical garbage images, while the attention mechanism is used to enhance the weight of features corresponding to the main garbage region and reduce the impact of the background region on model training.

[0064] Historical garbage images such as plastic bottles, aluminum cans, cardboard boxes, and fruit peels from the training sample set are fed into the input layer, and after processing by the feature extraction layer, garbage feature maps are generated.

[0065] Subsequently, the classification branch outputs the corresponding waste category based on the waste feature map, such as outputting predicted classification results like "recyclables", "kitchen waste", and "other waste";

[0066] The morphological feature extraction branch simultaneously extracts the corresponding morphological feature vectors and associates them with the collection time information of each historical garbage image for storage.

[0067] For example, for plastic bottle waste, corresponding waste images are acquired at multiple collection time stages. The morphological feature vectors corresponding to each image are extracted through the morphological feature extraction branch in the waste classification model, and then associated according to the collection time information.

[0068] During the historical sample analysis, the distribution of morphological features corresponding to different appearance states such as intact bottles, compressed bottles, and broken bottles in different collection stages was statistically analyzed, and the degree of change in morphological feature distribution between different collection stages was compared.

[0069] When new images of plastic bottle waste are collected, the waste classification model extracts the current morphological feature vector and compares it with the morphological feature distribution corresponding to historical plastic bottle waste.

[0070] If the current morphological characteristics are still within the distribution range corresponding to the historical samples, then the current appearance change of the plastic bottle waste is considered to be an existing change state; if the current morphological characteristics continue to exceed the distribution range corresponding to the historical samples, then the plastic bottle waste is considered to have a new appearance change pattern, and morphological drift warning information is generated.

[0071] After training all training samples is completed, the classification loss corresponding to the predicted classification result is calculated based on the classification label, and the model parameters of the deep learning model are continuously updated to finally obtain the garbage classification model.

[0072] The waste sorting model can output the corresponding morphological feature vectors while outputting the waste sorting results, providing basic data for subsequent construction of waste sorting time-series feature data and verification of historical sample range.

[0073] Optional; step S3 specifically includes:

[0074] The system acquires images of waste to be classified and inputs them into a waste classification model. The model then extracts the waste features from the images, outputs the corresponding waste classification results, and generates a morphological feature vector representing the appearance of the waste.

[0075] Obtain the confirmed classification information corresponding to the images of the waste to be classified, and verify the consistency between the actual waste classification result and the waste classification result.

[0076] When the verification is consistent, the actual garbage classification result is determined as the training label, and the garbage image to be classified, the training label, the morphological feature vector and the corresponding collection time information are associated and stored in the training sample set;

[0077] When the verification is inconsistent, the garbage images to be classified and the corresponding garbage classification results are marked as samples to be reviewed, and the training sample set is updated after the review is completed.

[0078] In an optional embodiment, after completing the training of the waste classification model, images of the waste to be classified are obtained and input into the waste classification model;

[0079] The waste sorting model first extracts features from the images of the waste to be sorted, obtaining waste features that reflect the texture, color, outline and structural information of the waste, and then outputs the corresponding waste sorting results based on the waste features;

[0080] In the waste sorting model, the morphological feature extraction branch simultaneously extracts morphological feature vectors that can characterize the appearance of waste, which are used to describe the appearance of the current waste.

[0081] Subsequently, during the waste sorting process, the corresponding confirmed classification information is obtained, and the consistency of the confirmed classification information with the waste sorting results output by the waste sorting model is verified to determine whether the classification label corresponding to the image of the waste to be sorted is accurate.

[0082] The actual waste sorting results can be obtained through manual verification, feedback from waste sorting terminals, or confirmation during the waste treatment process.

[0083] When the two are consistent, it means that the current garbage classification model has correctly classified the garbage image to be classified. The actual garbage classification result can be determined as the training label, and the garbage image to be classified, the training label, the morphological feature vector and the corresponding collection time information are associated and stored to form a new training sample, which is then added to the training sample set to continuously expand the model training data.

[0084] When the actual waste sorting result is inconsistent with the waste sorting result output by the waste sorting model, the image of the waste to be sorted and the corresponding waste sorting result are marked as a sample to be reviewed and enter the review process. The result is confirmed manually.

[0085] After the review is completed, the corresponding training labels are updated according to the confirmed garbage classification results. The updated garbage images to be classified, training labels, morphological feature vectors and collection time information are added to the training sample set to ensure the accuracy of the labels in the training sample set and improve the data quality of subsequent model training and morphological feature distribution change analysis.

[0086] Through the above process, the training samples of the waste sorting model are dynamically expanded during online operation, so that the training sample set can continuously reflect the actual waste sorting situation and provide a reliable data foundation for the subsequent construction of waste sorting time-series feature data and model iteration optimization.

[0087] For example, a smart waste sorting device can acquire a picture of waste to be sorted, in which the waste is a flattened plastic beverage bottle;

[0088] The garbage images to be classified are input into the garbage classification model. After the garbage classification model extracts the garbage features, it outputs the predicted classification result as "recyclable". At the same time, the morphological feature extraction branch generates the corresponding morphological feature vector and records the current collection time.

[0089] Subsequently, the actual waste sorting result corresponding to the image of the waste to be sorted is obtained; for example, the waste sorting terminal may provide feedback that the plastic beverage bottle belongs to "recyclable waste" based on subsequent manual confirmation.

[0090] The actual waste sorting results are checked against the waste sorting results output by the waste sorting model. The results are consistent, so "recyclable materials" are determined as the training label. The images of the waste to be sorted, the training labels, the morphological feature vectors and the collection time information are associated and stored to form new training samples, which are then added to the training sample set.

[0091] In another embodiment, the waste sorting model predicts a cardboard box heavily stained with oil as "other waste," while a human review confirms that it belongs to "recyclable waste."

[0092] The image of the waste to be classified is marked as a sample to be reviewed. Based on the review results, the training label is corrected to "recyclable". The corrected training label, along with the image of the waste to be classified, the morphological feature vector, and the collection time information, is then stored in the training sample set.

[0093] By continuously introducing new, validated training samples, the waste sorting model can gradually learn the characteristics of waste in complex scenarios, improve the accuracy of subsequent waste sorting, and provide a richer and more accurate data source for constructing time-series feature data of waste sorting in subsequent steps.

[0094] Optionally, step S4 specifically includes:

[0095] The training labels, morphological feature vectors, and collection time information in the training sample set are associated and organized according to the waste classification type and collection time order to form waste classification time series feature data;

[0096] Based on the time-series characteristic data of waste sorting, the morphological characteristics of each type of waste were statistically analyzed at different collection periods, and the changes in the morphological characteristics distribution between different collection periods were analyzed.

[0097] The current morphological feature vector of the garbage to be classified during the operation is obtained, and the current morphological feature vector is compared with the historical morphological feature distribution of the corresponding garbage classification type. Based on the degree of difference between the current morphological feature vector and the historical morphological feature distribution, it is determined whether the appearance of the garbage has changed in a new pattern.

[0098] When it is determined that the current appearance of the waste has changed abnormally relative to the distribution of historical morphological features, a morphological drift warning message corresponding to the waste classification type is generated.

[0099] In an optional embodiment, after the training sample set is continuously expanded, the training labels, morphological feature vectors and collection time information corresponding to each training sample are extracted from the training sample set, and then classified and organized according to the types of waste classification. The training samples belonging to the same type of waste classification are then sorted by time according to the collection time information to form waste classification time series feature data.

[0100] Since the same type of waste may have different appearances at different collection times, associating waste type, morphological feature vector, and collection time information can generate waste classification time-series feature data that reflects the distribution of the appearance features of various types of waste at different collection times.

[0101] Subsequently, based on the time-series characteristic data of waste classification, the morphological feature distribution of each waste classification type was statistically analyzed at different collection periods. The morphological feature vectors corresponding to the same waste classification type were organized according to the collection time sequence, and the degree of difference in morphological feature distribution between different collection periods was calculated. The degree of difference was measured by the Euclidean distance or cosine similarity between the mean values ​​of morphological feature vectors at each time period. Based on the consistency of the morphological feature distribution change results within a continuous collection period, the historical morphological feature distribution range of the corresponding waste classification type was established.

[0102] Based on the changes in the morphological feature vectors corresponding to each type of waste classification during the continuous collection period, the differences in the distribution of morphological features corresponding to different collection times are statistically analyzed.

[0103] During the process, the morphological feature vector corresponding to the current garbage image to be classified is obtained, and the current morphological feature vector is compared with the historical morphological feature distribution of the corresponding garbage classification. Based on the difference between the current morphological feature vector and the historical morphological feature distribution, it is determined whether the current appearance of the garbage is still within the range of morphological changes corresponding to the historical samples.

[0104] When the current morphological feature vector falls within the range of historical morphological features, it is determined that the current appearance of the garbage belongs to the range of existing samples.

[0105] When the current morphological feature vector does not exist within the range of historical morphological features, a new morphological change pattern is determined for the corresponding waste classification type, and morphological drift warning information for the corresponding waste classification type is generated.

[0106] Morphological drift warning information serves as the basis for subsequent steps to determine whether to update the waste sorting model and to select the model training method;

[0107] Through the above process, it is possible to continuously track changes in the appearance of various types of waste and generate timely warnings when significant changes occur in the form of waste, thereby improving the adaptability of the waste sorting model to new forms of waste.

[0108] For example, training samples corresponding to plastic beverage bottles are continuously accumulated in the training sample set. Each training sample contains training labels, morphological feature vectors, and collection time information.

[0109] First, the plastic beverage bottle samples were classified according to the waste classification category of "recyclable waste". Then, they were arranged in order according to the collection time information to form the waste classification time sequence characteristic data corresponding to the plastic beverage bottles.

[0110] The morphological feature vectors of plastic beverage bottles are grouped according to the collection time, and the distribution range of morphological features in each collection period is statistically analyzed to form the historical sample range corresponding to plastic beverage bottles.

[0111] When a plastic beverage bottle image is subsequently obtained, the waste classification model extracts its morphological feature vector and compares the current morphological feature vector with the historical morphological feature distribution range corresponding to the plastic beverage bottle. If the current morphological feature vector is within the historical morphological feature distribution range, it is determined that the appearance of the current plastic beverage bottle belongs to the historical sample coverage range, and no morphological drift warning information is generated.

[0112] If the plastic beverage bottle in the current image melts at high temperature and forms a clump-like appearance, and the resulting morphological feature vector exceeds the historical morphological feature distribution range, it is determined that the morphological feature distribution of the plastic beverage bottle has changed significantly. Corresponding morphological drift warning information is generated and sent to the subsequent model update module to adjust the training strategy of the waste classification model.

[0113] Optionally, step S5 specifically includes:

[0114] Extract the verified waste classification results, the newly added training sample set and the morphological drift warning information, and calculate the classification accuracy of the waste classification model based on the verified waste classification results, calculate the number of new samples based on the newly added training sample set, and determine the changes in the appearance of waste based on the morphological drift warning information.

[0115] Based on the classification accuracy of the waste classification model, the number of new samples, and changes in the appearance of waste, determine the model parameter update requirements for the waste classification model;

[0116] When no shape drift warning information is received, incremental training of the garbage classification model is performed using a newly added training sample set.

[0117] When the morphological drift warning information indicates that the morphological feature distribution corresponding to the current appearance of garbage is not within the range of historical samples, the garbage classification model is retrained using the updated training sample set to update the model parameters of the garbage classification model.

[0118] The updated model parameters are loaded into the waste sorting model to obtain the iteratively optimized waste sorting model;

[0119] In an optional embodiment, after obtaining the verified waste sorting results, the newly added training sample set, and the morphological drift warning information, the verified waste sorting results, the newly added training sample set, and the morphological drift warning information are first extracted.

[0120] Based on the verified waste sorting results, the accuracy of the waste sorting model in the current operating stage is statistically analyzed. Based on the newly added training sample set, the number of newly added training samples is statistically analyzed. Based on the morphological drift warning information, the changes in the appearance of waste corresponding to each waste sorting category are determined, thereby comprehensively reflecting the current operating status and data changes of the waste sorting model.

[0121] Subsequently, based on the classification accuracy of the waste classification model, the number of new samples, and the changes in the appearance of the waste, it is determined whether the current waste classification model needs to be updated.

[0122] When the number of new training samples continues to increase, the accuracy of garbage classification decreases, or the appearance of garbage produces a shape drift warning, it indicates that there has been a certain deviation between the existing garbage classification model and the current garbage data. The model parameters need to be adjusted to maintain the model's ability to classify new samples.

[0123] After determining that there is a need to update the model, different model training methods are selected based on the changes in the appearance of the garbage.

[0124] When no morphological drift warning information is generated for the appearance of garbage, it means that the current morphological features are still within the range of historical samples. At this time, incremental training is performed on the garbage classification model using the newly added training sample set, and only the model parameters affected by the new samples are updated to reduce the model training cost and improve the model update efficiency.

[0125] When a morphological drift warning is generated for the appearance of waste, it indicates that the current morphological features exceed the range of historical samples, and the existing model may not be able to accurately adapt to the new waste morphology. Therefore, the waste classification model is retrained using an updated training sample set. The updated training sample set is input into the waste classification model, and the waste classification model outputs the corresponding classification results and morphological feature vectors. Based on the classification loss between the output results and the training labels, and the feature difference between the output morphological feature vector and the morphological feature vector of the corresponding training sample in the training sample set, the model parameter adjustment amount is calculated, and the model parameters corresponding to the shared feature extraction layer, classification branch, and morphological feature extraction branch are updated through backpropagation.

[0126] The updated model parameters are loaded into the waste sorting model, replacing the currently running waste sorting model to obtain an iteratively optimized waste sorting model. The iteratively optimized waste sorting model is then redeployed into the waste sorting process to provide continuously optimized model support for the recognition of subsequent images of waste to be sorted.

[0127] For example, after the waste sorting model has been running continuously for a period of time, it has accumulated 3,000 new training samples that have been verified and confirmed. The new samples mainly come from waste sorting categories such as plastic bottles, cardboard boxes and kitchen waste.

[0128] At the same time, the classification accuracy of the current waste classification model is statistically analyzed based on the waste classification results, and the changes in the appearance of waste are analyzed in combination with morphological drift early warning information;

[0129] In the first case, it was found that the new training samples were mainly concentrated on the appearance of plastic bottles that had appeared in the historical data, such as slightly compressed plastic bottles and folded cardboard boxes. The morphological features corresponding to each type of waste classification were still within the range of historical samples. Therefore, the new training sample set was used to perform incremental training on the waste classification model, only updating the part of the model parameters that was affected by the new training samples, so that the waste classification model could quickly adapt to the new data, while reducing the computing resources and training time required for model retraining.

[0130] In the second case, a large number of plastic products are continuously detected to form block structures due to high temperature melting, or kitchen waste forms new appearance shapes due to long-term accumulation, and corresponding shape drift warning information is generated.

[0131] Since the morphological characteristics of this type of waste have exceeded the range of historical samples, it was determined that the existing waste classification model could not fully reflect the new morphological characteristics of waste. Therefore, the waste classification model was retrained using the updated training sample set, and the model parameters were re-optimized.

[0132] After training is complete, the updated model parameters are loaded into the waste sorting model, replacing the current running model. This allows the updated waste sorting model to accurately identify newly appearing waste appearances and continue to be used for subsequent waste sorting tasks.

[0133] A deep learning-based waste sorting system is provided to implement the aforementioned waste sorting method, specifically including:

[0134] The training sample construction component is used to acquire historical garbage classification data and construct a training sample set for model training.

[0135] The waste sorting and recognition component is used to classify and recognize waste images using a deep learning model and extract the appearance and morphological features of the waste.

[0136] The model optimization and update component is used to dynamically optimize the waste sorting model based on changes in the appearance and morphology of waste.

[0137] Optionally, the training sample construction component specifically includes:

[0138] The historical data acquisition unit is used to acquire historical waste sorting data and extract historical waste images corresponding to different waste sorting categories.

[0139] The image preprocessing unit is used to preprocess historical junk images, remove junk images that do not meet the requirements, and perform at least one of the following processing on the retained junk images: size unification, pixel value adjustment, and image enhancement.

[0140] The tag association unit is used to determine the classification tag corresponding to each historical waste image based on the waste classification standards, and to establish an association relationship between the classification tag and the corresponding historical waste image.

[0141] The temporal information construction unit is used to obtain the collection time information corresponding to historical garbage images and associate the collection time information with training samples to form a training sample set containing images, classification labels and collection time information.

[0142] Optionally, the waste sorting and identification component specifically includes:

[0143] Deep learning model building unit, used to build a deep learning model that includes an input layer, a feature extraction layer, a classification branch, and a morphological feature extraction branch;

[0144] The model training unit is used to input the training sample set into the deep learning model, calculate the classification loss through the classification labels, update the model parameters, and obtain the garbage classification model.

[0145] The waste identification unit is used to acquire images of waste to be classified, input them into the waste classification model, output the corresponding waste classification results, and extract the corresponding morphological feature vectors.

[0146] The result verification unit is used to obtain the actual garbage classification results, verify the garbage classification results output by the model, and use the verified classification results and corresponding images as new training samples.

[0147] Optionally, the model optimization and update component specifically includes:

[0148] The temporal feature analysis unit is used to associate the classification labels, morphological feature vectors and collection time information in the training samples to form waste classification temporal feature data, and to analyze the changes in the morphological feature distribution of different waste categories with the collection time.

[0149] The morphological drift detection unit is used to compare the morphological feature vector corresponding to the current garbage image with the historical sample range of the corresponding garbage classification category.

[0150] The model update decision unit is used to determine the model update requirement based on the new training samples and morphological drift warning information, and to select the model update method.

[0151] The model iteration unit is used to update the parameters of the waste classification model by incremental training or retraining to obtain the iteratively optimized waste classification model.

[0152] In this invention, a deep learning model containing classification and morphological feature extraction branches is constructed. While outputting waste classification results, morphological feature vectors characterizing the appearance of waste are extracted, providing a data foundation for subsequent morphological analysis. The training sample set is replenished by associating the verified waste classification results with the morphological feature vectors and collection time information, allowing the training data to continuously expand during system operation. By organizing the morphological feature vectors according to waste classification type and collection time sequence to form temporal feature data, the morphological feature distribution changes of various types of waste at different collection periods are monitored. The currently acquired morphological feature vectors are compared with the historical sample range, and morphological drift warning information is generated when the range exceeds the historical sample range. Model update requirements are determined based on classification accuracy, the number of new samples, and morphological changes. Incremental training or full retraining is selected to update the model parameters based on whether morphological drift warning information is received, enabling the model to adapt to changes in waste appearance over time. This forms a closed-loop process of classification, verification, monitoring, updating, and reclassification, allowing the waste classification model to continuously adjust its parameters using actual feedback data and newly emerging waste forms during operation, maintaining its ability to identify various types of waste.

[0153] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A waste sorting method based on deep learning, characterized in that, include: S1. Obtain historical waste sorting data and waste sorting standards, extract historical waste images corresponding to different waste sorting categories from the historical waste sorting data, and construct classification labels for the historical waste images according to the waste sorting standards to form a training sample set; S2. Build a deep learning model. Input historical garbage images from the training sample set into the deep learning model. The deep learning model extracts the feature information of the historical garbage images and trains the deep learning model based on the classification labels to obtain a garbage classification model. S3. Obtain images of waste to be classified, input them into the waste classification model, and have the waste classification model output waste classification results and extract the corresponding morphological feature vectors. Verify the waste classification results based on the actual waste classification results, use the verified waste classification results as training labels, and incorporate the images of waste to be classified, training labels, and corresponding morphological feature vectors into the training sample set. S4. Based on the training sample set and the garbage classification results, garbage classification time series feature data is generated. The distribution changes of morphological features of garbage categories in different collection periods are analyzed. When the current morphological features exceed the range of historical samples, morphological drift warning information is generated. S5. Based on the verified waste classification results, the newly added training sample set, and the morphological drift warning information, determine the model parameter update requirements of the waste classification model. Depending on the degree of morphological drift, select incremental training or full retraining to update the waste classification model and obtain the iteratively optimized waste classification model.

2. The deep learning-based waste classification method according to claim 1, characterized in that, Step S1 specifically includes: Acquire historical waste sorting data, extract images of different types of historical waste from the historical waste sorting data, and remove historical waste images with incomplete main bodies; Perform at least one of the following preprocessing operations on the retained historical junk images: size unification, pixel value scaling, random cropping, or color adjustment; Based on the waste classification standards, the classification labels corresponding to each historical waste image are determined, and the classification labels are associated with the corresponding historical waste images to form training samples. Obtain the collection time information corresponding to each historical waste image, and associate the collection time information with the corresponding training samples to form a training sample set containing historical waste images, classification labels, and collection time information.

3. The waste sorting method based on deep learning according to claim 2, characterized in that, Step S2 specifically includes: The input layer, feature extraction layer, classification branch, and morphological feature extraction branch of a deep learning model are established based on at least one of convolutional neural networks, attention-based neural networks, or hybrid neural networks that combine convolutional neural networks and attention mechanisms. Historical garbage images from the training sample set are fed into the input layer, where the feature extraction layer extracts features to obtain a garbage feature map. The garbage feature map is input into the classification branch and the morphological feature extraction branch respectively. The classification branch outputs the predicted classification result, and the morphological feature extraction branch extracts the morphological feature vector representing the appearance of the garbage. The morphological feature vector is then associated with the collection time information of the corresponding historical garbage images. The classification loss of the predicted classification result is calculated using the classification labels in the training sample set. The deep learning model is then iteratively trained based on the classification loss to update the model parameters and obtain the garbage classification model.

4. The waste classification method based on deep learning according to claim 3, characterized in that, Step S3 specifically includes: The system acquires images of waste to be classified and inputs them into a waste classification model. The model then extracts the waste features from the images, outputs the corresponding waste classification results, and generates a morphological feature vector representing the appearance of the waste. Obtain the confirmed classification information corresponding to the images of the waste to be classified, and verify the consistency between the actual waste classification result and the waste classification result. When the verification is consistent, the actual garbage classification result is determined as the training label, and the garbage image to be classified, the training label, the morphological feature vector and the corresponding collection time information are associated and stored in the training sample set; When the verification is inconsistent, the images of the garbage to be classified and the corresponding garbage classification results are marked as samples to be reviewed, and the training sample set is updated after the review is completed.

5. A deep learning-based waste sorting method according to claim 4, characterized in that, Step S4 specifically includes: The training labels, morphological feature vectors, and collection time information in the training sample set are associated and organized according to the waste classification type and collection time order to form waste classification time series feature data; Based on the time-series characteristic data of waste sorting, the morphological characteristics of each type of waste were statistically analyzed at different collection periods, and the changes in the morphological characteristics distribution between different collection periods were analyzed. The current morphological feature vector of the garbage to be classified during the operation is obtained, and the current morphological feature vector is compared with the historical morphological feature distribution of the corresponding garbage classification type. Based on the degree of difference between the current morphological feature vector and the historical morphological feature distribution, it is determined whether the appearance of the garbage has changed in a new pattern. When it is determined that the current appearance of waste has changed abnormally compared with the distribution of historical morphological features, a morphological drift warning message corresponding to the waste classification type is generated.

6. The waste sorting method based on deep learning according to claim 5, characterized in that, Step S5 specifically includes: Extract the verified waste classification results, the newly added training sample set and the morphological drift warning information, and calculate the classification accuracy of the waste classification model based on the verified waste classification results, calculate the number of new samples based on the newly added training sample set, and determine the changes in the appearance of waste based on the morphological drift warning information. Based on the classification accuracy of the waste classification model, the number of new samples, and changes in the appearance of waste, determine the model parameter update requirements for the waste classification model; When no shape drift warning information is received, incremental training of the garbage classification model is performed using a newly added training sample set. When the morphological drift warning information indicates that the morphological feature distribution corresponding to the current appearance of garbage is not within the range of historical samples, the garbage classification model is retrained using the updated training sample set to update the model parameters of the garbage classification model. The updated model parameters are loaded into the waste sorting model to obtain the iteratively optimized waste sorting model.

7. A deep learning-based waste sorting system for implementing the waste sorting method according to any one of claims 1-6, characterized in that, include: The training sample construction component is used to acquire historical garbage classification data and construct a training sample set for model training. The waste sorting and recognition component is used to classify and recognize waste images using a deep learning model and extract the appearance and morphological features of the waste. The model optimization and update component is used to dynamically optimize the waste sorting model based on changes in the appearance and morphology of waste.

8. A waste sorting system based on deep learning according to claim 7, characterized in that, The training sample construction component specifically includes: The historical data acquisition unit is used to acquire historical waste sorting data and extract historical waste images corresponding to different waste sorting categories. The image preprocessing unit is used to preprocess historical junk images, remove junk images that do not meet the requirements, and perform at least one of the following processing on the retained junk images: size unification, pixel value adjustment, and image enhancement. The tag association unit is used to determine the classification tag corresponding to each historical waste image based on the waste classification standards, and to establish an association relationship between the classification tag and the corresponding historical waste image. The temporal information construction unit is used to obtain the collection time information corresponding to historical garbage images and associate the collection time information with training samples to form a training sample set containing images, classification labels and collection time information.

9. A waste sorting system based on deep learning according to claim 8, characterized in that, The waste sorting and identification component specifically includes: Deep learning model building unit, used to build a deep learning model that includes an input layer, a feature extraction layer, a classification branch, and a morphological feature extraction branch; The model training unit is used to input the training sample set into the deep learning model, calculate the classification loss through the classification labels, update the model parameters, and obtain the garbage classification model. The waste identification unit is used to acquire images of waste to be classified, input them into the waste classification model, output the corresponding waste classification results, and extract the corresponding morphological feature vectors. The result verification unit is used to obtain the actual garbage classification results, verify the garbage classification results output by the model, and use the verified classification results and corresponding images as new training samples.

10. A waste sorting system based on deep learning according to claim 9, characterized in that, The model optimization and update component specifically includes: The temporal feature analysis unit is used to associate the classification labels, morphological feature vectors and collection time information in the training samples to form waste classification temporal feature data, and to analyze the changes in the morphological feature distribution of different waste categories with the collection time. The morphological drift detection unit is used to compare the morphological feature vector corresponding to the current garbage image with the historical sample range of the corresponding garbage classification category. The model update decision unit is used to determine the model update requirement based on the new training samples and morphological drift warning information, and to select the model update method. The model iteration unit is used to update the parameters of the waste classification model by incremental training or retraining to obtain the iteratively optimized waste classification model.

Images