Intelligent kitchen waste identification and classification method

Abstract

The present application relates to big data and machine learning technical field, especially to a kind of intelligent kitchen garbage identification classification method, comprising the following steps: S1: building kitchen garbage multi-label classification data set, including the temperature and humidity data of several different categories of kitchen garbage and corresponding kitchen garbage image of each several image;S2: building multi-label kitchen garbage classification model, image feature extraction module extracts image feature map, environmental feature extraction module extracts temperature and humidity characteristic value, the feature map extracted is sent into data convolution module again to adapt image capture area and sensor sensing area;S3: using kitchen garbage multi-label classification data set trains multi-label kitchen garbage classification model;S4: by collecting in garbage disposal place the kitchen garbage data to be predicted, model loading module is used to use the multi-label kitchen garbage classification model obtained by training, and the kitchen garbage data to be predicted is classified by multi-label, improve the accuracy and classification efficiency of garbage classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data and machine learning, and particularly relates to an intelligent kitchen waste identification and classification method. BACKGROUND

[0002] In the current environmental protection and resource recycling field, the classification of kitchen waste is an important issue. However, the existing kitchen waste classification method mostly relies on manual classification, which is not only inefficient, but also easily affected by human factors, leading to inaccurate classification. In addition, with the acceleration of urbanization, the amount of kitchen waste has increased dramatically, and the traditional manual classification method has been unable to meet the demand of large-scale and efficient classification. In recent years, although some automatic classification methods based on computer vision and machine learning have been proposed, these methods often only focus on image features, ignoring environmental features such as temperature and humidity, which are also important for accurate classification of kitchen waste. Some kitchen waste may change under certain temperature and humidity conditions, affecting the classification results. The existing multi-label classification method also has some challenges in processing kitchen waste. Due to the variety of kitchen waste and the possibility of containing multiple types of waste in an image, how to accurately identify and classify these waste is a difficult problem. The existing multi-label classification method often has unsatisfactory classification results when dealing with images with complex backgrounds and occlusions. The construction of a data set is also a difficulty in the existing technology. Due to the diversity of kitchen waste types and forms, it is a difficult task to construct a comprehensive, accurate, and multi-class and multi-environmental feature data set. The quality of the data set directly affects the performance and accuracy of the multi-label kitchen waste classification model, and there is an urgent need for an intelligent kitchen waste identification and classification method that can consider both image features and environmental features and effectively handle multi-label classification problems.

[0003] Therefore, an intelligent kitchen waste identification and classification method is proposed. SUMMARY

[0004] The present application solves the problem that the current traditional kitchen waste classification cannot judge the kitchen waste according to the actual seasonal changes, lacks flexibility and adaptability, and consumes a large amount of human resources. The existing kitchen waste classification method cannot identify the changes that some kitchen waste may undergo under certain temperature and humidity conditions, affecting the classification results, and has the problem of low classification accuracy. In addition, the existing multi-label classification method has the problems of slow processing speed and low classification efficiency.

[0005] The technical solution adopted by the present application to solve its technical problems is:

[0006] An intelligent kitchen waste identification and classification method, comprising the following steps:

[0007] S1: Constructing a multi-label classification data set of kitchen waste, including a plurality of different categories of kitchen waste images and corresponding temperature and humidity data of kitchen waste, and the label of each image includes one or more categories;

[0008] S2: Constructing a multi-label kitchen waste classification model, including an image feature extraction module, a temperature and humidity feature extraction module, and a dynamic data convolution module; the image feature extraction module extracts a feature map from the input image of the multi-label kitchen waste classification model, the environmental feature extraction module extracts a feature value from the temperature and humidity parameters input into the multi-label kitchen waste classification model, and the extracted feature map is sent to the data convolution module for processing to adapt to the image capture area and the sensor sensing area;

[0009] S3: Training the multi-label kitchen waste classification model using the constructed multi-label classification data set of kitchen waste, including a data preparation module, a loss function selection module, an optimizer setting module, and a model training module;

[0010] S4: By collecting the predicted kitchen waste data at the garbage disposal place, the model loading module is used to use the trained multi-label kitchen waste classification model to perform multi-label classification on the predicted kitchen waste data.

[0011] Further, S1 specifically includes:

[0012] S11: By installing an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal place, the image acquisition unit, the temperature sensor and the humidity sensor are connected with the data processing module through a wireless communication protocol;

[0013] S12: The Zigbee module is used to periodically send the collected data to the data processing, and the data is integrated, filtered and denoised;

[0014] S13: The data processing module selects a plurality of different category combinations of kitchen waste images and temperature data and humidity data from different seasons according to the collected kitchen waste categories, obtains a multi-label classification data set of kitchen waste, and then transmits the multi-label classification data set of kitchen waste to the image feature extraction module and the temperature and humidity feature extraction module.

[0015] Further, S2 specifically includes:

[0016] S21: The image feature extraction module is configured to, after receiving the kitchen waste multi-label classification dataset, use a pre-trained convolutional neural network as an image feature extractor, extract a feature map of an image in the kitchen waste multi-label classification dataset according to data convolution operation, ReLU activation function formula and pooling operation formula in the convolutional neural network, and then transmit the extracted feature map of the image in the kitchen waste multi-label classification dataset to the data convolution module.

[0017] S22: The environment feature extraction module is configured to, after receiving the kitchen waste multi-label classification dataset, extract a temperature and humidity feature value vector according to a multi-layer perceptron in a preset neural network layer and temperature data and humidity data of the kitchen waste multi-label classification dataset, and then transmit the extracted temperature and humidity feature value vector of the kitchen waste multi-label classification dataset to the data convolution module.

[0018] S23: The data convolution module is configured to, after receiving the feature map of the image in the kitchen waste multi-label classification dataset and the temperature and humidity feature value vector, perform global average pooling on the image feature map, adjust a parameter of a convolution kernel according to the preset multi-layer perceptron input image feature map and temperature and humidity feature value vector, and perform convolution operation on the image feature map using the adjusted parameter of the convolution kernel to output an output feature map after the convolution operation to adapt to an image capture area and a sensor sensing area.

[0019] Further, the way of extracting the feature map of the image in the kitchen waste multi-label classification dataset according to the data convolution operation, the ReLU activation function and the pooling operation in the convolutional neural network is as follows:

[0020] Convolution operation:

[0021] C1 = ∑ m ∑ n I(i+m, j+n) * K(m+n);

[0022] ReLU activation function:

[0023] R1 = ReLU(C1);

[0024] Pooling operation:

[0025] F = Pool(R1) = max m∈M,n∈N R1(i+m, j+n);

[0026] Wherein, F is the extracted feature map of the image, C1 is the output after the convolution layer operation, R1 is the output of the ReLU activation function layer, I is the image in the kitchen waste multi-label classification dataset, K is the convolution kernel, (i, j) is the coordinate of the output feature map, m and n are the size of the convolution kernel, and M and N are the size of the pooling window.

[0027] Further, the formula for extracting the feature value vector of temperature and humidity according to the preset multi-layer perceptron in the neural network layer and the temperature data and humidity data of the kitchen waste multi-label classification data set is:

[0028] O j = δ (W (L) *Z (L-1) + b (L) ) ;

[0029] wherein, O j is the feature value vector extracted from the temperature data and humidity data of the kitchen waste multi-label classification data set, δ is the ReLU activation function, W (L) is the weight of the Lth layer, and W (L) ∈ (0, 1), b (L) is the bias term, Z (L-1) is the output of the L-1th layer, and for the output layer, Z (0) = [T, H], T is the temperature vector, T = [t1, t2, …, t n ], t n is the temperature data of the nth day, with the unit of Celsius degree, H is the humidity vector H = [h1, h2, …, h n ], h n is the humidity data of the nth day, with the unit of RH%.

[0030] Further, S23 specifically comprises:

[0031] S231: the process of global average pooling of the image feature map is:

[0032] F total = GlobalAvgPool (F) ;

[0033] wherein, F total ∈ R C , is the statistical representation of the image feature map, F ∈ R C*Hi*W , F is the feature map of the image, wherein, C is the channel number, Hi is the height, W is the width, and R represents the real number set;

[0034] S232: adjusting the parameters of the convolution kernel according to the preset multi-layer perceptron input image feature map and temperature and humidity feature value vector:

[0035] W adapt = ParameterGenerationNetwork (F total , O j ) ;

[0036] wherein, W adapt is the adjusted convolution kernel parameter, F total ∈ RC is a statistical representation of the image feature map, O j ∈R D is a temperature and humidity feature value vector, R is a real number set, C and D are channel numbers;

[0037] S233: using the adjusted convolution kernel to perform convolution operation on the image feature map, outputting the output feature map after the convolution operation to adaptively capture the image capture area and the sensor sensing area:

[0038] FO promt = Conv2D(F total , W adapt );

[0039] wherein, FO promt is the output feature map after the output convolution operation, F total is a statistical representation of the image feature map, and W adapt is the adjusted convolution kernel parameter.

[0040] Further, S3 specifically comprises:

[0041] S31: the data preparation module loads the kitchen waste multi-label classification data set after receiving the kitchen waste multi-label classification data set; and is used for pre-processing the image data in the kitchen waste multi-label classification data set; and is used for dividing the pre-processed kitchen waste multi-label classification data set into a training set, a validation set and a test set, and then transmitting the training set, the validation set and the test set data to the model training module;

[0042] S32: the loss function selection module is used to select a binary cross entropy loss function to handle the independence of each class label;

[0043] S33: the optimizer setting module is used to select an Adam algorithm to update the weights of the multi-label kitchen waste classification model;

[0044] S34: the model training module is used to iteratively train the multi-label kitchen waste classification model according to the received pre-processed training set, validation set and test set data after receiving the pre-processed training set, validation set and test set data, calculate the loss through forward propagation, and update the weights through back propagation; and is used for dividing the data classified by the multi-label kitchen waste classification model in the test set into positive sample data and negative sample data for each garbage type; and is used for calculating the classification accuracy of each garbage type according to the positive sample data and the negative sample data of each garbage type.

[0045] Further, the formula for calculating the classification accuracy of each garbage type according to the positive sample data and the negative sample data of each garbage type is:

[0046]

[0047] wherein P c is the classification precision of type c, TP c is the positive sample data correctly predicted by the multi-label kitchen waste classification model as class c, FP c is the negative sample data incorrectly predicted by the multi-label kitchen waste classification model as class c.

[0048] Further, S4 specifically comprises:

[0049] S41: collecting image data and temperature and humidity data of the to-be-predicted kitchen waste at the kitchen waste disposal site by installing an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal site;

[0050] S42: the data preparation module receives the image data and temperature and humidity data of the to-be-predicted kitchen waste, pre-processes the image data and temperature and humidity data of the to-be-predicted kitchen waste, and then transmits the pre-processed image data and temperature and humidity data of the to-be-predicted kitchen waste to the model loading module;

[0051] S43: the model loading module is configured to load the trained multi-label kitchen waste classification model for classification and recognition after receiving the pre-processed image data and temperature and humidity data of the to-be-predicted kitchen waste; and is configured to compare the class probability output by the multi-label kitchen waste classification model, the data precision of each class, the classification precision of each garbage type and the preset probability threshold, and determine the kitchen waste classification result.

[0052] Further, the comparison of the class probability output by the multi-label kitchen waste classification model and the classification precision of each garbage type and the preset probability threshold in S43 is as follows:

[0053]

[0054] wherein, is the prediction probability of class c output by the multi-label kitchen waste classification model, P most is the preset prediction probability threshold of class c and T is in (0, 1), P c is the classification precision of class c, and when the Result output is true, the garbage type of class c is determined.

[0055] The beneficial effects of the present application are:

[0056] 1. The image and temperature and humidity data of the kitchen waste can be comprehensively collected to form a data set containing multi-label classification information. This helps the multi-label kitchen waste classification model to learn the characteristics of kitchen waste in different seasons and different environments, and improves the accuracy and generalization ability of classification.

[0057] 2. The image feature extraction module efficiently extracts key features from images, providing strong support for classification. The temperature and humidity feature extraction module enables the multi-label food waste classification model to comprehensively consider the impact of environmental factors on food waste classification. The data convolution module improves the adaptability of the multi-label food waste classification model to complex scenarios.

[0058] 3. By calculating channel descriptors, learning attention weights, and applying these weights to feature maps, we can focus on more important regions and channels in the image, thereby improving classification accuracy.

[0059] 4. By collecting and preprocessing data in real time, a pre-trained multi-label kitchen waste classification model is loaded, providing reliable input for the multi-label kitchen waste classification model. By comparing the output category probability identified by the multi-label kitchen waste classification model with the preset probability threshold, the efficiency of kitchen waste identification can be improved. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. The drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.

[0061] In the attached diagram:

[0062] Figure 1 This is a schematic diagram of an intelligent kitchen waste identification and classification method according to an embodiment of the present invention. Detailed Implementation

[0063] The following will describe the concept and technical effects of the present invention clearly and completely with reference to the embodiments, so as to fully understand the purpose, features and effects of the present invention.

[0064] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when describing a specific feature, structure, or characteristic in conjunction with embodiments, implementing such a feature, structure, or characteristic in conjunction with other embodiments should be within the knowledge of those skilled in the art.

[0065] In general, terminology can be understood at least in part from the context of usage. For example, the term "one or more" as used herein, depending at least in part upon context, can be used to describe any feature, structure, or characteristic in a singular sense or can be used to describe combinations of features, structures or characteristics in a plural sense. Additionally, the term "based on" can be understood as not necessarily being confined to factors that are clearly associated with a relevant art field described herein.

[0066] See Figure 1 ,

[0067] An intelligent kitchen waste identification and classification method, comprising the following steps:

[0068] S1: Constructing a kitchen waste multi-label classification data set, including a plurality of different categories of kitchen waste images and corresponding temperature and humidity data of the kitchen waste, and the label of each image includes one or more categories;

[0069] S2: Constructing a multi-label kitchen waste classification model, including an image feature extraction module, a temperature and humidity feature extraction module, and a dynamic data convolution module; the image feature extraction module extracts a feature map from the input image of the multi-label kitchen waste classification model, the environmental feature extraction module extracts a feature value from the temperature and humidity parameters input into the multi-label kitchen waste classification model, and the extracted feature map is sent to the data convolution module for processing to adapt to the image capture area and the sensor sensing area;

[0070] S3: Training the multi-label kitchen waste classification model using the constructed kitchen waste multi-label classification data set, including a data preparation module, a loss function selection module, an optimizer setting module, and a model training module;

[0071] S4: Collecting the kitchen waste data to be predicted at the garbage disposal place, and the model loading module is used to use the trained multi-label kitchen waste classification model to perform multi-label classification on the kitchen waste data to be predicted.

[0072] In the embodiment, S1 specifically comprises:

[0073] S11: Installing an image acquisition unit, a temperature sensor, a humidity sensor, and a Zigbee module at the kitchen waste disposal place, and the image acquisition unit, the temperature sensor, and the humidity sensor are connected to the data processing module through a wireless communication protocol;

[0074] S12: The Zigbee module is used to periodically send the collected data to the data processing module, and the data is integrated, filtered, and denoised;

[0075] S13: The data processing module selects multiple different category combinations of kitchen waste images and temperature and humidity data from different time periods of a year according to the collected categories of kitchen waste based on different seasons, obtains a kitchen waste multi-label classification data set, and then transmits the kitchen waste multi-label classification data set to the image feature extraction module and the temperature and humidity feature extraction module.

[0076] Through the combination of the above steps, the sensor and the image acquisition unit are connected with the data processing module through a wireless communication protocol, ensuring real-time transmission and efficient processing of data. Through data integration, data from different sensors and image acquisition units can be uniformly managed and analyzed. The data processing module selects multiple different category combinations of kitchen waste images and temperature and humidity data from different time periods of a year according to the collected categories of kitchen waste based on different seasons, which ensures that the data set contains rich category information and the influence of seasonal changes on kitchen waste, providing strong data support for subsequent multi-label kitchen waste classification model training. The kitchen waste multi-label classification data set is transmitted to the image feature extraction module and the temperature and humidity feature extraction module, providing necessary inputs for subsequent multi-label kitchen waste classification model training.

[0077] In this embodiment, S2 specifically comprises:

[0078] S21: The image feature extraction module is configured to, after receiving the kitchen waste multi-label classification data set, use a pre-trained convolutional neural network as an image feature extractor, extract feature maps of images in the kitchen waste multi-label classification data set according to data convolution operations, ReLU activation function formulas, and pooling operation formulas in the convolutional neural network, and then transmit the extracted feature maps of images in the kitchen waste multi-label classification data set to the data convolution module.

[0079] S22: The environmental feature extraction module is configured to, after receiving the kitchen waste multi-label classification data set, extract feature value vectors of temperature and humidity according to a plurality of multilayer perceptrons in a preset neural network layer and the temperature and humidity data of the kitchen waste multi-label classification data set, and then transmit the extracted feature value vectors of temperature and humidity of the kitchen waste multi-label classification data set to the data convolution module.

[0080] S23: The data convolution module is configured to, after receiving the feature maps of images and the feature value vectors of temperature and humidity in the kitchen waste multi-label classification data set, perform global average pooling on the image feature maps; and is configured to adjust parameters of a convolution kernel according to the input image feature maps and the temperature and humidity feature value vectors of the preset multilayer perceptron; and is configured to perform convolution operations on the image feature maps using the adjusted convolution kernel, output the output feature maps after the convolution operations, and adapt the image capture area and the sensor sensing area.

[0081] Through the combination of the above steps, the environmental feature extraction module extracts the feature value vector of temperature and humidity through the neural network layer, considers the influence of environmental factors on kitchen waste classification, improves the generalization ability of the multi-label kitchen waste classification model, and the data convolution module receives the feature maps and feature values from the image feature extraction module and the environmental feature extraction module, realizes the fusion of features. Introducing the attention mechanism or adaptive convolution layer, the data convolution module can dynamically adjust the parameters of the convolution kernel according to the image feature map, adaptively focus on the key areas in the image, and improve the classification accuracy. The data convolution module can also dynamically adjust according to the temperature and humidity feature values to adapt to the changes in the sensor sensing area, further improving the adaptability of the multi-label kitchen waste classification model.

[0082] In this embodiment, the way to extract the feature map of the image in the kitchen waste multi-label classification dataset according to the data convolution operation in the convolutional neural network, the ReLU activation function and the pooling operation is as follows:

[0083] Convolution operation:

[0084] C1 = ∑ m ∑ n I(i+m, j+n)*K(m+n);

[0085] ReLU activation function:

[0086] R1 = ReLU(C1);

[0087] Pooling operation:

[0088] F = Pool(R1) = max m∈M,n∈N R1(i+m, j+n);

[0089] Wherein, F is the extracted feature map of the image, C1 is the output after the convolution layer operation, R1 is the output of the ReLU activation function layer, I is the image in the kitchen waste multi-label classification dataset, K is the convolution kernel; (i, j) is the coordinate of the output feature map, m, n is the size of the convolution kernel, M and N are the size of the pooling window.

[0090] In this embodiment, the formula for extracting the feature value vector of temperature and humidity according to the multi-layer perceptron in the preset neural network layer and the temperature data and humidity data of the kitchen waste multi-label classification dataset is as follows:

[0091] O j = δ(W (L) *Z (L-1) )+b (L) );

[0092] Wherein, O jis the feature value vector of temperature data and humidity data extracted from the multi-label classification data set of kitchen waste, δ is the ReLU activation function, W (L) is the weight of the L-th layer, and W (L) (L) is a bias term, Z (L-1) is the output of the L-1-th layer, and for the output layer, Z (0) = [T, H], T is a temperature vector, T = [t1, t2, …, t n ], t n is the temperature data of the nth day, with the unit of Celsius, and H is a humidity vector H = [h1, h2, …, h n ], h n is the humidity data of the nth day, with the unit of RH%.

[0093] In this embodiment, S23 specifically comprises:

[0094] S231: The process of global average pooling of the image feature map is:

[0095] F total = GlobalAvgPool (F);

[0096] wherein F total ∈ R C is a statistical representation of the image feature map, F ∈ R C*Hi*W , F is the feature map of the image, wherein C is the number of channels, Hi is the height, W is the width, and R represents the real number set;

[0097] S232: Adjusting the parameters of the convolution kernel according to the preset multi-layer perceptron input image feature map and the temperature and humidity feature value vector:

[0098] W adapt = ParameterGenerationNetwork (F total , O j );

[0099] wherein W adapt is the adjusted convolution kernel parameter, F total ∈ R C is a statistical representation of the image feature map, O j ∈ R D is the temperature and humidity feature value vector, R is the real number set, and C and D are the number of channels;

[0100] S233: Convolution operation is performed on the image feature map using the adjusted parameters of the convolution kernel, and the output feature map after the convolution operation is output to adaptively capture the image capture area and the sensor sensing area:

[0101] FO​promt = Conv2D (F total , W adapt ) ;

[0102] wherein, FO promt is the output feature map after the output of the convolution operation, F total is the statistical representation of the image feature map, and W adapt is the adjusted convolution kernel parameter.

[0103] In this embodiment, S3 specifically comprises:

[0104] S31: The data preparation module loads the kitchen waste multi-label classification dataset after receiving the kitchen waste multi-label classification dataset; and is configured to preprocess the image data in the kitchen waste multi-label classification dataset; and is configured to divide the preprocessed kitchen waste multi-label classification dataset into a training set, a validation set and a test set, and then transmit the training set, the validation set and the test set data to the model training module;

[0105] S32: The loss function selection module is configured to select a binary cross-entropy loss function to handle the independence of each class label;

[0106] S33: The optimizer setting module is configured to select an Adam algorithm to update the weights of the multi-label kitchen waste classification model;

[0107] S34: The model training module is configured to iteratively train the multi-label kitchen waste classification model according to the received preprocessed training set, validation set and test set data after receiving the preprocessed training set, validation set and test set data, calculate the loss by forward propagation, and update the weights by back propagation; and is configured to divide the data classified by the multi-label kitchen waste classification model in the test set into positive sample data and negative sample data for each garbage type; and is configured to calculate the classification accuracy of each garbage type according to the positive sample data and the negative sample data of each garbage type.

[0108] Through the combination of the above steps, the data preparation module pre-processes the image data, the loss function selection module selects the binary cross-entropy loss function to handle the independence of each class label, the prediction error of each label is calculated independently, the Adam algorithm is an optimization algorithm based on gradient descent, with the characteristics of adaptive learning rate adjustment, which can efficiently find the global optimal solution, and the model training module receives the pre-processed training set, validation set and test set data, and optimizes the multi-label kitchen waste classification model parameters through iterative training. During the training process, the model training module calculates the loss by forward propagation, updates the weights by back propagation, and calculates the precision of each class of data, which is one of the important indicators for evaluating the performance of the multi-label kitchen waste classification model, and helps to understand the performance of the multi-label kitchen waste classification model in each class.

[0109] In this embodiment, the formula for calculating the classification precision of each garbage type according to the positive sample data and negative sample data of each garbage type is:

[0110]

[0111] where P c is the classification precision of type c, TP c is the positive sample data correctly predicted by the multi-label kitchen waste classification model as class c, FP c is the negative sample data incorrectly predicted by the multi-label kitchen waste classification model as class c.

[0112] In this embodiment, S4 specifically comprises:

[0113] S41: Collecting image data and temperature and humidity data of the predicted kitchen waste by installing an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal site;

[0114] S42: The data preparation module receives the image data and temperature and humidity data of the predicted kitchen waste, pre-processes the image data and temperature and humidity data of the predicted kitchen waste, and then transmits the pre-processed image data and temperature and humidity data of the predicted kitchen waste to the model loading module;

[0115] S43: The model loading module is used to load the trained multi-label kitchen waste classification model for classification and identification after receiving the pre-processed image data and temperature and humidity data of the predicted kitchen waste; and is used to compare the output class probability of the multi-label kitchen waste classification model, the precision of each class of data, the classification precision of each garbage type and the preset probability threshold to determine the kitchen waste classification result.

[0116] Through the combination of the above steps, the model loading module can quickly load the trained multi-label kitchen garbage classification model, and use the multi-label kitchen garbage classification model to classify and identify the pre-processed kitchen garbage data. By directly inputting the pre-processed data into the multi-label kitchen garbage classification model, the repeated labor of data processing is avoided, and the speed of classification and identification is improved. The model loading module classifies according to the output class probability of the multi-label kitchen garbage classification model, combines the data of each class to calculate the precision, and compares it with the preset probability threshold to determine the final kitchen garbage classification result. The output probability of the multi-label kitchen garbage classification model and the characteristics of the data itself are considered, and the accuracy and reliability of the classification result are improved.

[0117] In the present embodiment, the way of comparing the class probability output by the multi-label kitchen garbage classification model with the classification accuracy of each garbage type and the preset probability threshold in S43 is as follows:

[0118]

[0119] wherein, is the predicted probability of class c output by the multi-label kitchen garbage classification model, P most is the preset predicted probability threshold of class c and T e (0, 1), P c is the classification accuracy of class c, and when the Result output is true, the garbage type of class c is determined.

[0120] In summary, the whole method realizes the intelligent processing of kitchen waste classification, and by installing an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal place, a reliable data source is provided for subsequent data processing and classification recognition. The data preparation module receives the original data from the sensor, performs preprocessing operation to improve the data quality and multi-label kitchen waste classification model recognition effect, and divides the data set into training set, validation set and test set, which provides convenience for subsequent multi-label kitchen waste classification model training and performance evaluation. The preprocessed training set data is used to iteratively train the multi-label kitchen waste classification model. A suitable loss function and optimizer are selected to optimize the performance of the multi-label kitchen waste classification model. It is ensured that the multi-label kitchen waste classification model can accurately identify the category of kitchen waste, and the model loading module determines the final kitchen waste classification result by combining the category probability output by the multi-label kitchen waste classification model, the precision rate of each category and the preset probability threshold. This step realizes the rapid and accurate classification of kitchen waste. The whole method forms a complete data processing chain from data acquisition, preprocessing, multi-label kitchen waste classification model training to classification recognition, and has high practicability and scalability. In addition, the method can also be combined with other intelligent technologies, such as Internet of Things, big data, etc., to further improve the intelligent level of garbage classification.

[0121] The above embodiments are only part of the embodiments of the present application, not all embodiments, and other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative labor belong to the protection scope of the present application.

Claims

1. A smart kitchen waste identification and classification method, characterized in that: The method comprises the following steps: S1: constructing a kitchen waste multi-label classification data set, including a plurality of different categories of kitchen waste images and corresponding temperature and humidity data of the kitchen waste, and the label of each image including one or more categories; S2: constructing a multi-label kitchen waste classification model, including an image feature extraction module, a temperature and humidity feature extraction module, and a dynamic data convolution module; the image feature extraction module extracts a feature map from an image input into the multi-label kitchen waste classification model, the environment feature extraction module extracts a feature value from a temperature and humidity parameter input into the multi-label kitchen waste classification model, and the extracted feature map is sent to the data convolution module for processing to adapt to the image capture area and the sensor sensing area; S3: training the multi-label kitchen waste classification model using the constructed kitchen waste multi-label classification data set, including a data preparation module, a loss function selection module, an optimizer setting module, and a model training module; S4: collecting kitchen waste data to be predicted at a garbage disposal site, and a model loading module is used to use the trained multi-label kitchen waste classification model to perform multi-label classification on the kitchen waste data to be predicted; The S2 specifically comprises: S21: the image feature extraction module is used to use a pre-trained convolutional neural network as an image feature extractor after receiving the kitchen waste multi-label classification data set, extract a feature map of an image in the kitchen waste multi-label classification data set according to data convolution operations, ReLU activation function formulas, and pooling operation formulas in the convolutional neural network, and then send the extracted feature map of the image in the kitchen waste multi-label classification data set to the data convolution module; S22: the environment feature extraction module is used to extract a temperature and humidity feature value vector according to a plurality of multilayer perceptrons in a preset neural network layer and temperature data and humidity data of the kitchen waste multi-label classification data set after receiving the kitchen waste multi-label classification data set, and then transmit the extracted temperature and humidity feature value vector of the kitchen waste multi-label classification data set to the data convolution module; S23: the data convolution module is used to perform global average pooling on the image feature map after receiving the image feature map and the temperature and humidity feature value vector in the kitchen waste multi-label classification data set; and is used to adjust parameters of a convolution kernel according to the image feature map and the temperature and humidity feature value vector input into the preset multilayer perceptron; and is used to perform convolution operations on the image feature map using the adjusted parameters of the convolution kernel, and output an output feature map after the convolution operations to adapt to the image capture area and the sensor sensing area; The S23 specifically comprises: S231: the process of performing global average pooling on the image feature map is: ； wherein, is a statistical representation of the feature map of the image, , is a feature map of the image, wherein, is the number of channels, is the height, is the width, R denotes the set of real numbers; S232: adjusting parameters of a convolution kernel according to the image feature map and the temperature and humidity feature value vector input into the preset multilayer perceptron: ； wherein, is an adjusted convolution kernel parameter, is a statistical representation of the image feature map, is a temperature and humidity feature value vector, R is a real number set, and C and D are channel numbers. S233: performing convolution operations on the image feature map using the adjusted parameters of the convolution kernel, and outputting an output feature map after the convolution operations to adapt to the image capture area and the sensor sensing area: ； wherein, is an output feature map after an output convolution operation, is a statistical representation of the image feature map, is an adjusted convolution kernel parameter. 2.The intelligent kitchen waste identification and classification method of claim 1, wherein: The S1 specifically comprises: S11: install an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal site, wherein the image acquisition unit, the temperature sensor and the humidity sensor are connected to a data processing module through a wireless communication protocol; S12: the Zigbee module is used for periodically sending the collected data to the data processing module, and the data is integrated, filtered and denoised; S13: the data processing module selects a plurality of different category combinations of kitchen waste images and temperature data and humidity data from different time periods of a year according to the collected kitchen waste category based on different seasons, to obtain a kitchen waste multi-label classification data set, and then transmits the kitchen waste multi-label classification data set to an image feature extraction module and a temperature and humidity feature extraction module. 3.The intelligent kitchen waste identification and classification method of claim 2, wherein: The way of extracting the feature map of the image in the kitchen waste multi-label classification data set according to the data convolution operation in the convolutional neural network, the ReLU activation function and the pooling operation is: Convolution operation: ； ReLU activation function: ； Pooling operation: ； wherein, is a feature map of the extracted image, is an output after a convolutional layer operation, is an output of a ReLU activation function layer, is an image in a kitchen waste multi-label classification dataset, is a convolutional kernel; is a coordinate of the output feature map, , is a size of the convolutional kernel, and is a size of the pooling window.

4. The intelligent kitchen waste identification and classification method according to claim 3, characterized in that: The formula for extracting the feature value vector of temperature and humidity according to the multi-layer perceptron in the preset neural network layer and the temperature data and humidity data of the kitchen waste multi-label classification data set is: ； wherein, is a feature vector of temperature data and humidity data extracted from the kitchen waste multi-label classification dataset, is a ReLU activation function, is a weight of the Lth layer, and (0, 1), is a bias term, is an output of the Lth layer, and for the output layer, , , is a temperature vector, , is temperature data of the nth day in Celsius, is a humidity vector , is humidity data of the nth day in RH%.

5. The intelligent kitchen waste identification and classification method according to claim 4, characterized in that: S3 specifically includes: S31: the data preparation module loads the kitchen waste multi-label classification data set after receiving the kitchen waste multi-label classification data set, and is used for pre-processing the image data in the kitchen waste multi-label classification data set, and is used for dividing the pre-processed kitchen waste multi-label classification data set into a training set, a validation set and a test set, and then transmitting the training set, the validation set and the test set data to a model training module; S32: the loss function selection module is used for selecting a binary cross-entropy loss function to process the independence of each category label; S33: the optimizer setting module is used for selecting an Adam algorithm to update the weight of the multi-label kitchen waste classification model; S34: the model training module is used for iteratively training the multi-label kitchen waste classification model according to the received pre-processed training set, validation set and test set data after receiving the pre-processed training set, validation set and test set data, calculating the loss through forward propagation, and updating the weight through back propagation; and is used for dividing the data classified by the multi-label kitchen waste classification model in the test set into positive sample data and negative sample data; and is used for calculating the classification accuracy of each garbage type according to the positive sample data and the negative sample data of each garbage type. 6.The intelligent kitchen waste identification and classification method of claim 5, wherein: The formula for calculating the classification accuracy of each garbage type according to the positive sample data and the negative sample data of each garbage type is: ； wherein, is the classification precision of type c, is the positive sample data correctly predicted by the multi-label kitchen waste classification model as class c, is the negative sample data incorrectly predicted by the multi-label kitchen waste classification model as class c.

7. The intelligent kitchen waste identification and classification method of claim 6, wherein: S4 specifically includes: S41: install an image acquisition unit, a temperature sensor, a humidity sensor and a Zigbee module at the kitchen waste disposal site to collect image data and temperature and humidity data of the kitchen waste to be predicted at the disposal site; S42: The data preparation module receives the image data and temperature and humidity data of the kitchen garbage to be predicted, pre-processes the image data and temperature and humidity data of the kitchen garbage to be predicted, and then transmits the pre-processed image data and temperature and humidity data of the kitchen garbage to be predicted to the model loading module; S43: The model loading module is configured to load the trained multi-label kitchen garbage classification model for classification and identification after receiving the pre-processed image data and temperature and humidity data of the kitchen garbage to be predicted; and is configured to compare the class probability output by the multi-label kitchen garbage classification model, the precision of each class, the classification accuracy of each garbage type, and a preset probability threshold to determine the kitchen garbage classification result. 8.The intelligent kitchen waste identification and classification method of claim 7, wherein: The comparison between the class probability output by the multi-label kitchen garbage classification model and the classification accuracy of each garbage type and the preset probability threshold in S43 is performed in the following manner: ； wherein, is a predicted probability of a class output by the multi-label kitchen waste classification model, is a preset predicted probability threshold of the class and 0,1), is a classification precision of the class c when the output result is the waste type of the class is determined.​

Images