Garbage classification model training method, classification method, device and readable storage medium

Abstract

The present invention discloses a garbage classification model training method, a classification method, a device and a readable storage medium, and relates to artificial intelligence technology. The garbage classification model training method comprises: obtaining image data of associated garbage; constructing a mapping to obtain a first data set; inputting the first data set into a deep convolutional neural network for training to construct a pre-trained model; making a second data set; sharing the weights of the first data set and the second data set in a preset ratio to generate a feature map, the feature map includes a third data set and a fourth data set; inputting the fourth data set into a residual network branch to obtain a first output; inputting the third data set and the fourth data set into a common network branch to obtain a second output and a third output respectively; adding the first output and the second output and inputting them into a first classifier, and inputting the third output into a second classifier; iterating until the network converges to obtain a garbage classification model. The present invention can take into account both the cost of manual classification and the accuracy of garbage classification by the machine.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to a garbage classification model training method, a classification method, a device and a readable storage medium. Background Art

[0002] With the continuous deepening of environmental protection concepts, garbage classification has now become one of the symbols of urban civilization. Garbage classification can not only reduce the pollution of garbage to the environment, but also turn waste into treasure, make full use of materials, reduce the consumption of land resources, and has social, economic, and ecological benefits.

[0003] At present, garbage classification mainly relies on manual distinction. First, people consciously classify garbage into recyclable garbage, kitchen waste, hazardous waste, etc., and then they have to undergo manual subdivision when they arrive at the garbage disposal station. However, this method has high labor costs and is not conducive to the long-term development of the garbage classification industry. In addition, in existing garbage classification schemes based on artificial intelligence, a classification model is usually constructed by capturing images from a large number of web pages related to garbage. However, there is a problem of inaccurate classification caused by large background noise and label noise. Summary of the invention

[0004] The present invention aims at the problem that the cost and accuracy of existing garbage classification are not taken into account at the same time, and provides a garbage classification model training method, a classification method, a device and a readable storage medium.

[0005] The technical solution proposed by the present invention for the above-mentioned technical problem is as follows:

[0006] In a first aspect, the present invention provides a method for training a garbage classification model, the method comprising:

[0007] Get image data associated with garbage;

[0008] Acquire partial image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the partial image data and first labels corresponding to the first samples;

[0009] Inputting the first data set into a deep convolutional neural network for training to construct a pre-trained model;

[0010] Using another portion of the image data in the image data to create a second data set, the second data set includes a plurality of second samples in the another portion of the image data and respective second labels obtained by inputting respective second samples into the pre-trained model output;

[0011] The first data set and the second data set are weighted in a preset ratio to generate a feature map, wherein the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set;

[0012] Input the fourth data set into the residual network branch to obtain a first output; input the third data set and the fourth data set into the common network branch respectively to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet;

[0013] The first output and the second output are added and input into a first classifier, and the third output is input into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function;

[0014] The training process is repeated until the network converges to obtain a trained garbage classification model.

[0015] According to the above garbage classification model training method, the step of obtaining image data related to garbage includes:

[0016] Crawl a large amount of image data related to junk files from the Internet; and / or obtain image data related to junk files from an image library.

[0017] According to the above-mentioned garbage classification model training method, the preset ratio is 1:9.

[0018] According to the above garbage classification model training method, the cross entropy loss function of the first classifier is:

[0019]

[0020] Among them, x i Represented as a sample, y i Expressed as x i The corresponding label, G represents the first classifier, and n represents the number of samples.

[0021] According to the above garbage classification model training method, the cross entropy loss function of the second classifier is:

[0022]

[0023] Among them, H represents the second classifier.

[0024] The cross entropy loss function of the first classifier and the cross entropy loss function of the second classifier satisfy the following relationship:

[0025] L = arg min {αLG +(1-α)L H}, α∈(0,1)

[0026] Among them, α is the weight coefficient and L is the total loss function.

[0027] In a second aspect, the present invention provides a garbage classification method using the garbage classification model training method as described above, the method comprising:

[0028] Acquire an image to be analyzed;

[0029] Using the garbage classification model to determine the type of garbage in the image to be analyzed;

[0030] If the type of garbage in the image to be analyzed is determined, the garbage type information is output.

[0031] According to the above garbage classification method, the method further includes:

[0032] According to the garbage type information, a preset device is controlled to transport the garbage to a preset location.

[0033] In a third aspect, the present invention further provides a garbage classification model training device, the device comprising:

[0034] An acquisition module, used for acquiring image data associated with garbage;

[0035] A first data set making module, configured to obtain partial image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the partial image data and first labels corresponding to the first samples;

[0036] A model pre-construction module, used for inputting the first data set into a deep convolutional neural network for training to construct a pre-trained model;

[0037] A second data set making module, used to make a second data set using another part of the image data, wherein the second data set includes a plurality of second samples in the another part of the image data and second labels obtained by inputting each of the second samples into the pre-trained model output;

[0038] a feature map generation module, configured to share weights of the first data set and the second data set according to a preset ratio to generate a feature map, wherein the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set;

[0039] An input module, configured to input the fourth data set into a residual network branch to obtain a first output; input the third data set and the fourth data set into a common network branch to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet;

[0040] An output module, used for adding the first output and the second output and inputting the sum into a first classifier, and inputting the third output into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function;

[0041] The iteration module is used to iterate until the network converges to obtain a trained garbage classification model.

[0042] In a fourth aspect, the present invention further provides a computer, wherein the computer device comprises a processor, and the processor is used to implement the garbage classification model training method and / or the steps in the garbage classification method as described above when executing a computer program stored in a memory.

[0043] In a fifth aspect, the present invention also provides a readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, it implements the garbage classification model training method as described above and / or the steps in the garbage classification method as described above.

[0044] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0045] Compared with the traditional deep learning-based garbage classification model training method that requires the use of massive amounts of labeled garbage data to train the garbage classification model, which has the problem of relatively inaccurate garbage classification caused by background noise and label noise, the garbage classification model training method provided by the present invention uses a pre-trained model to generate noise labels for massive amounts of image data related to garbage, and then sends the noise labels and a small amount of manually labeled data to two classifiers for training. The entire process only requires a small amount of manual labeling to train a reliable garbage classification model, improve the accuracy of garbage classification, take into account the cost of manual classification and the accuracy of machine garbage classification, and at the same time improve the generalization ability of the garbage classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A schematic diagram of a flow chart of a garbage classification model training method provided by the present invention in one embodiment;

[0048] Figure 2 A schematic diagram of a flow chart of a garbage classification method provided by the present invention in one embodiment;

[0049] Figure 3 This is a functional module block diagram of the garbage classification model training device provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0051] See also Figure 1 , is a flow chart of a garbage classification model training method provided by the present invention under one embodiment. The garbage classification model training method is based on machine vision and uses artificial intelligence to classify garbage. It is mainly used in garbage classification systems. By building a relatively efficient and accurate garbage classification model to determine the type of garbage, it is conducive to efficient identification of garbage types and garbage recycling, and helps environmental protection. In addition, the visual garbage classification model training method provided by the present invention is also based on weakly supervised learning.

[0052] like Figure 1 As shown, the garbage classification model training method may include the following steps:

[0053] S101: Obtaining image data associated with garbage, where the acquisition method may include crawling a large amount of image data associated with garbage from the Internet; and / or obtaining image data associated with garbage from an image library.

[0054] Here, there are a large number of images of related garbage, such as hazardous waste such as related batteries, light bulbs, mercury thermometers, kitchen waste such as leftover rice, leftovers, fruit peels, and recyclable garbage such as cardboard boxes, foam, and plastic bottles.

[0055] S102: Acquire partial image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the partial image data and first labels corresponding to the first samples.

[0056] In this step, the first data set can be counted as D c {(x1,y1),(x2,y2),...,(x n ,y n )}, where x n Represents the nth sample, the corresponding label is y nIn the process of constructing the mapping, the image may be manually recognized, and the recognition result may be annotated as the first label after recognition to be associated with the corresponding image.

[0057] S103: Input the first data set into a deep convolutional neural network for training to construct a pre-trained model. Here, the deep convolutional neural network (CNN) is a type of feed-forward neural network that includes convolution calculations and has a deep structure. Inputting the first data set into the network for training can enable the pre-trained model to initially have the ability to judge the type of garbage.

[0058] S104: Using another part of the image data in the image data to create a second data set, the second data set includes multiple second samples in the other part of the image data and each second label obtained by inputting each of the second samples into the pre-trained model output.

[0059] In this step, another part of the image data (that is, the remaining part relative to the image data used to make the first data set) is input into the pre-trained model, and the second label generated for the corresponding second sample is a noisy label.

[0060] Here, the second data set can be calculated as D n {(x1,y1),(x2,y2),...,(x m ,y m )}, where x m Represents the mth sample, and the corresponding noise label is y m , preferably, n<<m.

[0061] S105: Share weights of the first data set and the second data set in a preset ratio to generate a feature map, where the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set.

[0062] In this step, the first sample in the first data set and the second sample in the second data set are scanned by the same convolution kernel. The weights are shared according to the preset ratio to extract features, and the garbage or garbage type features extracted from each sample can generate a garbage type feature map of the sample, that is, the garbage or garbage type feature map generated corresponding to the first sample can be used as the third sample in the third data set, and the corresponding first label remains unchanged; the garbage or garbage type feature map generated corresponding to the second sample can be used as the fourth sample in the fourth data set, and the corresponding second label also remains unchanged.

[0063] Here, the preset ratio may be 1:9, that is, for every input of a first sample and a first label in the first data set, nine second samples and second labels in the second data set are correspondingly input.

[0064] S106: Input the fourth data set into the residual network branch to obtain a first output; input the third data set and the fourth data set into the common network branch respectively to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet.

[0065] In this step, the residual network branch is used to learn the mapping from the feature space to the real label and the noise label to improve the accuracy of the label corresponding to the corresponding sample. The ordinary network branch is used to learn the mapping from the feature space to the real label space.

[0066] It is understandable that the common network branch may also adopt other neural networks such as GoogLeNet.

[0067] S107: Add the first output and the second output and input them into a first classifier, and input the third output into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function.

[0068] In this step, the cross entropy loss function of the first classifier can be:

[0069]

[0070] Among them, x i Represented as a sample, y i Expressed as x i The corresponding label, G represents the first classifier, and n represents the number of samples.

[0071] The cross entropy loss function of the second classifier can be:

[0072]

[0073] Among them, H represents the second classifier.

[0074] The cross entropy loss function of the first classifier and the cross entropy loss function of the second classifier satisfy the following relationship:

[0075] L = arg min {αL G +(1-α)L H}, α∈(0,1)

[0076] Among them, α is the weight coefficient and L is the total loss function.

[0077] By adjusting the weight of the cross entropy loss function of the first classifier and the second classifier, the total loss function value is minimized, that is, the label is closest to the real label.

[0078] S108: Iterate until the network converges to obtain a trained garbage classification model.

[0079] Compared with the traditional deep learning-based garbage classification model training method that requires the use of massive amounts of labeled garbage data to train the garbage classification model, which has the problem of relatively inaccurate garbage classification caused by background noise and label noise, the garbage classification model training method provided by the present invention uses a pre-trained model to generate noise labels for massive amounts of image data related to garbage, and then sends the noise labels and a small amount of manually labeled data to two classifiers for training. The entire process only requires a small amount of manual labeling to train a reliable garbage classification model, improve the accuracy of garbage classification, take into account the cost of manual classification and the accuracy of machine garbage classification, and at the same time improve the generalization ability of the garbage classification model.

[0080] See also Figure 2 , which is a flow chart of the garbage classification method provided by the present invention under one embodiment. The garbage classification method is mainly used in a garbage classification system. The acquired garbage images are quickly classified using the above-mentioned trained garbage classification model, which is also conducive to the efficient identification of garbage types and garbage recycling, and helps environmental protection.

[0081] like Figure 2 As shown, the garbage classification method may include the following steps:

[0082] S201: Acquire an image to be analyzed. Specifically, one or more cameras may be used to shoot pictures or videos as the image to be analyzed.

[0083] S202: Using the garbage classification model to determine the type of garbage in the image to be analyzed.

[0084] S203: If the garbage type in the image to be analyzed is determined, output garbage type information.

[0085] The garbage classification method provided by the present invention can quickly identify the types of garbage with a relatively high recognition accuracy, and can overcome the problem that traditional deep learning-based methods require the use of massive amounts of labeled garbage data to train garbage classification models, and that garbage classification is relatively inaccurate due to background noise and label noise.

[0086] The present invention also provides a garbage classification model training device, see Figure 3The garbage classification model training device 100 is used to obtain a garbage classification model and may include the following modules:

[0087] The acquisition module 11 is used to acquire image data associated with garbage.

[0088] The first data set making module 12 is used to obtain part of the image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the part of the image data and first labels corresponding to the first samples.

[0089] The model pre-construction module 13 is used to input the first data set into a deep convolutional neural network for training to construct a pre-trained model.

[0090] The second data set making module 14 is used to make a second data set using another part of the image data in the image data, wherein the second data set includes multiple second samples in the other part of the image data and each second label obtained by inputting each second sample into the pre-trained model output.

[0091] The feature map generation module 15 is used to share the weights of the first data set and the second data set in a preset ratio to generate a feature map, wherein the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set.

[0092] The input module 16 is used to input the fourth data set into the residual network branch to obtain a first output; input the third data set and the fourth data set into the common network branch respectively to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet.

[0093] The output module 17 is used to add the first output and the second output and input them into a first classifier, and input the third output into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function.

[0094] The iteration module 18 is used to iterate until the network converges to obtain a trained garbage classification model.

[0095] The modules cooperate with each other, so that the garbage classification model training device 100 can use the pre-trained model to generate noise labels for a large amount of image data related to garbage, and then send the noise labels and a small amount of manually labeled data to two classifiers for training. The whole process only requires a small amount of manual labeling to train a reliable garbage classification model, improve the accuracy of garbage classification, take into account the cost of manual classification and the accuracy of machine garbage classification, and at the same time improve the generalization ability of the garbage classification model.

[0096] It can be understood that the present invention can also provide a garbage classification device, which may include an image acquisition module, a judgment module and an information output module, wherein the image acquisition module is used to acquire the image to be analyzed; the judgment module is used to use the garbage classification model to determine the type of garbage in the image to be analyzed; and the information output module is used to output garbage type information when the type of garbage in the image to be analyzed is determined.

[0097] In addition, the present invention provides a computer device, which includes a processor, and the processor is used to implement the steps in the above-mentioned garbage classification model training method and garbage classification method when executing a computer program stored in a memory.

[0098] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0099] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0100] In addition, the present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the aforementioned garbage classification model training method and garbage classification method are implemented.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A garbage classification model training method, characterized in that: The method comprises: Get image data associated with garbage; Acquire partial image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the partial image data and first labels corresponding to the first samples; Inputting the first data set into a deep convolutional neural network for training to construct a pre-trained model; Using another portion of the image data in the image data to create a second data set, the second data set includes a plurality of second samples in the another portion of the image data and respective second labels obtained by inputting respective second samples into the pre-trained model output; The first data set and the second data set are weighted in a preset ratio to generate a feature map, wherein the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set; Input the fourth data set into the residual network branch to obtain a first output; input the third data set and the fourth data set into the common network branch respectively to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet; The first output and the second output are added and input into a first classifier, and the third output is input into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function; The training garbage classification model is obtained by iteration until the network converges.

2. The garbage classification model training method according to claim 1, characterized in that: The obtaining of image data associated with garbage includes: Crawl a large amount of image data related to junk files from the Internet; and / or obtain image data related to junk files from an image library.

3. The garbage classification model training method according to claim 1, characterized in that: The preset ratio is 1:

9.

4. The garbage classification model training method according to claim 1, characterized in that: The cross entropy loss function of the first classifier is: Among them, x i Represented as a sample, y i Expressed as x i The corresponding label, G represents the first classifier, and n represents the number of samples.

5. The garbage classification model training method according to claim 4 is characterized in that: The cross entropy loss function of the second classifier is: Among them, H represents the second classifier; The cross entropy loss function of the first classifier and the cross entropy loss function of the second classifier satisfy the following relationship: L=arg min{αL G +(1-α)L H },α∈(0,1) Among them, α is the weight coefficient and L is the total loss function.

6. A garbage classification method using the garbage classification model training method according to any one of claims 1 to 5, characterized in that: The method comprises: Acquire an image to be analyzed; Using the garbage classification model to determine the type of garbage in the image to be analyzed; If the type of garbage in the image to be analyzed is determined, the garbage type information is output.

7. The garbage classification method according to claim 6, characterized in that: The method further comprises: According to the garbage type information, a preset device is controlled to transport the garbage to a preset location.

8. A garbage classification model training device, characterized in that: The device comprises: An acquisition module, used for acquiring image data associated with garbage; A first data set making module, configured to obtain partial image data from the image data and construct a mapping to obtain a first data set, wherein the first data set includes a plurality of first samples in the partial image data and first labels corresponding to the first samples; A model pre-construction module, used for inputting the first data set into a deep convolutional neural network for training to construct a pre-trained model; A second data set making module, used to make a second data set using another part of the image data, wherein the second data set includes a plurality of second samples in the another part of the image data and second labels obtained by inputting each of the second samples into the pre-trained model output; a feature map generation module, configured to share weights of the first data set and the second data set according to a preset ratio to generate a feature map, wherein the feature map includes a third data set corresponding to the first data set and a fourth data set corresponding to the second data set; An input module, configured to input the fourth data set into a residual network branch to obtain a first output; input the third data set and the fourth data set into a common network branch to obtain a second output corresponding to the third data set and a third output corresponding to the fourth data set; the common network branch includes at least one or more of the following: LeNet-5, AlexNet and VGGNet; An output module, used for adding the first output and the second output and inputting the sum into a first classifier, and inputting the third output into a second classifier; wherein the loss functions of the first classifier and the second classifier both adopt a cross entropy loss function; The iteration module is used to iterate until the network converges to obtain a trained garbage classification model.

9. A computer, characterized in that: The computer includes a processor, and the processor is used to implement the steps in the garbage classification model training method as described in any one of claims 1 to 5 and / or the garbage classification method as described in any one of claims 6 to 7 when executing a computer program stored in a memory.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the garbage classification model training method as described in any one of claims 1-5 and / or the garbage classification method as described in any one of claims 6-7 are implemented.

Images