A garbage classification method based on neuron activation value

Abstract

The invention discloses a garbage classification method based on neuron activation value. The method uses the normalized garbage data set as a training data set; screens out spot features, and obtains spot noise through image cropping and threshold segmentation method; randomly adds spot noise to all images in the garbage data set, uses the image after adding spot noise as a noise sample, and the original image as a non-noise sample, and binds the noise sample and the non-noise sample into a noise sample pair; constructs a garbage classification model, and inputs the noise sample pair into the garbage classification model for training based on neuron activation value; inputs the garbage image into the trained garbage classification model for testing, obtains the garbage classification prediction matrix, and completes the classification of the garbage image. The garbage classification model constructed by the method of the invention enhances the environmental migration robustness of the image classification model. It improves the accuracy when applied in real scenes.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and in particular relates to a garbage classification method based on neuron activation values. Background Art

[0002] Image recognition has always been a core issue in the field of machine vision. In traditional methods, feature extraction mainly relies on manually designed extractors, which requires professional knowledge and a complex parameter adjustment process. At the same time, traditional methods are usually targeted at specific applications and have poor generalization and robustness.

[0003] Deep learning is mainly data-driven for feature extraction. Based on the learning of a large number of samples, deep, dataset-specific feature representations can be obtained. It is more efficient and accurate in expressing datasets, the extracted abstract features are more robust, have better generalization capabilities, and can achieve end-to-end mapping. The disadvantage is that the sample set has a greater impact and requires higher computing power. Deep learning technology has achieved advanced performance in many fields, but at the same time, the robustness problem of the model in this method has gradually become prominent and is increasingly valued by academia and industry. Deep learning models have natural defects. In image classification tasks, due to the large difference between the development environment and the actual application scenarios, the migration effect of related algorithms is not ideal.

[0004] Deep learning models are widely used due to their powerful function fitting capabilities, but they are often considered a black box, with an opaque and difficult-to-understand intermediate structure. The application scenarios of deep learning are also mainly focused on input and output. During the entire image recognition process, a lot of information is hidden inside the deep learning model. This example uses the information inside the deep learning model to provide guidance for improving the robustness of the model.

[0005] In recent years, researchers have introduced the image classification technology of deep learning models into the application scenarios of garbage classification. In the application process, researchers need to collect a large number of garbage pictures and annotate the data before training the model. However, the environment in which the garbage pictures are collected will greatly affect the accuracy of model recognition in the final application stage. When the application scenario and the garbage data set collection scenario are very different, the model recognition accuracy will be greatly reduced. Therefore, it is usually required that the data set collection environment scenario and the actual use scenario are as close as possible. However, given the high cost of re-collecting data and the fact that other objective conditions cannot be changed, it is very necessary to invent a method that can successfully migrate the model from the existing laboratory scenario to the real application scenario. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a garbage classification method based on neuron activation values.

[0007] To achieve the above technical objectives, the scheme of the present invention is:

[0008] A first aspect of an embodiment of the present invention provides a garbage classification method based on neuron activation values, the method specifically comprising the following steps:

[0009] (1) Collect garbage images, clean, screen and aggregate them to obtain a garbage dataset, normalize them and use the normalized garbage dataset as a training dataset;

[0010] (2) Filter out the light spot features from the garbage dataset and obtain the light spot noise through image cropping and threshold segmentation method;

[0011] (3) randomly adding the spot noise obtained in step (2) to all images in the garbage data set, taking the image after adding the spot noise as the noise-added sample and the original image as the non-noise-added sample, and binding the noisy sample and the non-noise-added sample as a noise sample pair;

[0012] (4) constructing a garbage classification model, inputting the noise sample obtained in step (3) into the constructed garbage classification model for training based on the neuron activation value; inputting the noise sample into the garbage classification model to obtain a first activation value, and inputting the non-noise sample into the garbage classification model to obtain a second activation value; using the first activation value and the second activation value as a loss function to adjust the parameters of the garbage classification model, and completing the training of the garbage classification model;

[0013] (5) Input the garbage image into the trained garbage classification model for testing, obtain the garbage classification prediction matrix, and complete the classification of the garbage image.

[0014] Furthermore, the normalized image is a 256*256 image.

[0015] Furthermore, the garbage classification model consists of two parts: a feature extractor and a classifier. The feature extractor uses 5 convolutional layers, of which the first two are low-level convolutional layers and the last three are high-level convolutional layers. The classifier uses a network composed of 2 fully connected layers, and the activation function uses the Relu function. The input of the garbage classification model is a picture of size 256*256, and the output is a 10-dimensional vector representing the image classification result.

[0016] Furthermore, cross entropy is used as the loss function; the formula is as follows:

[0017]

[0018] Among them, p i Represents the probability of garbage being classified as each category sample;

[0019] Input the noise sample pairs constructed in step (3) into the model in sequence, and record the activation values ​​of the neurons in the advanced convolutional layer of model M as O1 and O2 respectively;

[0020] The difference between O1 and O2 is directly used as the loss function, and the formula is as follows:

[0021]

[0022] Among them, 1i and O 2i Respectively represent the activation values ​​of the same neuron after the noise sample pair is input;

[0023] The size of the training batch is set. The warm-up learning rate strategy is adopted in the training phase. The optimizer is optimized using Adam. The garbage classification model is trained using the training set. The cross entropy is used as the loss function. The formula is as follows:

[0024] L=L2+L1

[0025] The training is stopped when the accuracy reaches the preset standard to obtain the trained garbage classification model.

[0026] A second aspect of an embodiment of the present invention provides a garbage classification device based on neuron activation values, characterized in that it includes one or more processors for implementing the above-mentioned garbage classification method based on neuron activation values.

[0027] A third aspect of an embodiment of the present invention provides a computer-readable storage medium on which a program is stored, characterized in that when the program is executed by a processor, it is used to implement the above-mentioned garbage classification method based on neuron activation values.

[0028] The beneficial effects of the present invention are as follows: the present invention constructs a garbage classification model based on neuron activation values, and realizes garbage identification in real application scenarios through the constructed garbage classification model. The garbage classification model constructed by the present invention enhances the environmental migration robustness of the image classification model and improves the accuracy when applied in real scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the spot noise of the data set;

[0030] Figure 2 It is a flow chart of the present invention;

[0031] Figure 3 It is a flowchart of the present invention;

[0032] Figure 4 A schematic diagram of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0034] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0036] The following is a detailed description of the garbage classification method based on neuron activation value of the present invention in conjunction with the accompanying drawings. The features of the following embodiments and implementations can be combined with each other without conflict. Figure 2 , Figure 3 shown.

[0037] (1) Use the trash dataset as the initial sample set:

[0038] The present invention uses the trash dataset as the training dataset. The trash dataset is a dataset that is cleaned and collected from various open source datasets collected on the Internet, that is, a garbage dataset, which contains 80,000 pictures of different types. Ten more representative categories among the 245 subcategories are retained, namely: waste paper boxes, glass, non-metal, paper, plastic, dry batteries, power banks, leftovers, fruit peels, and cigarette butts. The dataset after all images are normalized to 256*256 images is recorded as D0.

[0039] (2) Constructing spot noise:

[0040] Select more obvious spot features from the image, and obtain spot noise by image cropping and threshold segmentation. In the embodiment of the present invention, a total of 100 different spot noises are cropped from different images. Figure 1 As shown, Figure 1 Schematic diagram of the spot noise of the dataset.

[0041] (3) Add spot noise to construct noise sample pairs:

[0042] In all images, speckle noise is randomly added in a weighted manner by pixel values, and the noisy image and the original image are bound as noise sample pairs. All noise sample pairs together constitute a data set denoted as D.

[0043] (4) Robust garbage classification network training based on neuron activation value differences:

[0044] Deep learning models are widely used due to their powerful function fitting capabilities. However, due to the complexity of their structure, deep learning models are often considered black boxes, and the application scenarios of deep learning are mainly focused on input and output.

[0045] However, during the entire image recognition process, there is still a lot of information hidden inside the deep learning model. For example, for two different input samples, the difference in the activation values ​​of each neuron inside the deep learning model also explains the difference between the two samples. On the contrary, if the activation values ​​of the neurons inside the deep learning model are the same for two different samples, it means that in the eyes of the deep learning model, the two samples have a high degree of similarity, or in other words, the deep learning model is insensitive to the difference between the two samples.

[0046] Usually, the training of deep learning models uses the difference between the output layer and the true value as the loss function loss. When the loss gradually decreases through training, it means that the output of the model is getting closer and closer to the true value. In this example, in addition to the conventional loss function loss1, the difference in the activation value of the neuron for the noise sample pair is also used as the loss function loss2. When loss2 is gradually reduced through training, the sensitivity of the model to the noise image will also decrease, that is, the neural network has good spot noise robustness.

[0047] (4.1) Network model construction

[0048] The garbage classification network is a supervised model. The input of the model is a picture of size 256*256, and the output is a 10-dimensional vector representing the image classification result. In this embodiment, the constructed garbage classification network includes a feature extractor and a classifier. The feature extractor uses 5 convolutional layers, of which the first 2 convolutional layers are low-level convolutional layers, and the last 3 convolutional layers are high-level convolutional layers. The classifier uses a network composed of 2 fully connected layers, and the activation function uses the Relu function. The deep learning model is trained using the training set of the original data set D0. Cross entropy is used as the loss function. The model is denoted as M.

[0049] (4.2) Main task update strategy

[0050] The deep learning model is trained using the training set of the dataset D0 in step (2). The cross entropy is used as the loss function. The formula is as follows:

[0051]

[0052] where p i Represents the probability of garbage being classified as each category sample.

[0053] (4.3) Neuron difference minimization update strategy

[0054] Input the noise sample pairs constructed in step (3) into the model in sequence, and record the activation values ​​of the neurons in the advanced convolutional layer of model M as O1 and O2 respectively each time.

[0055] The difference between O1 and O2 is directly used as the loss function, and the formula is as follows:

[0056]

[0057] Among them, 1i and O 2i They respectively represent the activation values ​​of the same neuron after the noise sample pair is input.

[0058] (4.4) Model training

[0059] The size of the training batch is set to 100. The warm-up learning rate strategy is used in the training phase, and the optimizer is Adam. The garbage classification model is trained using the training set. The formula is as follows:

[0060] L=L1+L2

[0061] When the accuracy of the model on the validation set reaches the preset standard, the training is stopped and the model training is completed.

[0062] Corresponding to the above-mentioned embodiment of the garbage classification method based on neuron activation values, the present invention also provides an embodiment of a garbage classification device based on neuron activation values.

[0063] See also Figure 4 An embodiment of the present invention provides a garbage classification device based on neuron activation values, including one or more processors for implementing the garbage classification method based on neuron activation values ​​in the above embodiment.

[0064] The embodiments of the garbage classification device based on neuron activation values ​​of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 4 As shown, it is a hardware structure diagram of any device with data processing capability where the garbage classification device based on neuron activation value of the present invention is located. Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0065] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0066] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0067] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the garbage classification method based on neuron activation values ​​in the above embodiment is implemented.

[0068] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0069] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A garbage classification method based on neuron activation value, characterized in that: The method specifically comprises the following steps: (1) Collect garbage images, clean, screen and aggregate them to obtain a garbage dataset, normalize them and use the normalized garbage dataset as a training dataset; (2) Filter out the light spot features from the garbage dataset and obtain the light spot noise through image cropping and threshold segmentation method; (3) randomly adding the spot noise obtained in step (2) to all images in the garbage data set, taking the image after adding the spot noise as the noise-added sample and the original image as the non-noise-added sample, and binding the noisy sample and the non-noise-added sample as a noise sample pair; (4) constructing a garbage classification model, inputting the noise sample obtained in step (3) into the constructed garbage classification model for training based on the neuron activation value; inputting the noise sample into the garbage classification model to obtain a first activation value, and inputting the non-noise sample into the garbage classification model to obtain a second activation value; using the first activation value and the second activation value as a loss function to adjust the parameters of the garbage classification model, and completing the training of the garbage classification model; Among them, cross entropy is used as the loss function; the formula is as follows: Among them, p i Represents the probability of garbage being classified as each category sample; Input the noise sample pairs constructed in step (3) into the model in sequence, and record the activation values ​​of the neurons in the advanced convolutional layer of model M as O1 and O2 respectively; The difference between O1 and O2 is directly used as the loss function, and the formula is as follows: Among them, 1i and O 2i Respectively represent the activation values ​​of the same neuron after the noise sample pair is input; The size of the training batch is set. The warm-up learning rate strategy is adopted in the training phase. The optimizer is optimized using Adam. The garbage classification model is trained using the training set. The cross entropy is used as the loss function. The formula is as follows: L=L2+L1 The training is stopped until the accuracy reaches the preset standard, and the trained garbage classification model is obtained; (5) Input the garbage image into the trained garbage classification model for testing, obtain the garbage classification prediction matrix, and complete the classification of the garbage image.

2. The garbage classification method based on neuron activation value according to claim 1 is characterized in that: The normalized image is a 256*256 image.

3. The garbage classification method based on neuron activation value according to claim 1 is characterized in that: The garbage classification model consists of two parts: a feature extractor and a classifier. The feature extractor uses five convolutional layers, of which the first two are low-level convolutional layers and the last three are high-level convolutional layers. The classifier uses a network consisting of two fully connected layers, and the activation function uses the Relu function. The input of the garbage classification model is a picture of size 256*256, and the output is a 10-dimensional vector representing the image classification result.

4. A garbage classification device based on neuron activation value, characterized in that: It comprises one or more processors for implementing the garbage classification method based on neuron activation value as described in any one of claims 1 to 3.

5. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement the garbage classification method based on neuron activation value described in any one of claims 1 to 3.

Images