Garbage recognition and classification method based on ResNet50
By introducing a deep separable convolution and attention mechanism into the ResNet50 model and adopting a cosine annealing learning rate decay strategy, the problems of insufficient garbage recognition accuracy and high model complexity in the existing technology are solved, and more efficient garbage Identify and classify effects.
Patent Information
- Application Number
- CN202210661728.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-13
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-06-13
AI Technical Summary
The existing garbage identification and classification technology based on ResNet50 is not sufficient when processing data sets with rich image background information and large scale, and the recognition accuracy is insufficient and the model is complex, resulting in large consumption of computing resources and difficult to meet the needs of real-time applications.
Introduce deep separable convolution in the ResNet50 model, and add CBMA and SE attention mechanisms, combined with cosine annealing learning rate decay strategy, optimize network structure to improve recognition speed and accuracy.
Through the optimized ResNet50 model, the parameter quantity and calculation quantity are significantly reduced, the model convergence speed and classification accuracy are improved, and the real-time identification needs of large-scale data sets are adapted.
Smart Images

Figure CN115019173B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image recognition, and specifically, is a garbage recognition and classification method based on ResNet50. Background Art
[0002] In recent years, with the gradual improvement of social living standards, the corresponding garbage production is also growing rapidly. Therefore, it is of great significance to reasonably classify garbage and implement corresponding recycling operations. At the same time, it also plays a vital role in the green development and sustainable development currently advocated. However, it has been proved that there are still many difficulties in the classification of garbage in our actual lives. Although many provinces in my country have promulgated mandatory policies for garbage classification, people have long been lacking in their understanding of garbage classification, and even lack of knowledge on how to correctly classify garbage, so that garbage classification is not very ideal in current life. Because different types of garbage should adopt their own classification methods, if the classification is improper, it will not only hinder recycling, but also may cause various environmental pollution problems, so it is urgent to classify garbage reasonably.
[0003] At the same time, with the rapid development of deep learning, deep learning methods have been widely used in the fields of image classification and image detection, and multi-category image classification problems have been effectively solved. One of its most basic research goals is to classify several incompletely related category images. This technology has been gradually transformed into a new type of data-driven deep learning technology, and has been greatly improved and developed. Our existing classic networks, such as VGG, ResNet, SENet, etc., are also developed on this basis. Although the current research on image recognition and classification is relatively mature, there are few studies on the classification of junk images.
[0004] So far, for the garbage recognition and classification technology based on ResNet50, how to optimize and improve it based on the network model with relatively deep network layers to adapt to the data sets with richer image background information and larger scale, how to reduce the complexity of the model while ensuring a high recognition accuracy, reduce the time required for the algorithm and the computing resources used, and better meet real-time applications and needs, etc., there is still a lot of research work to be done in the future. Summary of the invention
[0005] The purpose of the present invention is to optimize and improve the selected ResNet50 model so as to further improve the speed of garbage recognition and the accuracy of classification. The scheme is: 1. Preprocessing the selected pictures, 2. Selecting ResNet50 as the benchmark model for optimization and improvement, firstly, using deep separable convolution in the residual structure, 3. Secondly, adding two attention mechanisms at the same time: CBMA attention mechanism and SE attention mechanism, adding CBMA attention mechanism before the first Block of ResNet50, and adding SE attention mechanism after the last Block; 4. In learning, by adopting the cosine annealing learning rate decay policy, the learning rate is increased at the beginning to skip the local optimum, and then the learning rate is slowly reduced, and a small learning rate is used to make the network gradually converge, thereby improving the recognition rate of the image and the accuracy of image classification.
[0006] In order to achieve the above object, the specific technical solution adopted by the present invention is as follows:
[0007] The garbage identification and classification method based on ResNet50 includes the following steps:
[0008] Step 1: Select a public dataset that contains a variety of garbage images commonly seen in real life, create a dataset suitable for garbage classification research, then divide the dataset into a training set and a test set, and preprocess the images in the training set.
[0009] Step 2: Select the ResNet50 network as the baseline model, add depthwise separable convolution to the residual structure, and add an attention mechanism to the network to build a new network model;
[0010] Step 3: Set the network model hyperparameters, select the loss function and optimization method to train the images in the training set, and then obtain the trained network model after continuous verification and training;
[0011] Step 4: Use the trained network model to detect junk images in the untrained test set, and finally obtain the junk identification and classification results.
[0012] In the above technical solution, image classification is to perform some operations on the input image and finally output a set of classifications that describe the content of the image or the probability of the classification of the content of the image; the input of image classification is the resolution and size of the image, that is, the pixel value array of input x = h×w×c, where h, w and c represent the various attributes of the image, namely height, width and depth. Depth is the number of color channels. For example, the depth of an RGB image is 3, while the depth of a grayscale image is 1. The corresponding array has h×w×c elements, where the value of each number, ranging from 0 to 255, describes the grayscale of the pixel corresponding to that point. The output is a number that describes the probability that the image belongs to a specific classification.
[0013] In a further improvement of the present invention, the method of performing a preprocessing operation on the image in step 1 specifically comprises:
[0014] Step 1.1, first convert the size of the input image into an input feature map of 512×512;
[0015] Step 1.2, randomly rotate the image converted in step 1.1, and the rotation angle can be set by yourself, for example, set to 15 degrees;
[0016] Step 1.3, randomly flip the image obtained after random rotation in step 1.2 horizontally;
[0017] Step 1.4, perform a center cropping operation on the image obtained by randomly horizontally flipping in step 1.3;
[0018] Step 1.5, convert the image obtained after center cropping in step 1.4 into a tensor;
[0019] Step 1.6, normalize the tensor obtained in step 1.5.
[0020] A further improvement of the present invention is that the image preprocessing operation uses the transformation function transforms in the tool set torchvision. The specific method is as follows:
[0021] The image size conversion method in step 1.1: transforms.Resize([h, w]), where h and w represent the height and width of the image respectively;
[0022] The method for randomly rotating the image in step 1.2: transforms.RandomRotation(degrees), where degrees represents the angle, that is, randomly rotate a certain angle according to degrees;
[0023] The method for randomly flipping the image horizontally in step 1.3: transforms.RandomHorizontalFlip();
[0024] The method for center cropping the image in step 1.4 is: transforms.CenterCrop(crop_size), where crop_size indicates the size to which the image is cropped, that is, a cropping operation of the specified crop_size is performed with the center point of the input image as the center;
[0025] The method to convert the image in step 1.5 into a tensor: transforms.ToTensor(), converts the data from the range [0,255] to [0,1] by dividing the original data by 255; at the same time, convert the image of size (h,w,c) to size (c,h,w), where h, w and c represent the height, width and depth of the image respectively;
[0026] Step 1.6 Normalization method: transforms.Normalize(mean,std), using the formula The data is standardized by channel, that is, the mean is first subtracted, and then divided by the standard deviation std, so that each element is distributed to (-1,1).
[0027] According to a further improvement of the present invention, the ResNet50 structure described in step 2 is divided into five stages, the first stage is relatively simple, and the following four stages are composed of Bottleneck, and the second, third, fourth and fifth stages contain 3, 4, 6 and 3 Bottleneck respectively.
[0028] The first stage input x of ResNet50 passes through the convolution layer, BN layer, ReLU activation function, and MaxPooling layer to obtain the output.
[0029] ① Calculation method of convolutional layer: Output Where x represents the pixel value array of input x = h × w × c, h, w and c represent the attributes of the image, namely height, width and depth respectively; padding represents the number of layers of 0 added to each edge of the input, and the height and width are increased by 2 × padding. If you want to maintain the same size of the feature map before and after the convolution layer, kernel_size indicates the size of the convolution kernel; stride indicates the convolution step, which is the multiple by which the input is enlarged.
[0030] ②BN layer, namely Batch Normalization, is calculated as follows:
[0031] Step 1: Calculate the mean Where x represents the input, m represents the mini-batch size,
[0032] Step 2: Calculate the standard deviation
[0033] The third step is normalization
[0034] Step 4: Reconstruct change y i =γ×x′ i +β;
[0035] ③The ReLU activation function is a piecewise function, the formula is: f(x) = max(0,x);
[0036] ④MaxPooling layer: The entire image is divided into several non-overlapping blocks of the same size (pooling size). In each block, only the largest number is taken, and then the other nodes are discarded, and the original plane structure is maintained to obtain the output.
[0037] 1. Furthermore, Bottleneck has two structures, one with the same number of input and output channels is called Identity Block, and the other with different numbers of input and output channels is called Conv Block;
[0038] The three convolution blocks (and related BN and ReLU) on the left side of the Identity Block are functions F(x), which are added to the input x to get F(x)+x, and then pass through a ReLU function to get the output of the Identity Block.
[0039] Compared with IdentityBlock, ConvBlock has an additional convolutional layer on the right. Let it be the function G(x). Similarly, the three convolutional blocks on the left (as well as the related BN and ReLU) are the function F(x). The output of Conv Block is obtained by F(x)+G(x).
[0040] A further improvement of the present invention is that the attention mechanisms added to the network in step 2 are the CBMA attention mechanism and the SE attention mechanism, wherein the CBMA attention mechanism is added after the first stage of ResNet50, and the SE attention mechanism is added after the fifth stage.
[0041] ①CBMA (Convolutional Block Attention Module), CBAM contains two independent sub-modules, Channel Attention Module (CAM) and Spatial Attention Module (SAM);
[0042] The calculation formula of CAM is: c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))
[0043] The input feature map F(h×w×c) is subjected to global maxpooling and global average pooling based on width and height respectively to obtain two 1×1×c feature maps. Then, they are sent to a two-layer neural network (MLP). The number of neurons in the first layer is c / r (r is the reduction rate), the activation function is ReLU, and the number of neurons in the second layer is c. The two layers of the neural network are shared. Then, the features output by the MLP are added based on element-wise operation, and then activated by σ (representing sigmoid) to generate the final channel attention feature, i.e., M c Finally, M c Perform element-wise multiplication with the input feature map F to generate the input features required by the Spatial attention module;
[0044] The calculation formula of SAM is: s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))
[0045] The feature map F output by the Channel attention module is used as the input feature map of this module. First, a global max pooling and global average pooling based on the channel are performed to obtain two h×w×1 feature maps, and then these two feature maps are concat-operated based on the channel (channel concatenation). Then, a 7×7 convolution operation is performed to reduce the dimension to 1 channel, i.e., h×w×1. Then, a sigmoid operation is performed to generate the spatial attention feature, i.e., M SFinally, the feature is multiplied by the input feature of the module to obtain the final generated feature.
[0046] ②SE attention mechanism (Squeeze-and-Excitation Networks), adds attention mechanism in the channel dimension, and the key operations include: Squeeze and Excitation.
[0047] Through automatic learning, that is, using another new neural network, the importance of each channel of the feature map is obtained, and then this importance is used to assign a weight value to each feature, so that the neural network can focus on certain feature channels. The channels of the feature map that are useful for the current task are enhanced, and the feature channels that are not very useful for the current task are suppressed.
[0048] The implementation process is as follows:
[0049] (1) Squeeze: The feature map is subjected to global average pooling to compress the two-dimensional features (h×w) of each channel into a real number, that is, compressing the feature map from [h, w, c] into a feature vector [1, 1, c];
[0050] (2) Excitation: Through a fully connected layer, the channel dimension of the feature map vector is reduced to the original 1 / r, that is, [1, 1, c×1 / r]; then it passes through the Swish activation function; through another fully connected layer, the feature map of the feature map vector is increased back to the original [1, 1, c]; and then it is converted into a normalized weight vector between (0, 1) through the sigmoid function.
[0051] (3) Scale: Multiply the normalized weights and the original input feature map channel by channel to generate a weighted feature map.
[0052] Max pooling refers to taking the maximum value of feature points in the neighborhood; the sigmoid function compresses the values in the real number range to between (0, 1), and its formula is Where z represents the linear regression equation.
[0053] A further improvement of the present invention is that the model training method described in step 3 is: using the torch deep learning framework, the optimization function is set to AdamW, the initial learning rate is set to 1e-4, and the learning rate decay type adopts cosine decay; the optimization function AdamW, the calculation process is as follows:
[0054] ① First, calculate the exponential moving average of the gradient
[0055] m=β 1 ×m+(1-β 1)×dx, where m is initialized to 0 and d x is the gradient at time t. m is the gradient momentum taking into account the previous time steps, β 1 The coefficient is an exponential decay rate that controls the distribution of weights and is usually close to 1, such as 0.99.
[0056] ②The second is to calculate the exponential moving average of the square of the gradient
[0057] v=β 2 ×v+(1-β 2 )×(dx) 2 , where the initial value of v is 0, β 2 is the exponential decay coefficient, controlling the influence of the previous gradient and performing a weighted mean on the square of the gradient, β 2 0.999 is often used.
[0058] ③Finally, update the parameters
[0059] The initial learning rate lr is multiplied by the ratio of the gradient mean to the square root of the gradient variance, where the default learning rate of lr is α = 0.001; ε = 10 -8 , wd is the weight decay coefficient, usually 0.01.
[0060] Cosine decay is to use the cosine-related method to decay the learning rate. The decay graph is similar to the cosine function. The calculation formula is as follows:
[0061] global_step=min(global_step,decay_steps),
[0062]
[0063] decaved_learning_rate=learning_rate×decay_rate;
[0064] Among them, global_step represents the number of rounds of the current iteration, decay_steps represents the number of rounds required to fully use the training data. From the formula, we can see that the decay coefficient decay_rate multiplied by the learning rate learning_rate decays from 1 to α as the global_step increases. Finally, the decay curve is similar to the cosine curve, and the rate of decline gradually becomes faster.
[0065] Compared with the prior art, the present invention has the following beneficial effects:
[0066] (1) The present invention is based on the optimization and improvement of the ResNet50 network, thereby solving the degradation of the existing network and the problem of multi-layer neural network;
[0067] (2) The present invention introduces a depth-separable convolution into the residual network structure. Compared with ordinary convolution, when the network depth is the same, the depth-separable convolution can greatly reduce the number of network parameters and the amount of calculation, thereby accelerating the speed of model convergence and improving the accuracy of model classification;
[0068] (3) The present invention adds two attention mechanisms, the CBAM attention mechanism module and the SE attention mechanism module, which significantly optimizes the features;
[0069] (4) The optimizer used in the present invention is adamw. Based on the large model and large data set used, the adamw optimization algorithm is more efficient in solving local deep learning problems than other optimizers;
[0070] (5) The learning rate decay type of the present invention adopts cosine decay, which increases the learning rate at the beginning to skip the local optimum, and then slowly decreases the learning rate, using a small learning rate to make the network gradually converge. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION
[0072] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments are only used to explain the present invention and do not limit the protection scope of the present invention.
[0073] Example: Figure 1 As shown in the figure, the garbage identification and classification method based on ResNet50 includes the following steps:
[0074] Step 1: Select a public dataset that contains a variety of garbage images commonly seen in real life, create a dataset suitable for garbage classification research, then divide the dataset into a training set and a test set, and preprocess the images in the training set.
[0075] Step 2: Select the ResNet50 network as the baseline model, add depthwise separable convolution to the residual structure, and add an attention mechanism to the network to build a new network model;
[0076] Step 3: Set the network model hyperparameters, select the loss function and optimization method to train the images in the training set, and then obtain the trained network model after continuous verification and training;
[0077] Step 4: Use the trained network model to detect junk images in the untrained test set, and finally obtain the junk identification and classification results.
[0078] The method of performing preprocessing operation on the image in step 1 specifically includes:
[0079] Step 1.1, first convert the size of the input image into an input feature map of 512×512;
[0080] Step 1.2, randomly rotate the image converted in step 1.1, and the rotation angle can be set by yourself, for example, set to 15 degrees;
[0081] Step 1.3, randomly flip the image obtained after random rotation in step 1.2 horizontally;
[0082] Step 1.4, perform a center cropping operation on the image obtained by randomly horizontally flipping in step 1.3;
[0083] Step 1.5, convert the image obtained after center cropping in step 1.4 into a tensor;
[0084] Step 1.6, normalize the tensor obtained in step 1.5.
[0085] The image size conversion method in step 1.1: transforms.Resize([h, w]), where h and w represent the height and width of the image respectively;
[0086] The method for randomly rotating the image in step 1.2: transforms.RandomRotation(degrees), where degrees represents the angle, that is, randomly rotate a certain angle according to degrees;
[0087] The method for randomly flipping the image horizontally in step 1.3: transforms.RandomHorizontalFlip();
[0088] The method for center cropping the image in step 1.4 is: transforms.CenterCrop(crop_size), where crop_size indicates the size to which the image is cropped, that is, a cropping operation of the specified crop_size is performed with the center point of the input image as the center;
[0089] The method to convert the image in step 1.5 into a tensor: transforms.ToTensor(), converts the data from the range [0,255] to [0,1] by dividing the original data by 255; at the same time, convert the image of size (h,w,c) to size (c,h,w), where h, w and c represent the height, width and depth of the image respectively;
[0090] Step 1.6 Normalization method: transforms.Normalize(mean,std), using the formula The data is standardized by channel, that is, the mean is first subtracted, and then divided by the standard deviation std, so that each element is distributed to (-1,1).
[0091] The ResNet50 structure described in step 2 is divided into 5 stages. The first stage is relatively simple, and the following four stages are composed of Bottleneck. The second, third, fourth and fifth stages contain 3, 4, 6 and 3 Bottlenecks respectively. The attention mechanisms added to the network in step 2 are CBMA attention mechanism and SE attention mechanism. The CBMA attention mechanism is added after the first stage of ResNet50, and the SE attention mechanism is added after the fifth stage.
[0092] The model training method described in step 3 is: using the torch deep learning framework, setting the optimization function to AdamW, setting the initial learning rate to 1e-4, and using the cosine decay type for the learning rate decay; the optimization function AdamW, the calculation process is as follows:
[0093] The first step is to calculate the exponential moving average of the gradient
[0094] m=β 1 ×m+(1-β 1 )×dx, where m is initialized to 0, dx is the gradient at time t. m is the gradient momentum considering the previous time step, β 1 The coefficient is an exponential decay rate that controls the distribution of weights and is usually close to 1, such as 0.99.
[0095] The second is to calculate the exponential moving average of the squared gradient
[0096] v=β 2 ×v+(1-β 2 )×(dx) 2 , where the initial value of v is 0, β 2 is the exponential decay coefficient, controlling the influence of the previous gradient and performing a weighted mean on the square of the gradient, β 2 0.999 is often used.
[0097] Finally, update the parameters
[0098] The initial learning rate 1r is the ratio of the gradient mean to the square root of the gradient variance, where 1r defaults to a learning rate of α = 0.001; ε = 10 -8 , wd is the weight decay coefficient, usually 0.01.
[0099] Cosine decay is to use the cosine-related method to decay the learning rate. The decay graph is similar to the cosine function. The calculation formula is as follows:
[0100] global_step=min(global_step,decay_steps),
[0101]
[0102] decayed_learning_rate=learning_rate×decay_rate;
[0103] Among them, global_step represents the number of rounds of the current iteration, decay_steps represents the number of rounds required to fully use the training data. From the formula, we can see that the decay coefficient decay_rate multiplied by the learning rate learning_rate decays from 1 to α as the global_step increases. Finally, the decay curve is similar to the cosine curve, and the rate of decline gradually becomes faster.
[0104] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A garbage identification and classification method based on ResNet50, It is characterized in that The following steps are involved: Step 1: Select a public dataset that contains a variety of garbage images commonly seen in real life, create a dataset suitable for garbage classification research, then divide the dataset into a training set and a test set, and preprocess the images in the training set; Step 2: Select the ResNet50 network as the baseline model, add depthwise separable convolution to the residual structure, and add an attention mechanism to the network to build a new network model; Step 3: Set the network model hyperparameters, select the loss function and optimization method to train the images in the training set, and then obtain the trained network model after continuous verification and training; Step 4: Use the trained network model to detect junk images in the untrained test set, and finally obtain the junk identification and classification results; The ResNet50 structure described in step 2 is divided into 5 stages: Phase 1: The input x passes through the convolution layer, BN layer, ReLU activation function, and MaxPooling layer to get the output: ① Calculation method of convolutional layer: Output Where x represents the pixel value array of input x = h × w × c, h, w and c represent the attributes of the image, namely height, width and depth respectively, padding represents the number of layers of 0 added to each edge of the input, and the height and width are increased by 2 × padding to maintain the same size of the feature map before and after the convolution layer. kernel_size indicates the size of the convolution kernel, stride indicates the convolution step length, and the multiple of input expansion; ②BN layer, namely Batch Normalization, is calculated as follows: Step 1: Calculate the mean Where x represents the input, m represents the mini-batch size, Step 2: Calculate the standard deviation The third step is normalization Step 4: Reconstruct change y i =γ×x i ′+β; ③The ReLU activation function is a piecewise function, and the formula is: f(x) = max(0, x); ④MaxPooling layer: The entire image is divided into several non-overlapping small blocks of the same size. In each small block, only the largest number is taken, and then the other nodes are discarded, and the original plane structure is maintained to obtain the output; The second to fifth stages are all composed of Bottlenecks, and the second, third, fourth and fifth stages contain 3, 4, 6 and 3 Bottlenecks respectively.
2. According to the garbage identification and classification method based on ResNet50 according to claim 1, It is characterized in that The method of performing preprocessing operation on the image in step 1 specifically includes: Step 1.1, first convert the size of the input image into an input feature map of 512×512; Step 1.2, randomly rotate the image transformed in step 1.1; Step 1.3, randomly flip the image obtained after random rotation in step 1.2 horizontally; Step 1.4, perform a center cropping operation on the image obtained by randomly horizontally flipping in step 1.3; Step 1.5, convert the image obtained after center cropping in step 1.4 into a tensor; Step 1.6, normalize the tensor obtained in step 1.
5.
3. According to the garbage identification and classification method based on ResNet50 according to claim 2, It is characterized in that The image preprocessing operation in step 1 uses the transformation function transforms in the toolset torchvision. The specific method is as follows: The image size conversion method in step 1.1: transforms.Resize([h, w]), where h and w represent the height and width of the image respectively; The method for randomly rotating the image in step 1.2: transforms.RandomRotation(degrees), where degrees represents the angle, that is, randomly rotate a certain angle according to degrees; The method for randomly flipping the image horizontally in step 1.3: transforms.RandomHorizontalFlip(); The method for center cropping the image in step 1.4 is: transforms.CenterCrop(crop_size), where crop_size indicates the size to which the image is cropped, that is, a cropping operation of the specified crop_size is performed with the center point of the input image as the center; The method to convert the image in step 1.5 into a tensor: transforms.ToTensor(), converts the data from the range [0, 255] to [0, 1] by dividing the original data by 255; at the same time, convert the image of size (h, w, c) to size (c, h, w), where h, w, and c represent the height, width, and depth of the image respectively; Step 1.6 Normalization method: transforms.Normalize(mean, std), using the formula The data is standardized by channel, that is, the mean mcan is first subtracted, and then divided by the standard deviation std, so that each element is distributed to (-1, 1).
4. According to the garbage identification and classification method based on ResNet50 according to claim 1, It is characterized in that The attention mechanisms added to the network in step 2 are the CBMA attention mechanism and the SE attention mechanism. The CBMA attention mechanism is added after the first stage of ResNet50, and the SE attention mechanism is added after the fifth stage.
5. According to the garbage identification and classification method based on ResNet50 according to claim 1, It is characterized in that The model training method described in step 3 is: using the torch deep learning framework, setting the optimization function to AdamW, setting the initial learning rate to 1e-4, and using cosine decay as the learning rate decay type.
6. According to the garbage identification and classification method based on ResNet50 according to claim 5, It is characterized in that The optimization function AdamW is calculated as follows: ① First, calculate the exponential moving average of the gradient m=β 1 ×m+(1-β 1 )×dx, where m is initialized to 0, dx is the gradient at time t, m is the gradient momentum considering the previous time step, β 1 The coefficient is the exponential decay rate, which controls the distribution of weights and is usually close to 1; ②The second is to calculate the exponential moving average of the square of the gradient v=β 2 ×v+(1-β 2 )×(dx) 2 , where the initial value of v is 0, β 2 is the exponential decay coefficient, which controls the influence of the previous gradient and performs a weighted mean on the square of the gradient. 2 Use 0.999; ③Finally, update the parameters The initial learning rate lr is multiplied by the ratio of the gradient mean to the square root of the gradient variance, where 1r defaults to a learning rate of α = 0.001; ε = 10 -8 , wd is the weight decay coefficient of 0.
01.
7. According to the garbage identification and classification method based on ResNet50 according to claim 6, It is characterized in that Cosine attenuation is to use the cosine-related method to decay the learning rate. The calculation formula is as follows: global_step=minn(global_step,decay_steps), decayed_learning_rate=learning_rate×decay_rate Among them, global_step represents the number of rounds of the current iteration, decay_steps represents the number of rounds required to fully use the training data. From the formula, we can see that the decay coefficient decay_rate multiplied by the learning rate learning_rate decays from 1 to α as the global_step increases. Finally, the decay curve is similar to the cosine curve, and the rate of decline gradually becomes faster.
Citation Information
Patent Citations
Weather image recognition method based on lightweight convolutional neural network
CN110929603A
Method for automatically classifying garbage
CN112241679A
Household garbage classification method based on deep learning
CN112364944A
Junk image classification method based on transfer learning
CN114463578A
