An improved method of image recognition based on garbage target detection

Abstract

The invention discloses an improved image recognition method based on garbage target detection. The method uses a garbage data set as a sample data set after marking a light spot area and normalizing it; then constructs and trains a target detection model for the light spot to obtain light spot area features; then complements the light spot area features to obtain a picture pixel weight matrix; then constructs and trains a garbage classification model; finally, the image to be tested is input into the trained garbage classification model, and the pixel values ​​of all pixels in the image to be tested are multiplied by the picture pixel weight matrix and input into the trained garbage classification model to obtain a classification result.

Description

Technical Field

[0001] The present invention relates to the problem of poor environmental robustness of image recognition of deep learning models, and proposes an improved method for image recognition of deep learning models based on target detection. Background Art

[0002] Deep learning models have a powerful ability to learn the intrinsic function laws of sample data sets and analyze abstract features. In recent years, they have assisted people in making decisions in many fields and provided solutions to many complex recognition and classification problems. Deep learning has played a very good role in bioinformatics, image recognition, speech recognition, driverless cars, artistic creation, emotion recognition, natural language processing, banks, prisons, career assessment, and criminal justice judgments. And with the continuous efforts of relevant scientific research staff, the efficiency of deep learning models has been continuously improved. In recent years, with the introduction of convolutional neural networks, image recognition technology based on deep neural networks has been brought to a new level.

[0003] Traditional garbage classification requires manual identification. In recent years, researchers have introduced image classification technology of deep learning models into garbage classification applications. However, due to the large difference between the development environment and the actual application scenario, the migration effect of the relevant algorithms is not ideal. Given the high cost of re-collecting data and the fact that other objective conditions cannot be changed, it is necessary to invent a method that can successfully migrate the model from the existing laboratory scenario to the real application scenario.

[0004] For example, when the lighting conditions in the laboratory are good, the actual application scenarios usually do not have good lighting conditions. Therefore, when using the data set created in the laboratory scenario to train the garbage classification model, the accuracy will be reduced in the actual application scenario. In this example, we will take lighting as an example to invent an improved method for deep learning model image recognition for special situations. Summary of the invention

[0005] In order to solve the problem that the existing deep learning model image recognition has poor effect after the application scenario migration, the present invention provides a deep learning model image recognition improvement method based on target detection, which relies on the main differences in the scenes to weaken special features and improve the accuracy of deep learning model image recognition after the application scenario migration, so as to alleviate the problem of poor model recognition accuracy when the model has certain differences in application scenarios and data sets.

[0006] To achieve the above technical objectives, the solution of the present invention is as follows: A first aspect of an embodiment of the present invention provides an improved image recognition method based on garbage target detection, the method specifically comprising the following steps:

[0007] S1, the garbage data set is used as the sample data set after marking the spot area and normalizing it;

[0008] S2, construct and train a target detection model for light spots, input the sample data set into the target detection model for light spots, and obtain the light spot area features in the image;

[0009] S3, complement the features of the spot area to obtain the image pixel weight matrix;

[0010] S4, build and train the garbage classification model;

[0011] S5, input the image to be tested into the trained garbage classification model, multiply the pixel values ​​of all pixels in the image to be tested by the image pixel weight matrix and input them into the trained garbage classification model to obtain a classification result.

[0012] Furthermore, the images in the garbage dataset include waste paper boxes, glass, non-metal, paper, plastic, dry batteries, power banks, leftovers, fruit peels, and cigarette butts.

[0013] Furthermore, the target detection model for the light spot is a supervised model, and its label is the light spot area label of the marked light spot area. In this embodiment, it includes two parts: a feature extractor and a confidence generator, wherein the feature extractor uses a network composed of 5 convolutional layers, and the light spot confidence generator uses a network composed of 2 deconvolutional layers, and the activation function uses a Relu function.

[0014] Furthermore, the target detection model for light spots is trained as follows: each picture in the garbage data set is used as a label, training batches are set, a warm-up learning rate strategy is adopted, the optimizer is optimized using Adam, the loss function is in the form of peak signal-to-noise ratio (PSNR), and the validation set is used for verification until the accuracy of the model reaches the preset standard or the loss function converges to complete the training.

[0015] Furthermore, the formula for complementing the spot area features is as follows:

[0016] W model =IW

[0017] Where I represents a matrix with all elements set to 1, W represents the feature of the spot area, and the obtained W model is the image pixel weight matrix.

[0018] Furthermore, the garbage classification model is a supervised model, which includes two parts: a feature extractor and a classifier. The feature extractor adopts a network composed of 5 convolutional layers and the classifier adopts a network composed of 2 fully connected layers.

[0019] Furthermore, the process of training the garbage classification model is as follows: divide the garbage data set into a training set and a test set, use the training set to train the garbage classification model, input each picture into the model in turn, and calculate the picture pixel weight matrix, multiply the RGB values ​​of all pictures by the corresponding picture pixel weight matrix and then input them into the model to achieve the purpose of weakening the influence of light spots; set the training batch, adopt the warm-up learning rate strategy in the training stage, use Adam to optimize the optimizer, and use the cross entropy form for the loss function; verify the model recognition accuracy through the validation set until the model accuracy reaches the preset standard or the loss function converges to complete the training.

[0020] A second aspect of an embodiment of the present invention provides an electronic device, comprising a memory and a processor, characterized in that the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned improved image recognition method based on garbage target detection.

[0021] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the improved image recognition method based on garbage target detection as described above is implemented.

[0022] The beneficial effects of the present invention are as follows: the method of the present invention improves the accuracy of garbage classification and the environmental robustness of the garbage image classification method. The method of the present invention constructs and trains a target detection model for light spots, obtains light spot area features, and automatically marks the light spot features, thereby avoiding the problem of difficulty in manufacturing data sets and large workload of manual marking. The method of the present invention is a method that still makes the final model have better effects when the data set is imperfect, and solves the problem of differences between the data distribution of the data set and the distribution of the actual data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of an improved method for deep learning model image recognition based on target detection provided by an example of the present invention;

[0025] Figure 2 An algorithm block diagram of a light spot robustness garbage classification model construction based on an improved method of deep learning model image recognition for target detection provided by an example of the present invention;

[0026] Figure 3 A first example diagram of marking the features of the spot area;

[0027] Figure 4 A second example diagram of marking the features of the light spot area;

[0028] Figure 5 A first example diagram of marking the features of the spot area;

[0029] Figure 6 A schematic diagram of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0031] This example will take illumination as an example to introduce a method for improving the image recognition network of a deep learning model based on target detection.

[0032] In the previous model use process, we found that due to the good lighting conditions in the data set collection environment, there were obvious light spots in the glass category images. The model used the light spots as an important feature of the glass category data during the training process. However, in the subsequent model application scenarios, due to poor lighting conditions, the glass could not have obvious light spots, which caused the model's recognition accuracy for glass items to be greatly reduced.

[0033] In order to solve the problem of inaccurate classification results caused by the difference between the training data set and the actual application scenario. This embodiment provides an improved method for image recognition based on deep learning model of target detection, such as Figure 1 As shown, the improved method for image recognition based on deep learning model of target detection includes the following steps:

[0034] (1) Dataset preparation and preprocessing.

[0035] (1.1) Use the trash data set as the initial sample set:

[0036] The present invention uses the trash data set as the training data set for the improved method of deep learning model image recognition based on target detection. The trash data set is a data set that is cleaned and collected from various open source data sets collected on the Internet, that is, a garbage data set, which contains 80,000 pictures of different types, each of which is marked with categories, including 245 small categories and 4 large categories in total. The 4 large categories are recyclable garbage, kitchen waste, hazardous waste and other garbage. All bright spots in the picture are marked, and the bright spot area is demarcated to prepare for subsequent model training. The processed data set is recorded as D.

[0037] (1.2) Dataset preprocessing:

[0038] We retained 10 representative categories out of the 245 categories, namely: waste cartons, glass, non-metal, paper, plastic, dry cell batteries, power banks, leftovers, fruit peels, and cigarette butts. We converted all the images into the same size to facilitate the input into the model.

[0039] (2) Train the target detection model M for the light spot od :

[0040] Adding the information of the spot recognition model to the main task model can provide a basis for reducing the impact of lighting when the main task model performs garbage classification. After processing, the spot confidence weight matrix output by the target detection model can be used to assign weights to the input garbage feature image to reduce the impact of the spot in the image. In addition, during the training phase, the target detection model can be migrated with different data sets and different robust improvement targets, and the weights can be automatically adjusted during the model training, so that the weights corresponding to images with robust features are reduced, reducing their impact on the final classification results.

[0041] (2.1) Design of target detection model structure for light spots based on target detection

[0042] The spot recognition model is a supervised model. The input of the model is an RGB image of size 256*256, and the output is a 256*256 spot confidence feature weight matrix W indicating the possibility of each pixel having a spot. Its label is the spot area label of the spot area marked with a frame in step (1). In this embodiment, the constructed spot recognition model includes two parts: a feature extractor and a confidence generator. The feature extractor uses a network composed of 5 convolutional layers, and the spot confidence generator uses a network composed of 2 deconvolution layers. The activation function uses the Relu function.

[0043] The feature extractor is responsible for extracting the features of the input image, and during the extraction process, the semantic information in the feature map extracted by convolution will gradually shift from low-level semantic features to high-level semantic features as the number of layers increases. That is, the features extracted by the first and second layers of convolution will tend to be local, such as lines, simple geometric figures, etc. However, as the number of convolution layers increases, the semantic features will tend to be high-level, such as the outline of the object, the overall shape structure, etc.

[0044] This example mainly focuses on the light spot feature. The light spot is a relatively obvious feature and a low-level semantic feature, so its most obvious performance will be in the first few convolutional layers. Therefore, the feature extractor of the target detection model in this example does not require too deep convolutional layers, which also ensures the model recognition efficiency and greatly simplifies the model recognition difficulty.

[0045] The final model is recorded as the target detection model M for the light spot od .

[0046] (2.2) Target detection model M for light spots od Conduct training

[0047] Each image in D is used as a label, the size of the training batch is set to 100, the warm-up learning rate strategy is used in the training phase, and the optimizer is optimized using Adam. The loss function is in the form of peak signal-to-noise ratio (PSNR), and the formula is as follows:

[0048]

[0049]

[0050] Among them, K(i,j is the confidence that the pixel at the position corresponding to the model output result is a bright spot area, I(i,j is the confidence that the pixel at the position corresponding to the label is a bright spot area, MAXI represents the confidence matrix, W is the maximum value of the bright spot area, m and n are the width and height of the image respectively, both of which are 256 in this example. Use the validation set for validation until the accuracy of the model reaches the preset standard or the loss function converges to complete the training.

[0051] The sample data set is input into the target detection model for light spots to obtain the light spot area features in the image. Figure 3 , Figure 4 and Figure 5 As shown, the light spot area in the image is automatically marked with a marking box through the trained target detection model for light spots.

[0052] (3) Target detection weight complementation

[0053] The target detection model M for the light spot in step (2) odDuring the training process, the light spot information is weighted using target detection, so that the model can highlight the features of the light spot more according to the target detection, so that the model can finally correctly classify the light spot. However, in the main task, we hope that the model recognition accuracy is less affected by the lighting environment, that is, the light spot has less impact on the classification result, which is consistent with the target detection M for the light spot. od The purpose of feature extraction is opposite. The formula is as follows

[0054] W model =IW

[0055] Where I represents a matrix of size 256*256 with all elements being 1, and the obtained W model is the image pixel weight matrix.

[0056] Therefore, we can make the main task model less affected by the illumination by complementing and weighting the original image. At the same time, since the light spot feature is a low-level semantic feature, the detection of the light spot target is relatively simple. Therefore, the model can still have a good recognition efficiency during use.

[0057] (4) Training the spot robust garbage classification model M

[0058] (4.1) Constructing the garbage classification model M

[0059] The garbage classification model is a supervised model. The input of the model is a picture of size 256*256, and the output is a ten-dimensional variable indicating the possibility of image classification (i.e., a weight matrix indicating the possibility that each pixel of the image is a light spot area). In this embodiment, the constructed garbage classification model includes a feature extractor and a classifier, wherein the feature extractor uses a network composed of 5 convolutional layers and the classifier uses a network composed of 2 fully connected layers, and the model is denoted as M.

[0060] (4.2) Training the garbage classification model M

[0061] Divide the dataset D into a training set and a test set, use the training set to train M, and input each image into the model M in turn. od And finally get the weight parameter W according to the method of step (3) model , the RGB values ​​of all images are multiplied by the weight parameter matrix and then input into the model to achieve the purpose of weakening the effect of light spots. The size of the training batch is set to 100, and the warm-up learning rate strategy is adopted in the training stage. The optimizer uses Adam for optimization. The loss function is in the form of cross entropy, and the formula is as follows:

[0062]

[0063] where p iRepresents the probability of being classified into each category of samples.

[0064] The model recognition accuracy is verified through the validation set. If the accuracy reaches the preset requirements, the model training is completed.

[0065] (5) Realize spot robust garbage classification

[0066] (5.1) The image to be predicted is input into the model M od

[0067] For a garbage picture that needs to be classified, input it into the spot target detection model M od In the above equation, we get the light spot prediction confidence matrix W.

[0068] (5.2) Target detection weight complement

[0069] In the training process of model M in step (2), the light spot information is weighted using target detection, so that the model can highlight the characteristics of the light spot more according to target detection, so that the model can finally correctly classify the light spot. However, in the main task, we hope that the model recognition accuracy is less affected by the lighting environment, that is, the light spot has less impact on the classification result, which is consistent with M od The purpose of feature extraction is opposite. The formula is as follows

[0070] W model =IW

[0071] Where I represents a matrix of size 256*256 with all elements being 1.

[0072] (5.3) Weighted input model of the graph to be predicted

[0073] Each image to be predicted is first input into the spot target detection model to obtain the spot confidence matrix, and the image pixel value weight matrix W is calculated model , sum the pixel values ​​of all pixels on each image to be classified and W model After multiplication, the data is input into the garbage classification model, and the prediction result output by the model is the classification result of the current main task model for the current image.

[0074] Corresponding to the above-mentioned embodiment of the method for improving image recognition based on garbage target detection, the present invention also provides an embodiment of an apparatus for improving image recognition based on garbage target detection.

[0075] See also Figure 6 An embodiment of the present invention provides an improved image recognition device based on garbage target detection, comprising one or more processors for implementing the improved image recognition method based on garbage target detection in the above embodiment.

[0076] The embodiments of the image recognition improvement device based on garbage target detection of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 6 As shown in the figure, it is a hardware structure diagram of any device with data processing capability where the image recognition improvement device based on garbage target detection of the present invention is located. Figure 6 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0077] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0078] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0079] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the improved image recognition method based on garbage target detection in the above embodiment is implemented.

[0080] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An improved method for image recognition based on garbage target detection, characterized in that: The method specifically comprises the following steps: S1, the garbage data set is used as the sample data set after marking the spot area and normalizing it; S2, construct and train a target detection model for light spots, input the sample data set into the target detection model for light spots, and obtain the light spot area features in the image; S3, complement the features of the spot area to obtain the image pixel weight matrix; S4, build and train the garbage classification model; The garbage classification model is a supervised model, which consists of two parts: a feature extractor and a classifier. The feature extractor uses a network consisting of five convolutional layers, and the classifier uses a network consisting of two fully connected layers. The process of training the garbage classification model is as follows: divide the garbage data set into a training set and a test set, use the training set to train the garbage classification model, input each picture into the model in turn, calculate the picture pixel weight matrix, multiply the RGB values ​​of all pictures by the corresponding picture pixel weight matrix and then input them into the model to achieve the purpose of weakening the influence of light spots; set the training batch, use the warm-up learning rate strategy in the training stage, use Adam to optimize the optimizer, and use the cross entropy form for the loss function; verify the model recognition accuracy through the validation set until the model accuracy reaches the preset standard or the loss function converges to complete the training; S5, input the image to be tested into the trained garbage classification model, multiply the pixel values ​​of all pixels in the image to be tested by the image pixel weight matrix and input them into the trained garbage classification model to obtain a classification result.

2. The improved image recognition method based on garbage target detection according to claim 1 is characterized in that: The images in the garbage dataset include waste cartons, glass, paper, plastic, dry batteries, power banks, leftovers, fruit peels, and cigarette butts.

3. The improved image recognition method based on garbage target detection according to claim 1 is characterized in that: The target detection model for light spots is a supervised model, and its label is the spot area label of the marked spot area; the target detection model for light spots includes two parts: a feature extractor and a confidence generator. The feature extractor adopts a network composed of 5 convolutional layers, and the light spot confidence generator adopts a network composed of 2 deconvolution layers, and the activation function adopts the Relu function.

4. The improved image recognition method based on garbage target detection according to claim 1 is characterized in that: The specific steps for training the target detection model for light spots are as follows: each picture in the garbage data set is used as a label, the training batch is set, the warm-up learning rate strategy is adopted, the optimizer is optimized using Adam, the loss function is in the form of peak signal-to-noise ratio, and the validation set is used for verification until the accuracy of the model reaches the preset standard or the loss function converges to complete the training.

5. The improved image recognition method based on garbage target detection according to claim 1 is characterized in that: The formula for complementing the spot area features is as follows: W model =IW Among them, I represents a matrix with all elements equal to 1, W is the feature of the spot area, and the obtained W model is the image pixel weight matrix.

6. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the improved image recognition method based on garbage target detection as described in any one of claims 1-5 above.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the improved image recognition method based on garbage target detection as described in any one of claims 1 to 5 is implemented.

Images