Method for Classifying and Identifying Recyclable Items

Abstract

The embodiment of the present application belongs to the technical field of recyclable item classification, and relates to a method for classifying and identifying recyclable items, including obtaining RGB data and IR data of recyclable items, and normalizing the RGB data and IR data respectively; inputting the normalized RGB data into an RGB feature extraction network to obtain an RGB feature map; inputting the normalized IR data into an IR feature extraction network to obtain an IR feature map, merging the RGB feature map and the IR feature map into a feature data set, and inputting the feature data set into a classification network to obtain the classification of the recyclable items, wherein the recyclable item classification includes metal waste, textile waste, and glass waste, and the metal waste classification includes iron, aluminum, and copper sub-classifications. The present application also provides an X device, a computer device, and a storage medium. The solution can improve the recognition success rate of recyclable items.

Description

Technical Field

[0001] The present application relates to the technical field of recyclable item classification, and in particular to a method for classifying and identifying recyclable items. Background Art

[0002] An automated recyclables recycling system needs to identify different types of recyclables so that they can be recycled separately or customers can be provided with recyclables processing fees. Automatic identification of recyclables is usually performed through pictures or images, and image features are extracted, sampled and classified through convolutional neural networks. Convolutional neural networks include multiple layers of processing, and each layer of processing involves several parameters, especially for processing image data. Because the amount of image data itself is very large, the amount of calculation is huge in the process of processing data. Although the existing method of image classification through convolutional neural networks has a high accuracy in the laboratory environment, in the application environment of recyclables recycling, because the ambient lighting conditions for recycling recyclables are usually weak, the features contained in the images extracted by the traditional RGB method are not obvious, so the actual recognition success rate of recyclables is low. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a recyclable item classification and identification method that can accurately identify the types of recyclable items.

[0004] In order to solve the above technical problems, the embodiment of the present application provides a method for classifying and identifying recyclable items, which adopts the following technical solution:

[0005] A method for classifying and identifying recyclable items, the method comprising:

[0006] Acquire RGB data and IR data of recyclable items, and normalize the RGB data and IR data respectively;

[0007] Inputting the normalized RGB data into an RGB feature extraction network to obtain an RGB feature map, wherein the RGB feature map represents the shape and texture of the recyclable item;

[0008] Input the normalized IR data into an IR feature extraction network to obtain an IR feature map, wherein the IR feature map represents the material of the recyclable item, and the IR feature map and the RGB feature map have the same length and width;

[0009] The RGB feature map and the IR feature map are merged into a feature data set, and the feature data set is input into a classification network to obtain the classification of the recyclable items, wherein the recyclable item classification includes metal waste, textile waste and glass waste, and the metal waste classification includes at least one sub-classification of iron, aluminum and copper.

[0010] Furthermore, the RGB feature extraction network and the IR feature extraction network respectively include a first convolutional network, the first convolutional network shares parameters with a deep residual network, and the training method of the RGB feature extraction network and the IR feature extraction network specifically includes:

[0011] Extracting a first convolutional network: training the deep residual network through a training set; forming a first convolutional network with parameters discarded by the deep residual network; and scoring the first convolutional network through a test set;

[0012] The deep residual network is cyclically executed to extract the first convolutional network to obtain multiple groups of first convolutional networks and scores corresponding to the first convolutional networks.

[0013] Multiple groups of the first convolutional networks with the highest scores are respectively designated as convolutional layers in the RGB feature extraction network or the IR feature extraction network.

[0014] Furthermore, in the deep residual network, the scaling parameter γ in the last BN layer in each residual block is set to 0 to output an all-0 vector.

[0015] Furthermore, the learning rate of the deep residual network is set to exponential decay, and the minimum learning rate of the deep residual network is set to 5% of the initial learning rate.

[0016] Furthermore, in the process of training the deep residual network through the training set, when a parameter in a convolution kernel in the deep residual network is less than a preset value, the parameter is set to 0 so that each convolution kernel forms a sparse matrix.

[0017] Furthermore, the non-zero parameters in the sparse matrix are concentrated to form blocks.

[0018] Furthermore, the depth of the deep residual network is set to 50 layers.

[0019] Furthermore, the method also includes: obtaining the weight of the recyclable items, and classifying the recyclable items to match the weight.

[0020] Furthermore, the method further includes: obtaining a classification weight of recyclable items under a classification;

[0021] Get the volume of recyclable items in a category;

[0022] When the classification weight or classification volume exceeds a preset value, a signal requiring recycling is issued.

[0023] In order to solve the above technical problems, the embodiment of the present application also provides a recyclable item classification and identification device, which adopts the following technical solution:

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0025] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for classifying and identifying recyclable items are implemented.

[0026] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: RGB data and IR data are obtained for recyclable items respectively, and RGB feature maps and IR feature maps are extracted respectively, wherein the RGB feature maps and IR feature maps reflect the image features of a recyclable item.

[0027] The RGB feature map contains relatively rich information, which can reflect the outer contour and texture of recyclable items, and is helpful for identifying the specific categories of recyclable items.

[0028] The IR characteristic map mainly reflects the material properties of recyclable items through reflection. IR data is obtained through infrared rays. Therefore, under various lighting conditions, the IR characteristic map can reflect the material properties of recyclable items.

[0029] The length and width of the IR feature map are equal to those of the RGB feature map. Therefore, the feature data set obtained by combining the IR feature map and the RGB feature map with the information in the RGB feature map and the IR feature map is put into the classification network to form a four-layer data set of RGBY. ​​After unified processing, it can classify the recyclable items in combination with their contours, textures and materials under various lighting environments. This solution can improve the success rate of identifying recyclable items. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0032] Figure 2 A flowchart of an embodiment of a method for classifying and identifying recyclable items according to the present application;

[0033] Figure 3 yes Figure 2 Flowchart of the training method of the RGB feature extraction network and the IR feature extraction network described in;

[0034] Figure 4 yes Figure 2 Additional flow chart of the method for classifying and identifying recyclable items in the middle step;

[0035] Figure 5 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the system architecture in which the technical solution described in the present application operates may include a terminal device 11, a terminal device 12, a terminal device 13, a network 14, and a server 15. The network 14 is used to provide a medium for a communication link between the terminal devices 11, 12, 13, and the server 15. The network 14 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0040] The user can use the terminal device 11, the terminal device 12, and the terminal device 13 to interact with the server 15 through the network 14 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 11, the terminal device 12, and the terminal device 13, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0041] Terminal device 11, terminal device 12, and terminal device 13 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0042] The server 15 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 11 , 12 , and 13 .

[0043] It should be noted that the recyclable item classification and identification method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, a recyclable item classification and identification device is generally arranged in a server / terminal device.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0045] refer to Figure 2 , showing a flow chart of an embodiment of a method for classifying and identifying recyclable items according to the present application.

[0046] A method for classifying and identifying recyclable items, the method comprising:

[0047] Step S100: Acquire RGB data and IR data of recyclable items, and normalize the RGB data and IR data respectively;

[0048] Step S200: inputting the normalized RGB data into an RGB feature extraction network to obtain an RGB feature map, wherein the RGB feature map represents the shape and texture of the recyclable item;

[0049] Step S300: inputting the normalized IR data into an IR feature extraction network to obtain an IR feature map, wherein the IR feature map represents the material of the recyclable item, and the IR feature map and the RGB feature map have the same length and width;

[0050] Step S400: Merge the RGB feature map and the IR feature map into a feature data set, and input the feature data set into a classification network to obtain the classification of the recyclable items, wherein the recyclable items classification includes metal waste, textile waste and glass waste, and the metal waste classification includes at least one sub-classification of iron, aluminum and copper. In one embodiment, the three sub-classifications of iron, aluminum and copper can be provided at the same time, or the two sub-classifications of iron and aluminum or iron and copper can be provided.

[0051] The embodiments of the present application mainly have the following beneficial effects: RGB data and IR data are obtained for recyclable items respectively, and RGB feature maps and IR feature maps are extracted respectively, wherein the RGB feature map and IR feature map reflect the image features of a recyclable item.

[0052] The RGB feature map contains relatively rich information, which can reflect the outer contour and texture of recyclable items, and is helpful for identifying the specific categories of recyclable items.

[0053] The IR characteristic map mainly reflects the material properties of recyclable items through reflection. IR data is obtained through infrared rays. Therefore, under various lighting conditions, the IR characteristic map can reflect the material properties of recyclable items.

[0054] The length and width of the IR feature map are equal to those of the RGB feature map. Therefore, the feature data set obtained by combining the IR feature map and the RGB feature map with the information in the RGB feature map and the IR feature map is put into the classification network to form a four-layer data set of RGBY. ​​After unified processing, it can classify the recyclable items in combination with their contours, textures and materials under various lighting environments. This solution can improve the success rate of identifying recyclable items.

[0055] refer to Figure 3 Furthermore, the RGB feature extraction network and the IR feature extraction network respectively include a first convolutional network, the first convolutional network shares parameters with a deep residual network, and the training method of the RGB feature extraction network and the IR feature extraction network specifically includes:

[0056] Step S11: extracting a first convolutional network: training the deep residual network through a training set; forming a first convolutional network with parameters discarded by the deep residual network; and scoring the first convolutional network through a test set;

[0057] Step S12: extracting the first convolutional network in a loop on the deep residual network to obtain multiple groups of first convolutional networks and scores corresponding to the first convolutional networks.

[0058] Step S13: respectively designate multiple groups of the first convolutional networks with the highest scores as convolutional layers in the RGB feature extraction network or the IR feature extraction network.

[0059] The RGB feature extraction network and the IR feature extraction network are trained separately. The steps required for training are the same. The required training set and parameter adjustment are set according to the requirements of the RGB feature extraction network or the IR feature extraction network. Taking the RGB feature extraction network as an example, this method trains the deep residual network through the training set. During the training process, in order to prevent the gradient from disappearing, the deep residual network randomly discards some parameters, and forms a first convolutional network with a smaller parameter scale by combining the discarded parameters with the feature data set.

[0060] As the training progresses, the first convolutional network is extracted once in each round of training. The parameters discarded by the deep residual network are random each time, so the discarded parameters are also trained. The overall performance of the obtained first convolutional network is also improved. Several first convolutional networks generated after several trainings are scored.

[0061] For the extracted multiple first convolution networks, the one with the highest score is extracted as the convolution part in the RGB feature extraction network. Similarly, the convolution part in the IR feature extraction network is also extracted and determined in the same way. This solution can ensure the accuracy of feature extraction, and on the basis of ensuring the classification accuracy, it can greatly reduce the size of the RGB feature extraction network and the IR feature extraction network, and reduce the number of parameters of the RGB feature extraction network and the IR feature extraction network, so as to reduce the computational overhead in the feature extraction process. Improve the efficiency of recyclable item recognition.

[0062] In one embodiment, the feature extraction accuracy of the first network obtained after several trainings are 87.4%, 88.6%, 92.7%, 94.1%, 96.2%, and 97.5%, respectively, while the feature extraction accuracy of the residual network itself after final training is 98.7%. It can be seen that the feature extraction accuracy of the first network with the highest accuracy is very close to the level of the residual network, while the computational overhead is greatly reduced.

[0063] Furthermore, in the deep residual network, the scaling parameter γ in the last BN layer in each residual block is set to 0 to output an all-0 vector.

[0064] By setting the parameter γ to 0 to output an all-0 vector, the residual block outputs an all-0 vector before the residual connection, thereby reducing the size of the residual network output matrix and improving the training efficiency of the first network. This solution can speed up the deployment of recyclables identification methods in any environment.

[0065] Furthermore, the learning rate of the deep residual network is set to exponential decay, and the minimum learning rate of the deep residual network is set to 5% of the initial learning rate.

[0066] This scheme maintains the learning rate of the deep residual network to prevent overfitting during the training of the first network. This scheme improves the richness of the recyclable features extracted by the first network to improve the accuracy of recyclable classification.

[0067] Furthermore, in the process of training the deep residual network through the training set, when a parameter in a convolution kernel in the deep residual network is less than a preset value, the parameter is set to 0 so that each convolution kernel forms a sparse matrix.

[0068] Specifically, this solution will reduce the number of parameters in the deep residual network and the first convolutional network. For some parameters whose values ​​are close to 0, they cannot provide obvious features and occupy computer resources. By setting these parameters to zero, the computational complexity of the neural network can be greatly reduced. At the same time, it can ensure that the features of recyclable items will not be lost. This solution improves the recognition efficiency of recyclable items.

[0069] Furthermore, the non-zero parameters in the sparse matrix are concentrated to form blocks.

[0070] This solution can concentrate the feature extraction in the image, and the recyclables can provide sufficient features at the position. This solution improves the accuracy of feature extraction.

[0071] Furthermore, the depth of the deep residual network is set to 50 layers.

[0072] Furthermore, the method also includes: obtaining the weight of the recyclable items, and classifying the recyclable items to match the weight.

[0073] refer to Figure 4 Furthermore, the method further includes: Step S501: obtaining the classification weight of recyclable items under a classification;

[0074] Step S502: Obtain the classification volume of recyclable items under a classification;

[0075] Step S503: sending a signal requiring recycling according to the classification weight or classification volume exceeding a preset value.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0077] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0078] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0079] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated CIRcuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0080] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0081] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as the program code of the recyclable item classification and identification method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0082] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the program code stored in the memory 61 or process data, such as running the program code of the recyclable item classification and identification method.

[0083] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0084] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores a recyclable item classification and identification program, and the recyclable item classification and identification program can be executed by at least one processor so that the at least one processor performs the steps of the recyclable item classification and identification method as described above.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0086] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for classifying and identifying recyclable items, characterized in that: The method includes: Acquire RGB data and IR data of recyclable items, and normalize the RGB data and IR data respectively; Inputting the normalized RGB data into an RGB feature extraction network to obtain an RGB feature map, wherein the RGB feature map represents the shape and texture of the recyclable item; Input the normalized IR data into an IR feature extraction network to obtain an IR feature map, wherein the IR feature map represents the material of the recyclable item, and the IR feature map and the RGB feature map have the same length and width; The RGB feature extraction network and the IR feature extraction network respectively include a first convolutional network, the first convolutional network shares parameters with a deep residual network, and the training method of the RGB feature extraction network and the IR feature extraction network specifically includes: Extracting the first convolutional network: training the deep residual network through the training set; during the training process, in order to prevent the gradient from disappearing, the deep residual network randomly discards a part of the parameters; the parameters discarded by the deep residual network form the first convolutional network; the first convolutional network has a smaller parameter scale than the deep residual network; scoring the first convolutional network through the test set; Looping through the deep residual network to extract the first convolutional network, so as to obtain multiple groups of first convolutional networks and scores corresponding to the first convolutional network; Specifying the groups of the first convolutional networks with the highest scores as convolutional layers in the RGB feature extraction network or the IR feature extraction network respectively; The RGB feature map and the IR feature map are merged into a feature data set, and the feature data set is input into a classification network to obtain the classification of the recyclable items, wherein the recyclable item classification includes metal waste, textile waste and glass waste, and the metal waste classification includes at least one sub-classification of iron, aluminum and copper.

2. A method for classifying and identifying recyclable items according to claim 1, characterized in that: In the deep residual network, the scaling parameter γ in the last BN layer in each residual block is set to 0 to output an all-0 vector.

3. A method for classifying and identifying recyclable items according to claim 1, characterized in that: The learning rate of the deep residual network is set to exponential decay, and the minimum learning rate of the deep residual network is set to 5% of the initial learning rate.

4. A method for classifying and identifying recyclable items according to claim 1, characterized in that: During the process of training the deep residual network through the training set, when a parameter in a convolution kernel in the deep residual network is less than a preset value, the parameter is set to 0 so that each convolution kernel forms a sparse matrix.

5. A method for classifying and identifying recyclable items according to claim 4, characterized in that: The non-zero parameters in the sparse matrix are concentrated to form blocks.

6. A method for classifying and identifying recyclable items according to claim 1, characterized in that: The depth of the deep residual network is set to 50 layers.

7. A method for classifying and identifying recyclable items according to claim 1, characterized in that: The method further includes obtaining a weight of the recyclable items and sorting the recyclable items to match the weight.

8. A method for classifying and identifying recyclable items according to claim 7, characterized in that: The method also includes: obtaining a classification weight of recyclable items under a classification; Get the volume of recyclable items in a category; When the classification weight or classification volume exceeds a preset value, a signal requiring recycling is issued.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of a method for classifying and identifying recyclable items as claimed in any one of claims 1 to 8 are implemented.

Images