Municipal solid waste collection and transportation method for classification treatment

Abstract

An urban solid waste collection and transportation method for classification treatment comprises the steps that picture information of solid waste is collected, and multiple pieces of sub-picture information are obtained after preprocessing; obtaining frequency spectrum feature data in each piece of sub-picture information; inputting the spectrum feature data into a pre-trained convolutional neural network module, and calculating a first feature vector corresponding to each solid waste in the sub-picture information; calculating a node feature matrix based on a trained graph neural network according to the first feature vector corresponding to each solid waste; and outputting a second classification result corresponding to each solid waste according to the node feature matrix. According to the method, on the basis of the convolutional neural network, the graph neural network is creatively introduced, the feature defect of the single solid waste is strengthened by associating the feature information of all the solid wastes in the same batch, and then the category of each solid waste can be more accurately judged.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more specifically, relates to a method for collecting and transporting urban solid waste for classified processing. Background Art

[0002] The classification and treatment of urban solid waste is an important environmental protection and resource recycling task. Through effective classification and treatment, environmental pollution can be minimized and sustainable resource utilization can be promoted. When dealing with urban solid waste, using convolutional neural networks to classify and identify image information containing solid waste can help achieve waste classification. However, in actual situations, due to the characteristics of solid waste itself, it is difficult to correctly classify image information.

[0003] Solid waste is characterized by incompleteness, which means that when it becomes garbage, it is often already damaged. For example, plastic boxes are often torn into pieces, cans are trampled, or bicycle parts are incomplete. These factors themselves will greatly affect the sample information, that is, the image information, thereby failing to or reducing the accuracy of convolutional neural network training, and thus causing the convolutional neural network module to be unable to correctly identify the type of solid waste.

[0004] Prior art CN112620165A discloses a method for automatically generating garbage images that can be used for garbage classification, including: manually collecting a number of garbage images, converting the training sample data of the original training sample set into TFRecord format; using a deep convolutional neural network to construct a generative network; using a sparse autoencoder deep convolutional neural network to construct a discriminant network; using an alternating iterative optimization method to train the generative network and the discriminant network until the model converges; establishing a garbage image generator to generate garbage images. However, the main feature of this method is cost saving, and it does not take into account the incomplete characteristics of urban solid waste. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the purpose of the present invention is to solve the above-mentioned defects and further propose a method for collecting and transporting urban solid waste for classified treatment.

[0006] The present invention adopts the following technical solution.

[0007] The first aspect of the present invention discloses a method for collecting and transporting urban solid waste for classified treatment, characterized in that it comprises steps 1 to 5;

[0008] Step 1, collecting the image information of the solid waste, and obtaining a plurality of sub-image information after preprocessing, wherein each sub-image information is associated with only one solid waste;

[0009] Step 2, obtaining the spectrum feature data in each sub-image information;

[0010] Step 3, inputting the spectrum feature data into a pre-trained convolutional neural network module to calculate the first feature vector corresponding to each solid waste in the sub-image information;

[0011] Step 4: Calculate the node feature matrix based on the trained graph neural network according to the first feature vector corresponding to each solid waste;

[0012] Step 5: Based on the graph neural network and the node feature matrix, output the second classification result corresponding to each solid waste.

[0013] The second aspect of the present invention discloses a municipal solid waste collection and transportation system for classified processing, which is used to execute the method described in the first aspect, and the system includes: a camera, a communication device, and a server terminal;

[0014] The camera is used to collect the image information of the solid waste; the communication device is used to upload the image information to the server terminal;

[0015] The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module and a graph neural network module;

[0016] The preprocessing module is used to preprocess the image information to obtain a plurality of sub-image information, wherein each sub-image information is associated with only one solid waste;

[0017] The Fourier transform module is used to obtain the frequency spectrum feature data in each sub-image information;

[0018] The convolutional neural network module includes a convolution layer, a feature extraction layer, a pooling layer, and a fully connected layer; the convolution layer is used to convolve the spectral feature data; the feature extraction layer is used to extract the time series feature data from the convolution result; the pooling layer is used to calculate the feature vector corresponding to each solid waste from the time series feature data;

[0019] The graph neural network module includes a graph convolution layer, a graph pooling layer and a graph fully connected layer; the graph convolution layer is used to calculate the node feature matrix according to the first feature vector corresponding to each solid waste; the graph pooling layer is used to reduce the dimension of the node feature matrix, and the graph fully connected layer outputs the second classification result according to the node feature matrix after the dimension reduction process.

[0020] The third aspect of the present invention discloses a terminal, including a processor and a storage medium; the characteristics are:

[0021] The storage medium is used to store instructions;

[0022] The processor is configured to operate according to the instructions to execute the method described in the first aspect.

[0023] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0024] The beneficial effects of the present invention are that, compared with the prior art, the present invention has the following advantages:

[0025] Based on the convolutional neural network, the present invention creatively introduces a graph neural network, which makes up for the incomplete characteristics of the solid waste by associating the characteristic information of all solid wastes in the same batch, and thus can more accurately judge the category of each solid waste.

[0026] The present invention extracts the color tone matrix for analysis and processing, thereby uniformly filtering the image information under illumination at different times. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The present invention is a flow chart of a method for collecting and transporting urban solid waste for classified treatment. DETAILED DESCRIPTION

[0028] The present application is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present application.

[0029] The present invention discloses a method for collecting and transporting urban solid waste oriented to classified treatment, such as Figure 1 As shown, it includes steps 1 to 5.

[0030] Step 1: collect image information of solid waste, and obtain multiple sub-image information after preprocessing, wherein each sub-image information is associated with only one solid waste.

[0031] In some embodiments, preprocessing includes at least: normalization processing and target detection. Target detection is to determine the area of ​​a single solid waste in the image information, which can be detected using the YOLO algorithm. The idea of ​​the YOLO algorithm is to cut the image information into grids, predict the bounding box and category probability of the target based on each grid, and thus cut each target, i.e., solid waste, out of the image information.

[0032] Step 2: Obtain the frequency spectrum feature data in each sub-image information.

[0033] Each pixel in the sub-image information usually includes information in three dimensions, for example, it can be RGB, YUV or HSB. RGB refers to red, green and blue respectively; YUV refers to brightness, first chromaticity and second chromaticity respectively; HSB refers to hue, saturation and brightness respectively. Considering that the pictures of solid waste can be taken at different times of the day, in order to ignore the changes in the sub-image information caused by solid waste under different brightness and different lighting conditions, in some embodiments, step 2 specifically includes steps 2.1 to 2.2.

[0034] Step 2.1: extract the hue value of each pixel in the sub-image information according to the HSB model and establish a hue matrix.

[0035] Step 2.2, using two-dimensional discrete Fourier transform to obtain the spectrum feature data in the hue matrix.

[0036] Specifically, the spectrum feature data F(x,y) is as follows:

[0037] p uv =(h uv ,s uv ,b uv )

[0038]

[0039] Among them, p uv is the value of the pixel in the sub-image information, which includes information in three dimensions, h uv ,s uv ,b uv Represent the hue value, saturation value and brightness value respectively, u and v represent the row and column of the pixel points in the sub-picture information respectively, where u=1,2,…,m, v=1,2,…,n; m and n are the width and height of the sub-picture information respectively; H is the hue matrix, i is an imaginary number, x and y are the horizontal and vertical coordinates in the frequency domain respectively.

[0040] It is understandable that since hue is not affected by lighting conditions, when photographing an object at different time points, even if the lighting conditions change, the hue value can remain relatively stable. This allows the HSV model to ignore the errors caused by different time points when processing colors.

[0041] Step 3: Input the spectral feature data into a pre-trained convolutional neural network (CNN) to calculate the first feature vector corresponding to each solid waste in the sub-image information.

[0042] Specifically, the convolutional neural network includes: a convolutional layer, a feature extraction layer, a pooling layer, and a fully connected layer. The convolutional layer is used to convolve the spectral feature data; the feature extraction layer is used to extract the time series feature data from the convolution result; the pooling layer is used to calculate the first feature vector corresponding to each solid waste from the time series feature data; the fully connected layer outputs the first classification result of the solid waste based on the first feature vector.

[0043] It should be noted that, in an embodiment of the present invention, the fully connected layer of the convolutional neural network is replaced by a graph neural network.

[0044] Step 4: According to the first feature vector corresponding to each solid waste, based on the trained graph neural network (GNN), a node feature matrix is ​​calculated.

[0045] The graph neural network is a K-layer model, each layer is a graph convolution layer, where K is the number of graph convolution layers, which is a fixed value. The larger the value, the greater the training effect, but the longer the iteration time. In the feature graph, each node corresponds to a solid waste, and the weight between nodes is used to characterize the first feature vector of the two and their positional relationship in the sub-image information.

[0046] Step 4 is specifically shown in the following formula:

[0047]

[0048] Among them, P s is the first eigenvector corresponding to the s-th solid waste, where s = 1, 2, ..., S, S is the number of all solid wastes; A is an adjacency matrix, whose element values ​​are all 1 or 0, indicating whether two nodes are adjacent; D is a degree matrix, which is a diagonal matrix. It can be understood that the value of each element on the diagonal is the number of associated neighbor nodes, and the value of each row of elements in the degree matrix, only the value of the element on the main diagonal is not 0, and the value of the element is the sum of the values ​​of all elements in the same row in A; ε is an activation function, which can be a sigmoid function or a ReLu function; Z k is the node feature matrix of the kth layer, k = 0, 1, 2, ..., K, W k Represents the weight matrix of the kth layer, which is obtained through graph neural network training. It can be understood that as a model parameter of the graph neural network, it can be continuously optimized by, for example, a back propagation algorithm.

[0049] It should be considered that there is a large difference between the test samples and actual samples of municipal solid waste. Municipal solid waste can include electronic waste, metal waste, plastic waste, etc. Most waste is often damaged when it becomes garbage. In addition to being exposed to the sun, wind and rain, it can also be that plastic boxes are often torn, cans are trampled, or bicycle parts are incomplete. These factors themselves will affect the accuracy of the convolutional neural network, resulting in the inability of the convolutional neural network to correctly identify the type of solid waste. However, from a common sense perspective, it should be considered that all solid wastes in the same sub-image information or the same batch of sub-image information have a certain correlation. For example, domestic waste is often dominated by plastic waste and kitchen waste, and industrial waste is mainly dominated by metal waste and electronic waste. Furthermore, solid wastes that are closer in a picture are often considered to be the same type of solid waste, assuming that their corresponding first eigenvectors are similar, and then the first eigenvectors of the two can be aggregated. Based on this idea, the present invention creatively introduces a graph neural network and takes each solid waste as a node in the feature graph, thereby utilizing this correlation to strengthen the feature information corresponding to each solid waste (for example: the first feature vector in step 3), and then being able to identify the type of each solid waste separately.

[0050] It should be noted that the “same batch” mentioned above refers to all sub-image information obtained at the same location and at the same time.

[0051] The significance of graph neural networks is that they can establish connections between different nodes, and are usually used in the fields of social network analysis, recommendation systems, knowledge graphs, etc. On this basis, the present invention creatively utilizes the ability of graph neural networks to aggregate information of nodes (equivalent to local information) and their surrounding nodes, thereby capturing more global and richer feature information.

[0052] Based on this, the adjacency matrix in the graph neural network is calculated based on the first eigenvector corresponding to each solid waste in the sub-image information.

[0053] In some embodiments, element a of the adjacency matrix A ij It can be shown as follows:

[0054]

[0055] w(i,j)=(1-λ)f1(P i ,P j ,Out i ,Out j )+λf2(Pos i ,Pos j )

[0056] Wherein, λ is a weight coefficient, which can be set to be slightly greater than 0, and can be set to 0.1 in the embodiment of the present invention. R is a preset correlation threshold, w(i,j) represents the correlation between the i-th and j-th solid wastes, P i ,P j is the first eigenvector corresponding to the i-th and j-th solid waste; Out i ,Out j are the first classification results corresponding to the i-th and j-th solid wastes, respectively. i ,Pos j are the position coordinates of the i-th and j-th solid waste in the sub-image information; f1 is used to measure Out i With Out j f2 is the normalized Euler distance function.

[0057] It is understandable that if Out i With Out j The first eigenvector of is completely different, then the value of f1 must be 0; and as Out i With Out j The higher the similarity of the elements contained in the i With Out j If the elements contained in are exactly the same, the value of f1 is 1. Normalization of f2 means limiting the value of f2 to [0, 1].

[0058] In some embodiments, in f1, Out i With Out j It does not participate in the calculation, but is only used as a verification method (i.e., the value of f1 is verified according to the method in the previous paragraph). i or Out j There are problems such as inconsistent vector lengths. It is understandable that, in this embodiment, the calculation process of f1 is basically consistent with the calculation process of the fully connected layer of the convolutional neural network.

[0059] Step 5: Based on the graph neural network and the node feature matrix, output the second classification result corresponding to each solid waste.

[0060] It can be understood that in step 4, the node feature matrix is ​​calculated by the graph convolution layer in the graph neural network, and the graph neural network also includes a graph pooling layer and a graph fully connected layer. The graph pooling layer is used to merge features, that is, to reduce the dimension of the node feature matrix, and the graph fully connected layer outputs the final prediction result, that is, the second classification result, based on the node feature matrix after dimensionality reduction.

[0061] Correspondingly, the present invention also discloses a municipal solid waste collection and transportation system for classified treatment, comprising: a camera, a communication device, and a server terminal.

[0062] Among them, the camera is used to collect image information of solid waste; the communication device is used to upload the image information to the server terminal.

[0063] The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module and a graph neural network module.

[0064] The preprocessing module is used to preprocess the image information to obtain a plurality of sub-image information, wherein each sub-image information is associated with only one solid waste.

[0065] The Fourier transform module is used to obtain the frequency spectrum feature data in each sub-image information.

[0066] The convolutional neural network module includes a convolution layer, a feature extraction layer, a pooling layer and a fully connected layer; the convolution layer is used to convolve the spectral feature data; the feature extraction layer is used to extract the time series feature data from the convolution result; the pooling layer is used to calculate the feature vector corresponding to each solid waste from the time series feature data.

[0067] The graph neural network module includes a graph convolution layer, a graph pooling layer and a graph fully connected layer; the graph convolution layer is used to calculate the node feature matrix according to the first feature vector corresponding to each solid waste; the graph pooling layer is used to reduce the dimension of the node feature matrix, and the graph fully connected layer outputs the second classification result according to the node feature matrix after the dimension reduction process.

[0068] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processors involved in each embodiment provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., but are not limited thereto.

[0069] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for collecting and transporting urban solid waste for classified treatment, characterized in that: Comprising steps 1 to 5; Step 1, collecting the image information of the solid waste, and obtaining a plurality of sub-image information after preprocessing, wherein each sub-image information is associated with only one solid waste; Step 2, obtaining the spectrum feature data in each sub-image information; Step 3, inputting the spectrum feature data into a pre-trained convolutional neural network module to calculate the first feature vector corresponding to each solid waste in the sub-image information; Step 4: Calculate the node feature matrix based on the trained graph neural network according to the first feature vector corresponding to each solid waste; Step 5: Based on the graph neural network and the node feature matrix, output the second classification result corresponding to each solid waste.

2. A method for collecting and transporting urban solid waste for classified treatment according to claim 1, characterized in that: In step 1, preprocessing includes: normalization processing and target detection, and target detection is used to determine the area of ​​a single solid waste in the image information.

3. The method for collecting and transporting urban solid waste for classified treatment according to claim 1, characterized in that: Step 2 specifically includes step 2.1 to step 2.2; Step 2.1, extract the hue value of each pixel in the sub-image information according to the HSB model and establish a hue matrix; Step 2.2, using two-dimensional discrete Fourier transform to obtain the spectrum feature data in the hue matrix.

4. A method for collecting and transporting urban solid waste for classified treatment according to claim 3, characterized in that: The spectrum feature data F(x,y) is as follows: p uv =(h uv ,s uv ,b uv ) Among them, p uv is the value of the pixel in the sub-image information, h uv ,s uv ,b uv Represent the hue value, saturation value and brightness value respectively, u and v represent the row and column of the pixel points in the sub-picture information respectively, where u=1,2,…,m, v=1,2,…,n; m and n are the width and height of the sub-picture information respectively; H is the hue matrix, i is an imaginary number, x and y are the horizontal and vertical coordinates in the frequency domain respectively.

5. The method for collecting and transporting urban solid waste for classified treatment according to claim 1, characterized in that: The convolutional neural network module includes: a convolutional layer, a feature extraction layer, a pooling layer and a fully connected layer; the convolutional layer is used to convolve the spectral feature data; the feature extraction layer is used to extract the time series feature data from the convolution result; the pooling layer is used to calculate the first feature vector corresponding to each solid waste from the time series feature data.

6. The method for collecting and transporting urban solid waste for classified treatment according to claim 1, characterized in that: Step 4 is specifically shown in the following formula: Among them, P s is the first eigenvector corresponding to the s-th solid waste, where s = 1, 2, …, S, S is the number of all solid wastes; A is the adjacency matrix, D is the degree matrix, ε is the activation function, and Z k is the node feature matrix of the kth layer, k = 0, 1, 2, ..., K, K is the number of graph convolutional layers, W k represents the weight matrix of the kth layer.

7. A method for collecting and transporting urban solid waste for classified treatment according to claim 6, characterized in that: Element a of the adjacency matrix A ij It can be shown as follows: w(i,j)=(1-λ)f1(out i ,Out j )+λf2(Pos i ,Pos j ) Among them, λ is a fixed weight coefficient, R is the preset correlation threshold, w(i,j) represents the correlation between the i-th and j-th solid wastes, Out i ,Out j are the first eigenvectors corresponding to the i-th and j-th solid wastes, respectively. i ,Pos j are the position coordinates of the i-th and j-th solid waste in the sub-image information; f1 is used to measure Out i With Out j The degree of closeness, f2 is the Euler distance function.

8. A municipal solid waste collection and transportation system for classified treatment, used to execute any of the methods of claims 1 to 7, characterized in that: The system comprises: a camera, a communication device, and a server terminal; The camera is used to collect the image information of the solid waste; the communication device is used to upload the image information to the server terminal; The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module and a graph neural network module; The preprocessing module is used to preprocess the image information to obtain a plurality of sub-image information, wherein each sub-image information is associated with only one solid waste; The Fourier transform module is used to obtain the frequency spectrum feature data in each sub-image information; The convolutional neural network module includes a convolution layer, a feature extraction layer, a pooling layer, and a fully connected layer; the convolution layer is used to convolve the spectral feature data; the feature extraction layer is used to extract the time series feature data from the convolution result; the pooling layer is used to calculate the feature vector corresponding to each solid waste from the time series feature data; The graph neural network module includes a graph convolution layer, a graph pooling layer and a graph fully connected layer; the graph convolution layer is used to calculate the node feature matrix according to the first feature vector corresponding to each solid waste; the graph pooling layer is used to reduce the dimension of the node feature matrix, and the graph fully connected layer outputs the second classification result according to the node feature matrix after the dimension reduction process.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images