A municipal solid waste collection and transportation method oriented to classification processing

Abstract

A kind of urban solid waste collection and transportation method for classification processing, comprising: collecting picture information of solid waste, and obtaining a plurality of sub-picture information after preprocessing;Obtain the spectral feature data in each sub-picture information;Spectral feature data is input into the convolutional neural network module trained in advance, and the first feature vector corresponding to each solid waste in sub-picture information is calculated;According to the first feature vector corresponding to each solid waste, based on the trained graph neural network, the node feature matrix is calculated;According to node feature matrix, the second classification result corresponding to each solid waste is output.The present application creatively introduces graph neural network on the basis of convolutional neural network, and the feature information of all solid wastes in the same batch is associated, the feature defects of single solid waste are strengthened, and then the category of each solid waste can be more accurately judged.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and more specifically, relates to a method for the collection and transportation of urban solid waste for classified processing. Background Technology

[0002] The sorting and treatment of urban solid waste is an important aspect of environmental protection and resource recycling. Effective sorting can minimize environmental pollution and promote the sustainable use of resources. When processing urban solid waste, using convolutional neural networks to classify and recognize images containing solid waste can help achieve waste sorting. However, in reality, due to the inherent characteristics of solid waste, it is difficult to correctly classify images.

[0003] Solid waste is characterized by its fragmented nature, meaning it is often damaged before it becomes garbage. For example, plastic boxes are often torn, aluminum cans are crushed, or bicycle parts are missing. These factors significantly affect the sample information, i.e., the image information, thus hindering or reducing the accuracy of convolutional neural network training, and consequently causing the convolutional neural network module to fail to correctly identify the type of solid waste.

[0004] Existing technology CN112620165A discloses an automatic garbage image generation method for garbage classification, which includes: manually collecting several garbage images, converting the training sample data of the original training sample set into TFRecord format; constructing a generator network using a deep convolutional neural network; constructing a discriminator network using a sparse autoencoder deep convolutional neural network; training the generator network and the discriminator network using an alternating iterative optimization method until the model converges; and establishing a garbage image generator to generate garbage images. However, the main feature of this method is cost-saving, and it does not take into account the fragmented characteristics of urban solid waste. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to overcome the aforementioned deficiencies and propose a method for the collection and transportation of urban solid waste for classified processing.

[0006] The present invention adopts the following technical solution.

[0007] The first aspect of this invention discloses a method for collecting and transporting urban solid waste for classified processing, characterized by comprising steps 1 to 5;

[0008] Step 1: Collect image information of solid waste, and after preprocessing, obtain multiple sub-image information. Each sub-image information is associated with one and only one solid waste.

[0009] Step 2: Obtain the spectral feature data from each sub-image information;

[0010] Step 3: Input the spectral feature data into the pre-trained convolutional neural network module to calculate the first feature vector corresponding to each solid waste in the sub-image information;

[0011] Step 4: Based on the first feature vector corresponding to each solid waste, calculate the node feature matrix using the trained graph neural network.

[0012] Step 5: Based on the graph neural network, output the second classification result corresponding to each solid waste according to the node feature matrix.

[0013] The second aspect of this invention discloses a system for the collection and transportation of urban solid waste for classified processing, used to perform the method described in the first aspect, the system comprising: a camera, a communication device, and a server terminal;

[0014] The camera is used to collect images of solid waste; the communication device is used to upload the images to the server terminal.

[0015] The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module, and a graph neural network module;

[0016] The preprocessing module is used to preprocess the image information to obtain multiple sub-image information, where each sub-image information is associated with one and only one solid waste.

[0017] The Fourier transform module is used to obtain the spectral feature data in each sub-image information;

[0018] The convolutional neural network module includes convolutional layers, feature extraction layers, pooling layers, and fully connected layers. The convolutional layers are used to convolve the spectral feature data; the feature extraction layers are used to extract temporal feature data from the convolutional results; and the pooling layers are used to calculate the feature vector corresponding to each solid waste from the temporal feature data.

[0019] The graph neural network module includes graph convolutional layers, graph pooling layers, and graph fully connected layers. The graph convolutional layers are used to calculate the node feature matrix based on the first feature vector corresponding to each solid waste. The graph pooling layers are used to reduce the dimensionality of the node feature matrix. The graph fully connected layers output the second classification result based on the dimensionality-reduced node feature matrix.

[0020] A third aspect of the present invention discloses a terminal, including a processor and a storage medium; characterized in that:

[0021] The storage medium is used to store instructions;

[0022] The processor is configured to operate according to the instructions to execute the method described in the first aspect.

[0023] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method described in the first aspect.

[0024] The beneficial effects of the present invention are as follows: Compared with the prior art, the present invention has the following advantages:

[0025] This invention creatively introduces graph neural networks on the basis of convolutional neural networks. By associating the feature information of all solid wastes in the same batch, it can make up for the incomplete characteristics of solid wastes and thus more accurately determine the category of each solid waste.

[0026] This invention extracts and analyzes the tone matrix to uniformly filter image information under different lighting conditions at different times. Attached Figure Description

[0027] Figure 1 This is a flowchart of a method for the collection and transportation of urban solid waste for classified processing. Detailed Implementation

[0028] The present application will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be construed as limiting the scope of protection of the present application.

[0029] This invention discloses a method for the collection and transportation of urban solid waste for classified processing, such as... Figure 1 As shown, it includes steps 1 through 5.

[0030] Step 1: Collect image information of solid waste, and after preprocessing, obtain multiple sub-image information. Each sub-image information is associated with one and only one solid waste.

[0031] In some embodiments, preprocessing includes at least: normalization and object detection. Object detection involves determining the region of a single solid waste object within the image information, which can be performed using the YOLO algorithm. The YOLO algorithm works by dividing the image information into a grid, predicting the bounding box and class probability of the target based on each grid, thereby extracting each target, i.e., the solid waste object, from the image information.

[0032] Step 2: Obtain the spectral feature data from each sub-image.

[0033] Each pixel in the sub-image information typically includes three dimensions of information, such as RGB, YUV, or HSB. RGB refers to red, green, and blue; YUV refers to luminance, primary chromaticity, and secondary chromaticity; and HSB refers to hue, saturation, and luminance. Considering that images of solid waste can be taken at different times of day, to ignore the changes in sub-image information caused by different lighting conditions and varying brightness levels, in some embodiments, step 2 specifically includes steps 2.1 to 2.2.

[0034] Step 2.1: Extract the hue value of each pixel in the sub-image information according to the HSB model and establish a hue matrix.

[0035] Step 2.2: Use two-dimensional discrete Fourier transform to obtain the spectral feature data in the hue matrix.

[0036] Specifically, the spectral feature data F(x,y) is shown in the following formula:

[0037] p uv =(h uv ,s uv ,b uv )

[0038]

[0039] Where, p uv The value of a pixel in the sub-image information includes information in three dimensions, h. uv ,s uv ,b uv represents the hue value, saturation value, and brightness value, respectively. u and v represent the row and column of the pixel in the sub-image information, respectively. u = 1, 2, ..., m, v = 1, 2, ..., n; m and n are the width and height of the sub-image information, respectively; H is the hue matrix, i is an imaginary number, and x and y are the horizontal and vertical coordinates in the frequency domain, respectively.

[0040] Understandably, since hue is not affected by lighting conditions, the hue value remains relatively stable even when lighting conditions change when an object is photographed at different times. This allows the HSV model to ignore errors caused by differences in time when processing color.

[0041] Step 3: Input the spectral feature data into a pre-trained convolutional neural network (CNN) to calculate the first feature vector corresponding to each solid waste in the sub-image information.

[0042] Specifically, a convolutional neural network includes: convolutional layers, feature extraction layers, pooling layers, and fully connected layers. The convolutional layers are used to convolve the spectral feature data; the feature extraction layers are used to extract temporal feature data from the convolutional results; the pooling layers are used to calculate the first feature vector corresponding to each solid waste from the temporal feature data; and the fully connected layers output the first classification result of the solid waste based on the first feature vector.

[0043] It should be noted that, in the embodiments of the present invention, the fully connected layers of the convolutional neural network are replaced by graph neural networks.

[0044] Step 4: Based on the first feature vector corresponding to each solid waste, calculate the node feature matrix using the trained Graph Neural Network (GNN).

[0045] A graph neural network is a K-layer model, where each layer is a graph convolutional layer. K is the number of graph convolutional layers, a fixed value. A larger K value results in better training performance but also longer iteration time. In the feature map, each node corresponds to a solid waste object. The weights between nodes represent their first feature vectors and their positional relationship within the sub-image information.

[0046] Step 4 is shown in the following formula:

[0047]

[0048] Among them, P s Let Z be the first feature vector corresponding to the s-th solid waste, where s = 1, 2, ..., S, and S is the total number of solid wastes; A is the adjacency matrix, where each element has a value of 1 or 0, indicating whether two nodes are adjacent; D is the degree matrix, which is a diagonal matrix. Each diagonal element represents the number of associated neighboring nodes, and in each row of the degree matrix, only the elements on the main diagonal have a non-zero value, and the value of that element is the sum of the values ​​of all elements in the same row of A; ε is the activation function, which can be either the sigmoid function or the ReLU function; Z k Let W be the node feature matrix of the k-th layer, where k = 0, 1, 2, ..., K. k This represents the weight matrix of the k-th layer, obtained through training the graph neural network. Understandably, as model parameters of the graph neural network, it can be continuously optimized using algorithms such as backpropagation.

[0049] It's important to consider the significant differences between experimental and actual samples of urban solid waste. Urban solid waste can include electronic waste, metal waste, plastic waste, and more. Most waste is often damaged when it becomes garbage. Besides exposure to intense sunlight and rain, plastic boxes are often torn, aluminum cans are crushed, or bicycle parts are missing. These factors can affect the accuracy of convolutional neural networks, causing them to fail to correctly identify the type of solid waste. However, from a common-sense perspective, it should be assumed that all solid waste within the same sub-image or batch of sub-images has a certain correlation. For example, household waste is often dominated by plastic waste and kitchen waste, while industrial waste is mainly dominated by metal waste and electronic waste. Furthermore, solid waste items that are close together in an image, assuming their corresponding first feature vectors are similar, can be considered the same type of solid waste, and their first feature vectors can be aggregated. Based on this idea, the present invention creatively introduces a graph neural network, treating each solid waste as a node in the feature graph, thereby utilizing this correlation to strengthen the feature information corresponding to each solid waste (e.g., the first feature vector in step 3), and thus being able to identify the type of each solid waste.

[0050] It should be noted that "the same batch" in the above text refers to all sub-image information acquired at the same location and at the same time.

[0051] The significance of graph neural networks lies in their ability to establish connections between different nodes, and they are commonly used in fields such as social network analysis, recommendation systems, and knowledge graphs. Building upon this, this invention creatively utilizes the ability of graph neural networks to aggregate information from nodes (equivalent to local information) and their surrounding nodes, thereby capturing more global and richer feature information.

[0052] Based on this, the adjacency matrix in the graph neural network is calculated based on the first feature vector corresponding to each solid waste in the sub-image information.

[0053] In some embodiments, the element a of the adjacency matrix A ij It can be represented as follows:

[0054]

[0055] w(i,j)=(1-λ)f1(P i ,P j Out i Out j )+λf2(Pos i Pos j )

[0056] Where λ is the weighting coefficient, which can be set slightly greater than 0, and in the embodiments of this invention, it can be 0.1. R is a preset correlation threshold, w(i,j) represents the correlation between the i-th and j-th solid wastes, and P i ,P j Let Out be the first feature vector corresponding to the i-th and j-th solid wastes; i Out j Pos represents the first classification result corresponding to the i-th and j-th solid wastes, respectively. i Pos j These are the position coordinates of the i-th and j-th solid waste items in the sub-image information, respectively; f1 is used to measure Out. i With Out j The degree of closeness, f2 is the normalized Euler distance function.

[0057] Understandably, if Out i With Out j If the first eigenvectors are completely different, then the value of f1 must be 0; and as Out... i With Out j The higher the similarity of the elements contained within, the larger the value of f1; until Out. i With Out j If all elements contained in f1 are exactly the same, then the value of f1 is 1. Normalization of f2 means that the value of f2 is also limited to the range [0, 1].

[0058] In some embodiments, in f1, Out i With Out j It does not participate in the calculation; it is only used as a verification method (i.e., verifying the value of f1 in the manner described above). This is because Out i or Out j Issues such as inconsistent vector lengths exist. Understandably, in this embodiment, the calculation process of f1 is essentially the same as that of a fully connected layer in a convolutional neural network.

[0059] Step 5: Based on the graph neural network, output the second classification result corresponding to each solid waste according to the node feature matrix.

[0060] Understandably, in step 4, the node feature matrix is ​​calculated through the graph convolutional layer in the graph neural network, which also includes graph pooling layers and graph fully connected layers. The graph pooling layer is used to merge features, that is, to reduce the dimensionality of the node feature matrix. The graph fully connected layer outputs the final prediction result, i.e., the second classification result, based on the dimensionality-reduced node feature matrix.

[0061] Accordingly, the present invention also discloses an urban solid waste collection and transportation system for classified processing, including: a camera, a communication device, and a server terminal.

[0062] The camera is used to collect images of solid waste; the communication device is used to upload the images to the server terminal.

[0063] The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module, and a graph neural network module.

[0064] The preprocessing module is used to preprocess the image information to obtain multiple sub-image information, where each sub-image information is associated with one and only one solid waste.

[0065] The Fourier transform module is used to obtain the spectral feature data in each sub-image information.

[0066] The convolutional neural network module includes convolutional layers, feature extraction layers, pooling layers, and fully connected layers. The convolutional layers are used to convolve the spectral feature data; the feature extraction layers are used to extract temporal feature data from the convolutional results; and the pooling layers are used to calculate the feature vector corresponding to each solid waste from the temporal feature data.

[0067] The graph neural network module includes graph convolutional layers, graph pooling layers, and graph fully connected layers. The graph convolutional layers are used to calculate the node feature matrix based on the first feature vector corresponding to each solid waste. The graph pooling layers are used to reduce the dimensionality of the node feature matrix. The graph fully connected layers output the second classification result based on the dimensionality-reduced node feature matrix.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for collecting and transporting urban solid waste for classified processing, characterized in that, Includes steps 1 through 5; Step 1: Collect image information of solid waste, and after preprocessing, obtain multiple sub-image information. Each sub-image information is associated with one and only one solid waste. Step 2: Obtain the spectral feature data from each sub-image information; Step 3: Input the spectral feature data into a pre-trained convolutional neural network module to calculate the first feature vector corresponding to each solid waste item in the sub-image information. The convolutional neural network module includes: a convolutional layer, a feature extraction layer, a pooling layer, and a fully connected layer. The convolutional layer performs convolution on the spectral feature data; the feature extraction layer extracts feature data from the convolutional result; the pooling layer calculates the first feature vector corresponding to each solid waste item from the feature data; and the fully connected layer outputs the first classification result of the solid waste item based on the first feature vector. Step 4: Based on the first feature vector corresponding to each solid waste, calculate the node feature matrix based on the trained graph neural network, where each node corresponds to a fixed waste. Step 5: Based on the graph neural network, output the second classification result corresponding to each solid waste according to the node feature matrix; Step 4 is shown in the following formula: Among them, P s Let Z be the first feature vector corresponding to the s-th solid waste, where s = 1, 2, ..., S, and S is the number of all solid wastes; A is the adjacency matrix, D is the degree matrix, ε is the activation function, and Z is the first feature vector. k Let W be the node feature matrix of the k-th layer, where k = 0, 1, 2, ..., K, and K is the number of graph convolutional layers. k This represents the weight matrix of the k-th layer; The element a of adjacency matrix A ij It can be represented as follows: w(i,j)=(1-λ)f1(P i ,P j ,Out i ,Out j )+λf2(Pos i ,Pos j ) Where λ is a fixed weighting coefficient, R is a preset correlation threshold, w(i,j) represents the correlation between the i-th and j-th solid wastes, and P i ,P j Let Out be the first feature vector corresponding to the i-th and j-th solid wastes. i Out j These are the first classification results for the i-th and j-th solid wastes, respectively; f1 is used to measure Out i With Out j The degree of closeness, f2 is the Euler distance function.

2. The method for collecting and transporting urban solid waste for classified processing according to claim 1, characterized in that, In step 1, preprocessing includes normalization and object detection, where object detection is used to determine the region of a single solid waste item in the image information.

3. The method for collecting and transporting urban solid waste for classified processing according to claim 1, characterized in that, Step 2 specifically includes steps 2.1 to 2.2; Step 2.1: Extract the hue value of each pixel in the sub-image information according to the HSB model and establish a hue matrix; Step 2.2: Use two-dimensional discrete Fourier transform to obtain the spectral feature data in the hue matrix.

4. A method for collecting and transporting urban solid waste for classified processing according to claim 3, characterized in that, The spectral feature data F(x,y) is shown in the following formula: p uv =(h uv ,s uv ,b uv ) Where, p uv h represents the pixel value in the sub-image information. uv ,s uv ,b uv represents the hue value, saturation value, and brightness value, respectively. u and v represent the row and column of the pixel in the sub-image information, respectively. u = 1, 2, ..., m, v = 1, 2, ..., n; m and n are the width and height of the sub-image information, respectively; H is the hue matrix, i is an imaginary number, and x and y are the horizontal and vertical coordinates in the frequency domain, respectively.

5. A system for collecting and transporting urban solid waste for classified processing, used to perform the method according to any one of claims 1 to 4, characterized in that, The system includes: a camera, a communication device, and a server terminal; The camera is used to collect images of solid waste; the communication device is used to upload the images to the server terminal. The server terminal includes: a preprocessing module, a Fourier transform module, a convolutional neural network module, and a graph neural network module; The preprocessing module is used to preprocess the image information to obtain multiple sub-image information, where each sub-image information is associated with one and only one solid waste. The Fourier transform module is used to obtain the spectral feature data in each sub-image information; The convolutional neural network module includes convolutional layers, feature extraction layers, pooling layers, and fully connected layers. The convolutional layers are used to convolve the spectral feature data; the feature extraction layers are used to extract feature data from the convolutional results; and the pooling layers are used to calculate the feature vector corresponding to each solid waste from the feature data. The graph neural network module includes graph convolutional layers, graph pooling layers, and graph fully connected layers. The graph convolutional layers are used to calculate the node feature matrix based on the first feature vector corresponding to each solid waste. The graph pooling layers are used to reduce the dimensionality of the node feature matrix. The graph fully connected layers output the second classification result based on the dimensionality-reduced node feature matrix.

6. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-4.

Images