Kitchen garbage accurate identification and sorting system and method

Abstract

The invention relates to a kitchen garbage accurate identification and sorting system and method, and the system comprises a multispectral imaging module which comprises a visible light camera and a near-infrared camera and is used for obtaining a visible light image and a near-infrared image of garbage; the deep learning processing module is used for processing and analyzing the visible light image and the near-infrared image acquired by the multispectral imaging module by adopting a deep learning algorithm, and outputting the category of garbage; the pressure sensor module is used for monitoring the weight of the garbage in real time; the control and data management module is used for calibrating the recognition result of the deep learning processing module according to the garbage weight data and generating a corresponding mechanical arm sorting instruction according to the recognition result and the calibration result of the deep learning processing module; and the mechanical arm sorting module is used for controlling a mechanical arm to sort different types of garbage based on the mechanical arm sorting instruction. According to the invention, the problem of unbalanced feature information caused by simple splicing in the prior art can be solved, and the precision and efficiency of garbage sorting are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent sanitation technology, and in particular to a system and method for accurately identifying and sorting kitchen waste. Background Art

[0002] With the acceleration of urbanization, the problem of urban domestic waste disposal is becoming increasingly prominent. As a significant component of urban domestic waste, the effective sorting of food waste is crucial for its subsequent processing and recycling. Traditional food waste sorting relies primarily on manual labor, which is not only inefficient but also creates harsh working environments and presents potential safety risks.

[0003] In recent years, with the rapid development of computer vision and machine learning technologies, visual sorting systems have gradually been introduced to the field of food waste sorting. However, existing visual sorting systems have a low recognition rate for wet food waste. Because food waste often contains a large amount of water, its surface reflective properties differ from those of dry waste, making it difficult for existing single-vision recognition methods to accurately determine the type of waste.

[0004] Secondly, the mixed loading inside the garbage transfer truck also increases the difficulty of sorting, and traditional sorting equipment finds it difficult to accurately separate different types of garbage.

[0005] Furthermore, existing technologies have significant flaws in feature splicing and fusion. In multimodal feature processing, traditional feature splicing methods typically simply concatenate feature vectors from different modalities in sequence, such as directly concatenating visible light image feature vectors with near-infrared image feature vectors. However, this simple splicing approach fails to fully consider the correlation and importance differences between the features of the two modalities. This can result in the dilution of certain important features in the fused feature vector, while retaining some irrelevant or noisy features, seriously affecting the accuracy and efficiency of subsequent classification and recognition.

[0006] Therefore, it is necessary to develop a system and method for accurately identifying and sorting kitchen waste. Summary of the Invention

[0007] The purpose of the present invention is to provide a system and method for accurately identifying and sorting kitchen waste, so as to solve the problem of unbalanced feature information caused by simple splicing in the prior art and improve the accuracy and efficiency of waste sorting.

[0008] In a first aspect, the present invention provides a kitchen waste accurate identification and sorting system, comprising:

[0009] A multispectral imaging module, including a visible light camera and a near-infrared camera, is used to obtain visible light and near-infrared images of the garbage;

[0010] The deep learning processing module uses a deep learning algorithm to process and analyze the visible light images and near-infrared images acquired by the multispectral imaging module and outputs the category of garbage;

[0011] The pressure sensor module is installed on the garbage sorting device to monitor the weight of the garbage in real time;

[0012] The control and data management module is used to calibrate the recognition results of the deep learning processing module based on the garbage weight data monitored in real time by the pressure sensor, and to generate corresponding robotic arm sorting instructions based on the recognition and calibration results of the deep learning processing module;

[0013] The robot arm sorting module controls the robot arm to perform sorting operations on different types of garbage based on the robot arm sorting instructions.

[0014] Optionally, the deep learning processing module includes:

[0015] A visible light image feature extraction unit is used to extract features from the visible light image and obtain a visible light image feature vector, including edges, textures, and shapes;

[0016] A near-infrared image feature extraction unit is used to perform similar feature extraction operations on the near-infrared image to obtain the near-infrared image feature vector of the garbage in the near-infrared band, including reflectivity and spectral curve;

[0017] A feature splicing and fusion unit calculates the correlation matrix between the visible light image feature vector and the near-infrared image feature vector, calculates the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector based on the correlation matrix, and performs weighted splicing on the visible light image feature vector and the near-infrared image feature vector based on the calculated weights to form a fused feature vector;

[0018] The garbage classification and recognition unit uses a convolutional neural network as a deep learning model, receives fused feature vectors, calculates and outputs the probability value of each type of garbage through forward propagation, and determines the garbage category.

[0019] Optionally, the control and data management module includes:

[0020] The garbage classification calibration unit is used to integrate the real-time monitored garbage weight data with the recognition results of the deep learning processing module. It calculates the theoretical weight range of each type of garbage based on the average density information of different types of garbage. If the weight of the garbage exceeds the theoretical weight range of the garbage category, the garbage category judgment is calibrated in combination with the recognition results of the deep learning model.

[0021] The robotic arm sorting control and execution unit is used to generate corresponding robotic arm sorting instructions based on the final recognition results of the deep learning processing module and the calibration information after the pressure sensor data is fused.

[0022] Optionally, the robotic arm sorting module includes:

[0023] The high-precision motion control unit controls the robotic arm to follow the preset motion trajectory and action mode to grab different types of garbage and place them into the corresponding collection container;

[0024] The real-time feedback control unit is used to monitor the motion state and grasping force of the robotic arm in real time through position sensors and force sensors, and adjust the position, speed and force of the robotic arm to ensure the accuracy and stability of the sorting operation.

[0025] In a second aspect, a method for accurately identifying and sorting kitchen waste according to the present invention uses the accurate kitchen waste identification and sorting system according to the present invention, and the method comprises the following steps:

[0026] Obtain visible light and near-infrared images of the garbage;

[0027] Use deep learning algorithms to process and analyze the acquired visible light and near-infrared images and output the category of garbage;

[0028] Real-time monitoring of garbage weight;

[0029] The real-time monitored garbage weight data is used to calibrate the recognition results, and the corresponding robotic arm sorting instructions are generated based on the recognition and calibration results;

[0030] Based on the robotic arm sorting instructions, the robotic arm is controlled to perform sorting operations on different types of garbage.

[0031] Optionally, a deep learning algorithm is used to process and analyze the acquired visible light image and near infrared image, and output the category of garbage, specifically including:

[0032] Obtain visible light image feature vectors, including edges, textures, and shapes;

[0033] Perform similar feature extraction operations on the near-infrared image to obtain the near-infrared image feature vector of the garbage in the near-infrared band, including reflectivity and spectral curve;

[0034] Calculate the correlation matrix between the visible light image feature vector and the near-infrared image feature vector, calculate the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector based on the correlation matrix, and perform weighted concatenation of the visible light image feature vector and the near-infrared image feature vector based on the calculated weights to form a fused feature vector;

[0035] A convolutional neural network is used as a deep learning model to receive fused feature vectors, calculate and output the probability value of each type of garbage, and determine the garbage category.

[0036] Optionally, a correlation matrix between the visible light image feature vector and the near infrared image feature vector is calculated, specifically:

[0037]

[0038] Among them, R ij represents the correlation between the i-th element of the visible light feature vector and the j-th element of the near-infrared feature vector, V visible (k,i) represents the i-th element in the visible light feature vector of the k-th sample, V NIR (k,j) represents the jth element in the near-infrared feature vector of the kth sample, represents the mean of the i-th element of the visible light feature vector, represents the mean of the jth element of the near-infrared feature vector, and n represents the total number of samples;

[0039] The weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector are calculated according to the correlation matrix, specifically:

[0040]

[0041] Among them, W visible,i represents the weight of the i-th element of the visible light feature vector, m1 represents the total number of elements in the visible light feature vector, and W NIR,j represents the weight of the jth element of the near-infrared feature vector, and m2 represents the total number of elements in the near-infrared feature vector;

[0042] Perform weighted concatenation on the visible light image feature vector and the near-infrared image feature vector to form a fused feature vector, specifically:

[0043] V fusion =[W visible,1 ·V visible,1 ,W visible,2 ·V visible,2 ,…,W visible,m1 ·V visible,m1 ,W NIR,1 ·V NIR,1 ,W NIR,2

[0044] ·V NIR,2 ,…,W NIR,m2 ·V NIR,m2 ]

[0045] Among them, V fusion Represents the fused feature vector, V visible,i Represents the i-th element of the visible light feature vector, V NIR,j Represents the j-th element of the near-infrared feature vector.

[0046] Optionally, calculate and output the probability value of each type of garbage to determine the garbage category, specifically:

[0047] The fused feature vector V fusion As the input of the deep learning model, the probability vector P = [p1, p2, ..., p d ];

[0048] Among them, p d Indicates the probability value that the garbage belongs to the dth category, and the category with the largest probability value is selected as the final recognition result.

[0049] Optionally, the recognition result is calibrated with the garbage weight data monitored in real time, and corresponding robotic arm sorting instructions are generated according to the recognition result and the calibration result, specifically including:

[0050] The real-time monitored garbage weight data is integrated with the recognition results of the deep learning processing module. The theoretical weight range of each type of garbage is calculated based on the average density information of different types of garbage. If the weight of the garbage exceeds the theoretical weight range of the garbage category, the recognition results of the deep learning model are combined to calibrate the garbage type judgment;

[0051] The corresponding robotic arm sorting instructions are generated based on the final recognition results of the deep learning processing module and the calibration information after the pressure sensor data is fused.

[0052] Optionally, controlling the robotic arm to perform sorting operations on different types of garbage based on the robotic arm sorting instruction specifically includes:

[0053] Based on the sorting instructions of the robotic arm, the robotic arm is controlled to follow the preset motion trajectory and action mode to grab different types of garbage and place them into corresponding collection containers;

[0054] The motion state and gripping force of the robotic arm are monitored in real time through position sensors and force sensors, and the position, speed and force of the robotic arm are adjusted to ensure the accuracy and stability of the sorting operation.

[0055] Beneficial effects of the present invention:

[0056] 1. Improved sorting accuracy: An improved feature splicing and fusion algorithm better integrates the feature information of multispectral images, providing higher-quality feature input for subsequent classification and recognition. Combined with deep learning algorithms and pressure sensor data fusion and calibration, it can more accurately identify different types of food waste, especially significantly improving the recognition rate of wet waste, effectively solving the problems of existing technologies.

[0057] 2. Efficient sorting efficiency: The robotic arm sorting module has high-precision motion control capabilities and fast sorting actions, which can realize rapid sorting of garbage, greatly improving sorting efficiency. Compared with manual sorting, it can greatly save time and labor costs.

[0058] 3. Adaptive optimization capability: The deep learning model can automatically update its weights by continuously receiving new training data and sorting result feedback to adapt to changes in garbage types and characteristics. It has strong adaptability and sustainable optimization capabilities, ensuring that the system maintains good sorting performance during long-term operation.

[0059] 4. Data Management and Analysis: The control and data management module records and stores various data from the sorting process, providing detailed data support for waste treatment and recycling. This data analysis allows for a better understanding of waste distribution and generation patterns, providing a basis for optimizing waste treatment processes and formulating relevant policies.

[0060] To sum up, accurate kitchen waste sorting helps to improve the recycling rate of kitchen waste, reduce the pollution caused by landfill and incineration to the environment, and promote the recycling of resources. It is of great significance to promote the sustainable development of cities and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a principle block diagram of the kitchen waste accurate identification and sorting system described in the embodiment of the present application;

[0062] Figure 2 This is a principle block diagram of the deep learning processing module in the embodiment of the present application;

[0063] Figure 3 This is a principle block diagram of the control and data management module in the embodiment of the present application;

[0064] Figure 4 This is a principle block diagram of the robotic arm sorting module in an embodiment of the present application;

[0065] Figure 5 This is a flow chart of the method for accurately identifying and sorting kitchen waste described in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will be able to understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for the purpose of illustrating the present invention and are not intended to limit the scope of protection of the present invention.

[0067] like Figure 1 As shown, in an embodiment of the present application, a kitchen waste accurate identification and sorting system includes a multispectral imaging module, a deep learning processing module, a pressure sensor module, a control and data management module and a robotic arm sorting module. Among them, the multispectral imaging module includes a visible light camera and a near-infrared camera, which are used to obtain visible light images and near-infrared images of garbage. The deep learning processing module uses a deep learning algorithm to process and analyze the visible light images and near-infrared images obtained by the multispectral imaging module, and outputs the category of the garbage. The pressure sensor module is installed on the garbage sorting device to monitor the weight of the garbage in real time. The control and data management module is used to calibrate the recognition results of the deep learning processing module based on the garbage weight data monitored in real time by the pressure sensor, and to generate corresponding robotic arm sorting instructions according to the recognition results and calibration results of the deep learning processing module. The robotic arm sorting module controls the robotic arm to perform sorting operations on different types of garbage based on the robotic arm sorting instructions.

[0068] like Figure 2 As shown, in one possible embodiment, the deep learning processing module includes a visible light image feature extraction unit, a near-infrared image feature extraction unit, a feature splicing and fusion unit, and a garbage classification and recognition unit. The visible light image feature extraction unit is used to extract features from the visible light image, obtaining a visible light image feature vector, including edges, texture, and shape. The near-infrared image feature extraction unit is used to perform similar feature extraction operations on the near-infrared image, obtaining a near-infrared image feature vector of the garbage in the near-infrared band, including reflectance and spectral curves. The feature splicing and fusion unit is used to calculate a correlation matrix between the visible light image feature vector and the near-infrared image feature vector. Based on the correlation matrix, the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector are calculated. Based on the calculated weights, the visible light image feature vector and the near-infrared image feature vector are weightedly spliced ​​to form a fused feature vector. The garbage classification and recognition unit uses a convolutional neural network as a deep learning model. It receives the fused feature vector and, through forward propagation, calculates and outputs a probability value for each type of garbage to determine the garbage category.

[0069] like Figure 3As shown, in one possible embodiment, the control and data management module includes a garbage classification calibration unit and a robotic arm sorting control and execution unit. The garbage classification calibration unit is used to integrate real-time monitored garbage weight data with the recognition results of the deep learning processing module. Based on the average density information of different garbage types, the theoretical weight range of each type of garbage is calculated. If the weight of the garbage exceeds the theoretical weight range of the garbage type, the garbage type is calibrated based on the recognition results of the deep learning model. The robotic arm sorting control and execution unit is used to generate corresponding robotic arm sorting instructions based on the final recognition results of the deep learning processing module and the calibration information obtained by integrating the pressure sensor data.

[0070] like Figure 4 As shown, in one possible embodiment, the robotic arm sorting module includes a high-precision motion control unit and a real-time feedback control unit. The high-precision motion control unit controls the robotic arm according to preset motion trajectories and action patterns, grabbing different types of garbage and placing them into corresponding collection containers. The real-time feedback control unit monitors the robotic arm's motion state and gripping force in real time using position and force sensors, adjusting the robotic arm's position, speed, and force to ensure the accuracy and stability of the sorting operation.

[0071] like Figure 5 As shown, in an embodiment of the present application, a method for accurately identifying and sorting kitchen waste adopts the accurate identification and sorting system of kitchen waste in the present invention, and the method includes the following steps:

[0072] Obtain visible light and near-infrared images of the garbage.

[0073] A deep learning algorithm is used to process and analyze the acquired visible light images and near-infrared images, and output the category of garbage.

[0074] Monitor the weight of garbage in real time.

[0075] The recognition results are calibrated with the garbage weight data monitored in real time, and corresponding robotic arm sorting instructions are generated based on the recognition results and calibration results.

[0076] Based on the robotic arm sorting instructions, the robotic arm is controlled to perform sorting operations on different types of garbage.

[0077] The following is a detailed description of each step.

[0078] Before executing this method, initialize and calibrate the system:

[0079] Calibrate the multispectral imaging module, including setting parameters for the visible light and near-infrared cameras, such as exposure time and gain, to ensure clear and accurate images. Also, calibrate the pressure sensor module, setting the zero point and range for weight measurement to eliminate errors. Start the deep learning processing module, load the weights of the pre-trained deep learning model, and initialize the deep learning model. Check the operating status of the robotic arm sorting module and perform the home operation to ensure it is in its initial position and ready for sorting.

[0080] The food waste to be sorted is transported to the sorting system's work area via a conveyor. The multispectral imaging module then begins operating, using visible light and near-infrared cameras to simultaneously capture images of the waste, capturing the spectral signature of its reflected light in the 450nm-900nm band. Simultaneously, the pressure sensor module monitors the weight of the waste in real time and transmits this weight data to the control and data management module. This module performs preliminary processing on the acquired image and weight data, such as grayscale conversion and noise removal, to prepare for subsequent deep learning processing.

[0081] Visible light image feature extraction: This method performs feature extraction on visible light images, using the convolutional and pooling layers of a convolutional neural network to extract image features such as edges, textures, and shapes. For example, a 3×3 convolution kernel is used to convolve the image. Assuming the input image size is 256×256 pixels, the convolution kernel size is 3×3, the stride is 1, and the padding is "valid," the output feature map size is 254×254 pixels. Through multiple convolution and pooling operations, deep-level image features are gradually extracted.

[0082] Near-infrared image feature extraction: Similar feature extraction operations are performed on near-infrared images to obtain characteristic information about garbage in the near-infrared band, such as reflectance and spectral curves. Convolutional neural networks are also used for feature extraction. For example, a 5×5 convolution kernel is used to convolve the near-infrared image. Assuming the input near-infrared image size is 256×256 pixels, after convolution and pooling operations, the output feature map size is 127×127 pixels.

[0083] Feature correlation analysis: Calculate the correlation matrix between the visible light image feature vector and the near-infrared image feature vector. The elements of the correlation matrix represent the correlation between the corresponding elements in the two feature vectors. The calculation formula is:

[0084]

[0085] Among them, R ij represents the correlation between the i-th element of the visible light feature vector and the j-th element of the near-infrared feature vector, Vvisible (k,i) represents the i-th element in the visible light feature vector of the k-th sample, V NIR (k,j) represents the jth element in the near-infrared feature vector of the kth sample, represents the mean of the i-th element of the visible light feature vector, represents the mean of the jth element of the near-infrared feature vector, and n represents the total number of samples.

[0086] Feature weight calculation: Calculate the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector based on the correlation matrix. The weight calculation formula is:

[0087]

[0088] Among them, W visible,i represents the weight of the i-th element of the visible light feature vector, m1 represents the total number of elements in the visible light feature vector, and W NIR,j represents the weight of the jth element of the near-infrared feature vector, and m2 represents the total number of elements in the near-infrared feature vector.

[0089] Taking the actual kitchen waste image as an example, assuming there are 10 multispectral images of kitchen waste, the visible light image feature vector V is obtained after feature extraction of each image. visible (dimension is 10×256) and near infrared image feature vector V NIR (Dimension is 10×128). Calculate the correlation matrix, whose dimension is 256×128. Assume that during the calculation process, for the first element of the visible light feature vector and the first element of the near infrared feature vector, the calculated correlation R 11 =0.8. This indicates that there is a high positive correlation between the two feature elements. The weight vector W calculated based on the correlation matrix visible and W NIR For example, as an example, W visible =0.05, W NIR =0.06, the weight value reflects the importance of the corresponding feature element in the fusion process.

[0090] Weighted feature concatenation: Based on the calculated weights, the visible light image feature vector and the near-infrared image feature vector are weighted concatenated to form a fused feature vector. The calculation formula for the fused feature vector is:

[0091] V fusion =[W visible,1 ·V visible,1 ,W visible,2 ·V visible,2 ,…,W visible,m1 ·Vvisible,m1 ,W NIR,1 ·V NIR,1 ,W NIR,2 ·V NIR,2 ,…,W NIR,m2 ·V NIR,m2 ]

[0092] Among them, V fusion Represents the fused feature vector, V visible,i Represents the i-th element of the visible light feature vector, V NIR,j In this way, weights are assigned to features of different modalities based on their correlation, so that the fused feature vector highlights important features and reduces the influence of irrelevant features, thereby improving the accuracy and efficiency of subsequent classification and recognition.

[0093] Continuing with the above embodiment, for the visible light feature vector V of the first sample visible,1 =[0.1,0.3,0.5,…,0.8] and near-infrared feature vector V NIR,1 =[0.2,0.4,0.6,…,0.9], using the calculated weight vector W visible,1 =[0.05,0.04,…] and W NIR,1 = [0.06, 0.05, ...], and perform weighted concatenation. For example, the fused feature vector V fusion,1 The first few elements of are: 0.05 × 0.1 = 0.005 (the first element from the visible light feature), 0.04 × 0.3 = 0.012 (the second element from the visible light feature), and so on; followed by 0.06 × 0.2 = 0.012 (the first element from the near-infrared feature), 0.05 × 0.4 = 0.02 (the second element from the near-infrared feature), and so on. The resulting fused feature vector will serve as the input to the subsequent deep learning model for garbage classification and identification.

[0094] Deep learning model structure: A convolutional neural network is used as the deep learning model, and its structure includes an input layer, multiple convolutional layers, a pooling layer, a fully connected layer, and an output layer. As an embodiment of this solution, a CNN model consisting of 3 convolutional layers and 2 fully connected layers can be constructed. The input layer receives the fused feature vector. The first convolutional layer has 32 filters, each with a filter size of 3×3, a step size of 1, and a padding mode of "same"; the second convolutional layer has 64 filters, a filter size of 3×3, a step size of 1, and a padding mode of "same"; the third convolutional layer has 128 filters, a filter size of 3×3, a step size of 1, and a padding mode of "same". Each convolutional layer is followed by a maximum pooling layer with a pooling window size of 2×2 and a step size of 2. After convolution and pooling operations, the feature map is flattened into a one-dimensional vector and input into the first fully connected layer, which has 256 neurons and uses the ReLU activation function. The second fully connected layer has 10 neurons (assuming there are 10 types of garbage) and uses the Softmax activation function to output the probability value of each type of garbage.

[0095] Model Training and Optimization: The deep learning model was trained using the Adam optimization algorithm, with a cross-entropy loss function. For example, the training dataset consisted of 10,000 multispectral images of different types of garbage and their corresponding labels. The dataset was divided into a training set (8,000 images) and a validation set (2,000 images). The learning rate was set to 0.001, the number of training epochs to 50, and the batch size to 32. During training, the backpropagation algorithm continuously updated the weight parameters of the deep learning model, gradually reducing the loss function and improving the classification accuracy of the deep learning model.

[0096] Classification and recognition process: The fused feature vector is input into the trained deep learning model, and the deep learning model outputs the probability value of each type of garbage through forward propagation calculation.

[0097] For example, for the above fusion feature vector V fusion , after calculation by the deep learning model, the output probability vector P = [p1, p2, ..., p d ], where p d Indicates the probability that the garbage belongs to the dth category. The category with the largest probability value is selected as the final recognition result. Assuming the output probability vector is: P = [0.1, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.5], the recognition result is category 10 garbage. Table example:

[0098]

[0099] Pressure sensor data fusion and calibration: The control and data management module fuses the garbage weight data monitored in real time by the pressure sensor with the recognition results of the deep learning processing module. According to the average density information of different types of garbage (pre-established database), the theoretical weight range of each type of garbage is calculated. Compare the actual measured garbage weight with the theoretical weight range. If it exceeds a certain error range, the type of garbage is calibrated in combination with the recognition results of the deep learning model. For example, if the recognition result is a certain type of garbage, but the actual weight is obviously heavier or lighter, it may mean that the garbage is mixed with other types of garbage. At this time, the recognition result can be corrected, or the garbage can be marked as a mixed garbage category for subsequent processing. Through this data fusion and calibration method, the accuracy of garbage sorting can be further improved and the occurrence of misjudgment can be reduced.

[0100] Robotic arm sorting control and execution: Based on the final recognition results of the deep learning processing module and the calibration information after the fusion of pressure sensor data, the control and data management module generates the corresponding robotic arm sorting instructions. After receiving the instructions, the robotic arm sorting module controls the robotic arm to accurately grasp different types of garbage according to the preset motion trajectory and action mode, and place them into the corresponding collection container. During the robotic arm sorting process, the position, speed, and force parameters of the robotic arm are adjusted through a real-time feedback control mechanism to ensure the accuracy and stability of the sorting operation. For example, the position sensor and force sensor on the robotic arm are used to monitor the motion state and grasping force of the robotic arm in real time. When deviations or abnormal conditions occur, corrections are made in time to avoid garbage falling or sorting errors.

[0101] This system integrates a multispectral imaging module (visible light + near-infrared) and a deep learning processing module. By capturing the reflective spectral characteristics of kitchen waste in the 450nm-900nm band, it establishes a material database and realizes accurate identification of different types of garbage.

[0102] This system uses an improved feature splicing and fusion algorithm to calculate the correlation matrix between the feature vectors of visible light images and near-infrared images, and assigns weights to features of different modalities based on the correlation, so that the fused feature vectors can highlight important features and reduce the influence of irrelevant features, thereby improving the accuracy and efficiency of subsequent classification and recognition.

[0103] This system combines pressure sensors to calibrate garbage weight deviations in real time, fuses pressure sensor data with deep learning recognition results, improves the accuracy and reliability of garbage sorting, and effectively solves the problem of low recognition rate of wet garbage in existing visual sorting systems.

[0104] The robotic arm sorting module in this system performs high-precision garbage sorting operations based on the recognition results of the deep learning processing module, and ensures the accuracy and stability of the sorting process through a real-time feedback control mechanism.

[0105] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A kitchen waste precise identification and sorting system, characterized by: A multispectral imaging module, including a visible light camera and a near-infrared camera, is used to obtain visible light and near-infrared images of the garbage; The deep learning processing module uses a deep learning algorithm to process and analyze the visible light images and near-infrared images acquired by the multispectral imaging module and outputs the category of garbage; The pressure sensor module is installed on the garbage sorting device to monitor the weight of the garbage in real time; The control and data management module calibrates the recognition results of the deep learning processing module based on the garbage weight data monitored in real time by the pressure sensor, and generates corresponding robotic arm sorting instructions based on the recognition and calibration results of the deep learning processing module; The robot arm sorting module controls the robot arm to perform sorting operations on different types of garbage based on the robot arm sorting instructions.

2. The kitchen waste accurate identification and sorting system according to claim 1 is characterized in that: The deep learning processing module includes: A visible light image feature extraction unit is used to extract features from the visible light image and obtain a visible light image feature vector, including edges, textures, and shapes; A near-infrared image feature extraction unit is used to perform similar feature extraction operations on the near-infrared image to obtain the near-infrared image feature vector of the garbage in the near-infrared band, including reflectivity and spectral curve; A feature splicing and fusion unit calculates the correlation matrix between the visible light image feature vector and the near-infrared image feature vector, calculates the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector based on the correlation matrix, and performs weighted splicing on the visible light image feature vector and the near-infrared image feature vector based on the calculated weights to form a fused feature vector; The garbage classification and recognition unit uses a convolutional neural network as a deep learning model, receives fused feature vectors, calculates and outputs the probability value of each type of garbage through forward propagation, and determines the garbage category.

3. The method for accurately identifying and sorting kitchen waste according to claim 2, characterized in that: The control and data management module includes: The garbage classification calibration unit is used to integrate the real-time monitored garbage weight data with the recognition results of the deep learning processing module. It calculates the theoretical weight range of each type of garbage based on the average density information of different types of garbage. If the weight of the garbage exceeds the theoretical weight range of the garbage category, the garbage category judgment is calibrated in combination with the recognition results of the deep learning model. The robotic arm sorting control and execution unit is used to generate corresponding robotic arm sorting instructions based on the final recognition results of the deep learning processing module and the calibration information after the pressure sensor data is fused.

4. The kitchen waste accurate identification and sorting system according to claim 1, characterized in that: The robotic arm sorting module includes: A high-precision motion control unit is used to control the robotic arm to grab different types of garbage and place them into corresponding collection containers according to preset motion trajectories and action patterns; The real-time feedback control unit is used to monitor the motion state and grasping force of the robotic arm in real time through position sensors and force sensors, and adjust the position, speed and force of the robotic arm to ensure the accuracy and stability of the sorting operation.

5. A method for accurately identifying and sorting kitchen waste, characterized in that: The kitchen waste precise identification and sorting system according to any one of claims 1 to 4 is used, and the method comprises the following steps: Obtain visible light and near-infrared images of the garbage; Use deep learning algorithms to process and analyze the acquired visible light and near-infrared images and output the category of garbage; Real-time monitoring of garbage weight; The real-time monitored garbage weight data is used to calibrate the recognition results, and the corresponding robotic arm sorting instructions are generated based on the recognition and calibration results; Based on the robotic arm sorting instructions, the robotic arm is controlled to perform sorting operations on different types of garbage.

6. The method for accurately identifying and sorting kitchen waste according to claim 5, characterized in that: A deep learning algorithm is used to process and analyze the acquired visible light and near-infrared images, and output the categories of garbage, including: Obtain visible light image feature vectors, including edges, textures, and shapes; Perform similar feature extraction operations on the near-infrared image to obtain the near-infrared image feature vector of the garbage in the near-infrared band, including reflectivity and spectral curve; Calculate the correlation matrix between the visible light image feature vector and the near-infrared image feature vector, calculate the weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector based on the correlation matrix, and perform weighted concatenation of the visible light image feature vector and the near-infrared image feature vector based on the calculated weights to form a fused feature vector; A convolutional neural network is used as a deep learning model to receive fused feature vectors, calculate and output the probability value of each type of garbage, and determine the garbage category.

7. The method for accurately identifying and sorting kitchen waste according to claim 6, characterized in that: Calculate the correlation matrix between the visible light image feature vector and the near infrared image feature vector, specifically: Among them, R ij represents the correlation between the i-th element of the visible light feature vector and the j-th element of the near-infrared feature vector, V visible (k,i) represents the i-th element in the visible light feature vector of the k-th sample, V NIR (k,j) represents the jth element in the near-infrared feature vector of the kth sample, represents the mean of the i-th element of the visible light feature vector, represents the mean of the jth element of the near-infrared feature vector, and n represents the total number of samples; The weight of each element in the visible light feature vector and the weight of each element in the near-infrared image feature vector are calculated according to the correlation matrix, specifically: Among them, W visible,i represents the weight of the i-th element of the visible light feature vector, m1 represents the total number of elements in the visible light feature vector, and W NIR,j represents the weight of the jth element of the near-infrared feature vector, and m2 represents the total number of elements in the near-infrared feature vector; Perform weighted concatenation on the visible light image feature vector and the near-infrared image feature vector to form a fused feature vector, specifically: V fusion =[W visible,1 ·V visible,1 ,W visible,2 ·V visible,2 ,…,W visible,m1 ·V visible,m1 ,W NIR,1 ·V NIR,1 ,W NIR,2 ·V NIR,2 ,…,W NIR,m2 ·V NIR,m2 ] Among them, V fusion Represents the fused feature vector, V visible,i Represents the i-th element of the visible light feature vector, V NIR,j Represents the j-th element of the near-infrared feature vector.

8. The method for accurately identifying and sorting kitchen waste according to claim 7, characterized in that: Calculate and output the probability value of each type of garbage to determine the garbage category, specifically: The fused feature vector V fusion As the input of the deep learning model, the probability vector P = [p1, p2, ..., p d ]; Among them, p d Indicates the probability value that the garbage belongs to the dth category, and the category with the largest probability value is selected as the final recognition result.

9. The method for accurately identifying and sorting kitchen waste according to claim 6, characterized in that: The real-time monitored garbage weight data is used to calibrate the recognition results, and the corresponding robotic arm sorting instructions are generated based on the recognition and calibration results, specifically including: The real-time monitored garbage weight data is integrated with the recognition results of the deep learning processing module. The theoretical weight range of each type of garbage is calculated based on the average density information of different types of garbage. If the weight of the garbage exceeds the theoretical weight range of the garbage category, the recognition results of the deep learning model are combined to calibrate the garbage type judgment; The corresponding robotic arm sorting instructions are generated based on the final recognition results of the deep learning processing module and the calibration information after the pressure sensor data is fused.

10. The method for accurately identifying and sorting kitchen waste according to claim 6, characterized in that: Controlling the robotic arm to perform sorting operations on different types of garbage based on the robotic arm sorting instructions specifically includes: Based on the sorting instructions of the robotic arm, the robotic arm is controlled to follow the preset motion trajectory and action mode to grab different types of garbage and place them into corresponding collection containers; The motion state and gripping force of the robotic arm are monitored in real time through position sensors and force sensors, and the position, speed and force of the robotic arm are adjusted to ensure the accuracy and stability of the sorting operation.

Images