Intelligent garbage classification robot system and method based on multi-modal perception

Abstract

The invention provides an intelligent garbage classification robot system and method based on multi-modal sensing, and the system comprises a multi-modal sensing subsystem which is used for detecting the garbage form in real time, judging the material rigidity in real time, and carrying out the component recognition; the sorting execution module is used for sorting the garbage in real time according to the recognition result of the multi-mode sensing subsystem; and the rebate control module is used for calculating a rebate amount and carrying out fee payment. According to the technical scheme, the sorting efficiency and sorting quality of garbage classification are improved.

Description

Technical Field

[0001] The present application relates to the field of solid waste treatment technology, and in particular to an intelligent garbage sorting robot system and method based on multimodal perception. Background Art

[0002] Residents' participation rate in garbage sorting is low, and the cost of manual sorting is high; traditional sorting equipment relies on single visual recognition, and the error rate in identifying mixed garbage and irregular objects is too high; the rebate process for recyclables is cumbersome, making it difficult to motivate users to continue participating.

[0003] Existing technologies only use weight sensors to determine the type of garbage, but cannot distinguish between materials. This may require pre-set garbage delivery ports, such as dedicated ports for metal, plastic, and paper, and does not implement the "no need to classify" feature. Summary of the Invention

[0004] The present application provides an intelligent garbage sorting robot system and method based on multimodal perception to improve the sorting efficiency and sorting quality of garbage sorting.

[0005] In a first aspect, an intelligent garbage sorting robot system based on multimodal perception is provided, comprising:

[0006] Multimodal sensing subsystem for real-time detection of garbage morphology, determination of material stiffness, and component identification;

[0007] A sorting execution module, configured to sort garbage in real time according to the recognition result of the multimodal sensing subsystem;

[0008] The rebate control module is used to calculate the rebate amount and make fee payments.

[0009] In the above technical solution, a multimodal sensing subsystem is set up to detect the shape of garbage in real time, determine the material stiffness in real time and identify the components; a sorting execution module is used to sort the garbage in real time according to the recognition results of the multimodal sensing subsystem; and a rebate control module is used to calculate the rebate amount and make fee payments; thereby improving the sorting efficiency and sorting quality of garbage classification.

[0010] In a specific embodiment, the multimodal perception subsystem includes:

[0011] Vision module for real-time detection of garbage forms;

[0012] The tactile module is used to judge the material stiffness through grasping feedback;

[0013] The component analysis module is used to identify the type of plastic and metal components using near-infrared spectroscopy.

[0014] In a specific implementation scheme, the visual module uses the YOLOv7 algorithm to detect garbage forms in real time.

[0015] In a specific implementation scheme, the sorting execution module includes:

[0016] Sorting compartment, used to direct garbage to different cavities based on identification results;

[0017] Collaborative robotic arm for grasping objects.

[0018] In a specific implementation scheme, the rebate control module includes:

[0019] Dynamic pricing unit, used to calculate rebate amounts by weight or quantity based on a cloud-based price database;

[0020] The contactless payment interface is used to support QR code / NFC binding of resident accounts and automatic settlement to the user terminal.

[0021] In a specific embodiment, it further includes: a data management module, wherein:

[0022] The data management module is used for garbage calorific value prediction and self-optimization of sorting strategies.

[0023] In a specific implementation scheme, the data management module includes:

[0024] Garbage calorific value prediction unit, used to predict fluctuations in calorific value supply to power plants using garbage composition data;

[0025] Sorting strategy self-optimization unit is used to dynamically adjust the sorting priority.

[0026] In a specific implementation scheme, the sorting strategy self-optimization unit trains an LSTM model based on historical data and dynamically adjusts the sorting priority.

[0027] In a second aspect, a method for an intelligent garbage sorting robot based on multimodal perception is provided, comprising the following steps:

[0028] Utilize the multimodal sensing subsystem to detect garbage shape in real time, determine material stiffness, and identify components in real time;

[0029] Using a sorting execution module to sort garbage in real time according to the recognition results of the multimodal perception subsystem;

[0030] Use the rebate control module to calculate the rebate amount and make fee payments.

[0031] In the above technical solution, a multimodal sensing subsystem is set up to detect the shape of garbage in real time, determine the material stiffness in real time and identify the components; a sorting execution module is used to sort the garbage in real time according to the recognition results of the multimodal sensing subsystem; and a rebate control module is used to calculate the rebate amount and make fee payments; thereby improving the sorting efficiency and sorting quality of garbage classification.

[0032] In a specific embodiment, it also includes:

[0033] The data management module is used to predict the calorific value of garbage and self-optimize the sorting strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A structural block diagram of an intelligent garbage sorting robot system based on multimodal perception provided in an embodiment of the present application;

[0035] Figure 2 A flowchart of the intelligent garbage sorting robot method based on multimodal perception provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below with reference to the accompanying drawings and examples, through which the features and advantages of the present application will become more clearly understood.

[0037] The word "exemplary" is used exclusively herein to mean "serving as an example, example, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0038] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0039] To facilitate understanding of the intelligent garbage sorting robot system and method based on multimodal perception provided by the embodiment of the present application, its application scenario is first explained. The intelligent garbage sorting robot system and method based on multimodal perception provided by the embodiment of the present application are used to improve the sorting efficiency and sorting quality of garbage sorting. The participation rate of residents in garbage sorting is low, and the cost of manual sorting is high; traditional sorting equipment relies on single visual recognition, and the error rate for identifying mixed garbage and irregular objects is too large; the rebate process for recyclables is cumbersome and it is difficult to motivate users to continue participating. The existing technology only determines the type of garbage by weight sensor and cannot distinguish the material. Or it is necessary to preset garbage delivery ports, such as special ports for metal, plastic, and paper, and the "no need to classify" function is not realized. For this reason, the embodiment of the present application provides an intelligent garbage sorting robot system and method based on multimodal perception to improve the sorting efficiency and sorting quality of garbage sorting. The following is a detailed description of the embodiment with reference to specific drawings.

[0040] refer to Figure 1 and Figure 2 , Figure 1 A structural block diagram of an intelligent garbage sorting robot system based on multimodal perception provided in an embodiment of the present application; Figure 2 A flowchart of the intelligent garbage sorting robot method based on multimodal perception provided in an embodiment of the present application.

[0041] exist Figure 1 In the embodiment of the present application, an intelligent garbage sorting robot system based on multimodal perception is provided, comprising:

[0042] Multimodal sensing subsystem for real-time detection of garbage morphology, determination of material stiffness, and component identification;

[0043] A sorting execution module, configured to sort garbage in real time according to the recognition result of the multimodal sensing subsystem;

[0044] The rebate control module is used to calculate the rebate amount and make fee payments.

[0045] In the above technical solution, a multimodal sensing subsystem is set up to detect the shape of garbage in real time, determine the material stiffness in real time and identify the components; a sorting execution module is used to sort the garbage in real time according to the recognition results of the multimodal sensing subsystem; and a rebate control module is used to calculate the rebate amount and make fee payments; thereby improving the sorting efficiency and sorting quality of garbage classification.

[0046] Specifically, the beneficial effects of the intelligent garbage sorting robot system based on multimodal perception include:

[0047] Improve sorting efficiency

[0048] Rapid Perception and Decision-Making: The multimodal perception subsystem can detect garbage shape, determine material stiffness, and identify its composition in real time. This multi-dimensional, rapid information acquisition capability enables the robotic system to accurately determine garbage properties in a fraction of the time. Compared to traditional manual sorting or other machine sorting methods that rely on a single perception method, this significantly shortens garbage identification and decision-making time, thereby significantly improving overall sorting efficiency.

[0049] Real-time sorting execution: The sorting execution module, based on the rapid identification results of the multimodal perception subsystem, can immediately sort garbage in real time. The entire process requires no human intervention or waiting for additional information processing. The modules work closely together to achieve a seamless transition from identification to sorting, further accelerating sorting speed and enabling more garbage to be processed per unit time, meeting the needs of large-scale waste sorting scenarios.

[0050] Improve sorting quality

[0051] Accurate Identification Capabilities: The multimodal sensing subsystem utilizes a variety of sensing methods (such as vision, touch, and spectral analysis) to comprehensively inspect garbage. Compared to a single sensing method, it can obtain garbage characteristic information more comprehensively and accurately. This multimodal sensing method effectively avoids misjudgments caused by the limitations of a single sensing method, thereby greatly improving the accuracy of garbage component identification, ensuring that different types of garbage can be correctly classified, and significantly improving sorting quality.

[0052] Adapting to complex waste situations: In actual waste sorting scenarios, the shapes, materials, and composition of waste are often complex and diverse. Leveraging the powerful sensing capabilities of its multimodal perception subsystem, the robotic system can better cope with these complex situations. For example, it can effectively distinguish and accurately classify mixed materials, irregularly shaped waste, and similar-looking waste of different materials, reducing sorting errors and further improving sorting accuracy.

[0053] Promote resource recycling and environmental benefits

[0054] Optimizing resource recycling: Because the system can improve the quality and efficiency of waste sorting, recyclables can be separated more accurately. This helps to increase the purity and recovery value of recyclables, providing a better raw material base for subsequent resource recycling, promoting the circular use of resources, reducing the demand for new resources, and alleviating the pressure on the environment caused by resource extraction.

[0055] Reduced environmental pollution: Accurate waste sorting ensures that hazardous waste is properly handled, preventing it from mixing with other waste and causing environmental pollution. Efficient sorting also helps reduce secondary pollution during waste disposal, such as reducing harmful gas emissions from the mixing of biodegradable and non-biodegradable materials in landfills, thus positively impacting environmental quality.

[0056] Improved economic benefits

[0057] Rebate Incentives: The rebate control module provides financial incentives for users. By calculating rebate amounts and processing payments, users are encouraged to actively participate in waste sorting, increasing their enthusiasm and cooperation. This helps increase the amount of recyclables collected, generating greater economic benefits for businesses and related organizations, while also reducing the overall cost of waste disposal.

[0058] Reduced operating costs: The intelligent waste sorting robot system's efficient sorting capabilities reduce the significant labor costs associated with manual sorting. Furthermore, the improved sorting quality reduces subsequent processing costs due to sorting errors (such as re-sorting costs and waste costs due to misclassification), thus reducing the overall operating costs of waste sorting and improving the project's economic benefits.

[0059] Promote the development of intelligent garbage sorting

[0060] Technological Innovation Demonstration: This system integrates multiple advanced technologies, including multimodal sensing and intelligent control, providing new technical solutions and demonstration cases for the waste sorting sector. Its successful application will help drive the entire waste sorting industry towards intelligent and automated development, and promote further research and innovation in related technologies.

[0061] Data accumulation and analysis: During operation, the system accumulates a large amount of waste sorting data, including characteristic information of different types of waste and sorting efficiency data. This data can provide strong support for subsequent algorithm optimization, equipment improvement, and the formulation of waste sorting policies, promoting more scientific and refined waste sorting management.

[0062] In a specific embodiment, the multimodal perception subsystem includes:

[0063] Vision module for real-time detection of garbage forms;

[0064] The tactile module is used to judge the material stiffness through grasping feedback;

[0065] The component analysis module is used to identify the type of plastic and metal components using near-infrared spectroscopy.

[0066] Specifically, the beneficial effects of the multimodal perception subsystem include:

[0067] Vision module: Accurately captures garbage shapes and lays the foundation for efficient sorting

[0068] Comprehensive shape perception: The vision module can detect the shape of garbage in real time, clearly capturing both regular shapes (such as cans and cartons) and irregular shapes (such as used clothing and knotted plastic bags). This comprehensive shape perception capability provides detailed foundational information for subsequent sorting decisions, enabling the robotic system to more accurately determine the garbage grabbing method and sorting path, avoiding sorting errors and inefficiencies caused by misjudgment of shape.

[0069] Adapting to Complex Environments: In waste sorting scenarios, the environment in which waste is stored is often complex, with occlusion and overlap. The vision module, with its high-resolution imaging and advanced image processing algorithms, can effectively cope with these complex situations, accurately identifying the morphological characteristics of each individual piece of waste. It can also precisely locate target waste even in a pile, ensuring a smooth sorting process.

[0070] Assisting with intelligent identification and classification: By detecting the shape of garbage in real time, the vision module can compare it with a pre-set garbage shape database to quickly identify the type of garbage (such as recyclables, hazardous waste, kitchen waste, etc.). This intelligent identification function greatly shortens the decision-making time for garbage classification, improves sorting efficiency, and also reduces the workload and subjective errors of manual classification.

[0071] Tactile module: Accurately senses material stiffness, improving sorting safety and accuracy

[0072] Material Stiffness Determination: The tactile module uses gripping feedback to determine material stiffness, accurately distinguishing between different types of waste, such as soft paper, flexible plastic bottles, and hard metal products. This precise perception of material stiffness is crucial for the sorting execution module to select the appropriate gripping force and method. This prevents waste damage or sorting failures caused by improper gripping force, thereby improving sorting accuracy and success rates.

[0073] Enhanced sorting safety: During the sorting process, inaccurate judgment of the rigidity of the waste material can cause collisions or jams between the sorting equipment and the waste, potentially damaging the equipment. The tactile module can effectively prevent this from happening. It can sense the material characteristics of the waste in real time and promptly adjust the actions of the sorting execution module to ensure the safety and stability of the sorting process.

[0074] Adapting to diverse waste materials: Waste comes from a wide variety of materials, and the stiffness of different materials varies significantly. The introduction of a tactile module enables the robotic system to better adapt to this diversity. Whether it's common paper, plastic, or metal waste, or some special materials, tactile feedback can be used to accurately identify and process it, improving the system's versatility and adaptability.

[0075] Composition analysis module: quickly identify plastic and metal components and optimize resource recycling

[0076] Accurately Identify Plastic Types: The component analysis module uses near-infrared spectroscopy to identify plastic types, accurately distinguishing between different types of plastics, such as polyethylene (PE), polypropylene (PP), and polyvinyl chloride (PVC). Different types of plastics have varying recycling values and treatment methods. Accurately identifying plastic types allows for separate recycling, improving the purity and quality of recycled plastics, providing a better raw material base for subsequent plastic recycling, and promoting resource recycling.

[0077] Accurate Metal Composition Analysis: For metal waste, the component analysis module can accurately analyze its metal composition, such as iron, aluminum, and copper. This not only helps to classify different metals for recycling, improving the efficiency and quality of metal recycling, but also provides metal smelting companies with more accurate raw material information, allowing them to carry out reasonable smelting and processing based on metal composition, reducing production costs and improving resource utilization.

[0078] Improving the value of recycled resources: By accurately identifying plastic types and metal components, this module enables more refined sorting and processing of recyclables, thereby increasing their market value. This also complies with national policies on resource recycling and helps promote the development of the waste sorting industry towards higher quality and more sustainable development.

[0079] The combined beneficial effects include:

[0080] Significantly improves sorting efficiency and quality: The three modules of the multimodal perception subsystem work together to comprehensively perceive and analyze waste from different dimensions, providing accurate and detailed information to the sorting execution module. This enables the sorting execution module to make sorting decisions quickly and accurately, significantly improving the efficiency and quality of waste sorting and reducing sorting errors and duplication.

[0081] Reduced labor costs and intensity: This intelligent waste sorting robot system automates and intelligentizes waste sorting, reducing reliance on human labor. Powered by a multimodal perception subsystem, the robot independently detects waste morphology, identifies its material, and identifies its composition. This reduces the cost and intensity of manual sorting, while also avoiding potential fatigue and misjudgment during the process.

[0082] Driving intelligent upgrades in the waste sorting industry: This intelligent waste sorting robot system, based on multimodal sensing technology, has brought new technological changes to the waste sorting industry. Its successful application will help drive the entire industry towards intelligent and automated development, improving the industry's overall technological level and competitiveness.

[0083] In a specific implementation scheme, the visual module uses the YOLOv7 algorithm to detect garbage forms in real time.

[0084] Specifically, the beneficial effects of the visual module using the YOLOv7 algorithm to detect garbage forms in real time include:

[0085] 1. Real-time detection and efficient response

[0086] Fast processing capability: The YOLOv7 algorithm is renowned for its exceptional speed, processing input images and outputting detection results in record time. In intelligent waste sorting robot systems, the vision module employs the YOLOv7 algorithm to detect waste shape in real time, ensuring the robot system can quickly respond to waste and adjust sorting strategies, improving the efficiency of the entire sorting process. For example, on a conveyor belt, where waste moves at a constant speed, the YOLOv7 algorithm can quickly identify the shape of each piece of waste, ensuring the robot doesn't miss a chance to sort due to detection delays.

[0087] Low latency: The algorithm's low latency enables the vision module to work closely with the sorting execution module, achieving a seamless transition from detection to sorting. This low-latency detection capability is particularly important in real-time waste sorting scenarios, effectively reducing waste accumulation during transport and improving overall system throughput.

[0088] 2. High-precision detection to reduce misjudgment

[0089] Accurately Identify Waste Shapes: The YOLOv7 algorithm demonstrates high accuracy in object detection tasks, accurately identifying waste of various shapes, sizes, and colors. Whether it's regular-shaped cans and mineral water bottles or irregularly shaped items like used clothing and plastic bags, the algorithm can accurately detect them. This high-precision detection capability helps reduce sorting errors caused by misjudgment and improves the accuracy of waste sorting.

[0090] Strong anti-interference capabilities: In real-world waste sorting scenarios, interference factors such as changing lighting, complex backgrounds, and occlusion of trash can occur. The YOLOv7 algorithm, through its advanced network structure and training methods, possesses strong anti-interference capabilities, enabling it to stably detect trash shapes in complex environments. For example, in dimly lit waste processing workshops or garbage dumps, the algorithm can still accurately identify the location and shape of trash.

[0091] 3. Strong adaptability to cope with various types of garbage

[0092] Handling Multiple Waste Types: Garbage comes in many different forms and types. The YOLOv7 algorithm can learn the characteristics of different types of garbage through extensive training data, enabling detection of various forms of garbage. Whether it's recyclables (such as paper, plastic, and metal), hazardous waste (such as batteries and light bulbs), or kitchen waste (such as fruit peels and vegetable leaves), the algorithm can effectively identify them, meeting the diverse waste handling needs of intelligent waste sorting robot systems.

[0093] Adapting to new types of waste: As society develops and people's lifestyles change, new types of waste may emerge. The YOLOv7 algorithm has good scalability. By adding new training samples, it can quickly adapt to the detection of new types of waste without requiring large-scale modifications to the entire system, reducing system maintenance costs and upgrade difficulties.

[0094] 4. Lightweight design, easy to deploy

[0095] Small model size: Compared to some other complex object detection algorithms, the YOLOv7 algorithm optimizes model size while maintaining detection accuracy, resulting in smaller model parameters and computational complexity. This allows the visual module to be deployed using lightweight hardware, reducing system hardware costs and power consumption. For example, at waste sorting sites with cost and space constraints, embedded devices equipped with the YOLOv7 algorithm can implement visual detection capabilities.

[0096] Easy Integration: The open-source nature and excellent compatibility of the YOLOv7 algorithm make it easy to integrate with other modules and systems. In the intelligent waste sorting robot system, the vision module can easily communicate and collaborate with the sorting execution module and rebate control module, achieving seamless integration and efficient operation of the entire system.

[0097] 5. Continuous optimization to improve performance

[0098] Algorithm Updateability: As an open-source object detection algorithm, the YOLOv7 algorithm continuously produces new research results and improved versions. The vision module of the intelligent garbage sorting robot system can be updated and optimized in a timely manner based on actual needs and technological developments to improve detection performance and adapt to new application scenarios. For example, more advanced network structures or training techniques can be adopted to further enhance the algorithm's detection accuracy and speed.

[0099] Data-driven optimization: By collecting and analyzing real-world garbage detection data, we can optimize the YOLOv7 algorithm for specific applications. For example, by adding training samples or adjusting algorithm parameters to address specific types of garbage or complex detection scenarios, we can better adapt the vision module to actual workloads and continuously improve the accuracy and efficiency of garbage classification.

[0100] In a specific implementation scheme, the sorting execution module includes:

[0101] Sorting compartment, used to direct garbage to different cavities based on identification results;

[0102] Collaborative robotic arm for grasping objects.

[0103] Specifically, the beneficial effects of the sorting execution module include:

[0104] 1. For sorting compartments

[0105] Accurate classification to improve resource recycling quality

[0106] Accurate guidance based on recognition results: The sorting chamber can accurately direct different types of waste to different cavities based on the recognition results of multimodal perception subsystems (such as vision, touch, and component analysis modules). For example, the sorting chamber can send recyclables (plastic bottles, cans, etc.), hazardous waste (batteries, light bulbs, etc.), and kitchen waste (fruit peels, leftovers, etc.) into corresponding cavities, avoiding the mixing of different types of waste. This improves the purity of each type of waste, provides high-quality raw materials for subsequent resource recycling, and enhances the value and efficiency of resource recovery.

[0107] Reduced secondary sorting costs: By achieving more accurate classification during the sorting phase, the workload and cost of subsequent manual or machine sorting are significantly reduced. Traditional waste sorting methods can suffer from inaccurate classification, resulting in the need for significant additional manpower and resources for further sorting during the recycling process. The precise guidance function of the sorting chamber effectively avoids this problem, reducing the cost of the entire waste sorting process.

[0108] Optimize space utilization and improve sorting efficiency

[0109] Reasonable cavity layout: The sorting chamber is designed with different cavities, which can be rationally arranged according to actual waste sorting needs and site conditions. This layout fully utilizes space, allowing waste to be quickly and orderly distributed to the corresponding cavities during the sorting process, reducing the time waste spends in the sorting area and improving sorting efficiency. For example, at waste disposal sites with limited space, optimizing the cavity layout can achieve efficient waste sorting within the confined space.

[0110] Parallel processing capability: Multiple cavities can simultaneously receive different types of waste, enabling parallel processing. This means the sorting chamber can process more waste in the same amount of time, further increasing the throughput of the entire waste sorting system and meeting the needs of large-scale waste sorting and processing.

[0111] 2. For collaborative robotic arms

[0112] Flexible grabbing to adapt to various types of garbage

[0113] Coping with Complex Waste Shapes: The collaborative robotic arm possesses a high degree of flexibility and motion precision, adapting to various complex waste handling tasks. Whether the waste is regularly shaped (such as cube-shaped cardboard boxes and cylindrical cans) or irregularly shaped (such as knotted plastic bags and tangled wires), the robotic arm can accurately grasp the target waste by adjusting its joint angle and gripping force, ensuring that the waste enters the sorting compartment smoothly for classification.

[0114] Handling waste of varying materials: Different waste materials have varying physical properties, such as hardness, weight, and surface smoothness. The collaborative robotic arm can automatically adjust its gripping strategy based on the material characteristics of the waste. For example, for soft paper waste, the robotic arm can use a lighter grip to avoid tearing the waste; whereas for hard metal products, the robotic arm can apply a stronger grip to ensure stable grip. This adaptability to diverse waste materials improves the versatility and reliability of the sorting execution module.

[0115] Efficient collaboration to improve overall performance

[0116] Close coordination with the sorting chamber: The collaborative robotic arm and the sorting chamber achieve efficient collaboration. After grabbing trash, the robotic arm quickly and accurately places it in the designated location in the sorting chamber, ensuring smooth entry into the appropriate cavity. This close coordination reduces waste transfer time and errors, improving the efficiency of the entire sorting process.

[0117] Intelligent Scheduling and Optimization: By connecting to an intelligent control system, the collaborative robotic arms can achieve intelligent scheduling and optimization. For example, based on the distribution of garbage and the priority of sorting tasks, the control system can rationally arrange the working order and paths of the robotic arms, avoiding collisions and conflicts between the robotic arms, and improving their efficiency and utilization.

[0118] 3. For the overall sorting execution module

[0119] High degree of automation, reducing labor costs

[0120] Reduced Human Intervention: The sorting execution module automates waste sorting through the collaborative operation of the sorting cabin and the collaborative robotic arm. This eliminates the need for extensive manual sorting and handling throughout the sorting process, significantly reducing human intervention and lowering labor costs. This also avoids potential problems such as fatigue and misjudgment that can arise during manual sorting, improving sorting accuracy and stability.

[0121] Adaptable to harsh working environments: Waste sorting often involves harsh working environments, with harmful factors such as odor, dust, and bacteria. The automated operation of the sorting execution module eliminates the need for workers to be exposed to such environments for extended periods, ensuring their health and safety.

[0122] Strong scalability to adapt to future development

[0123] Modular Design: The sorting execution module adopts a modular design, allowing components such as the sorting chamber and collaborative robotic arms to be flexibly configured and expanded according to actual needs. For example, as the types of waste to be sorted increase or the processing volume increases, the number of cavities in the sorting chamber or the number of collaborative robotic arms can be increased to meet the needs without requiring a large-scale renovation of the entire system, reducing the cost and difficulty of system upgrades.

[0124] Compatibility with technology upgrades: As technology continues to advance, the sorting execution module can easily integrate new technologies and equipment. For example, in the future, more advanced sensor technology can be used to improve the accuracy of waste identification, or more efficient robotic arm drive technology can be used to increase grasping speed and precision. The sorting execution module architecture is well compatible with these technological upgrades, maintaining the system's advanced nature and competitiveness.

[0125] In a specific implementation scheme, the rebate control module includes:

[0126] Dynamic pricing unit, used to calculate rebate amounts by weight or quantity based on a cloud-based price database;

[0127] The contactless payment interface is used to support QR code / NFC binding of resident accounts and automatic settlement to the user terminal.

[0128] Specifically, the beneficial effects of the rebate control module include:

[0129] For dynamic pricing units

[0130] Accurate pricing to protect user rights

[0131] Based on a cloud-based price database: The dynamic pricing unit connects to a cloud-based price database that provides real-time updates on market prices for various recyclables. For example, if the market price of plastic bottles fluctuates due to changes in raw material supply or demand, the cloud-based database will promptly reflect this change. When calculating rebate amounts, the system uses the latest price data to ensure that user rebates align with current market values. This prevents user losses due to lagging price information and protects users' economic interests in the waste sorting and recycling process.

[0132] Flexible pricing: Rebate amounts can be calculated by weight or quantity, adapting to the characteristics of different types of waste. For waste that is heavy but small in quantity, such as scrap metal, pricing by weight more accurately reflects its value; whereas for waste that is large in quantity but light in weight, such as waste paper, pricing by quantity is more appropriate. This flexible pricing method allows for fairer and more reasonable rebate calculations, increasing user participation in waste sorting and recycling.

[0133] Promote rational allocation of resources and guide environmentally friendly behavior

[0134] Reflecting resource scarcity: Through dynamic pricing, rebate amounts vary with market price fluctuations for different recyclables. When a resource becomes scarce, its market price rises, and the corresponding rebate amount also increases. For example, with the strengthening of environmental protection policies and advancements in resource recycling technology, the recycling value of certain rare metals continues to increase. The dynamic pricing unit will adjust the rebate amount based on market prices, guiding users to prioritize the recycling of these resources and promoting the rational allocation and efficient utilization of resources.

[0135] Incentivizing environmentally friendly behavior: Users can intuitively see the difference in rebate amounts for different types of waste, which in turn motivates them to participate more actively in waste sorting and recycling. For example, when users learn that recycling used batteries not only earns them a certain rebate, but also that the rebate amount is adjusted based on market conditions, they are more willing to collect used batteries for recycling rather than discarding them, which helps raise environmental awareness and improve waste sorting and recycling rates across society.

[0136] For contactless payment interface

[0137] Convenient payment, improved user experience

[0138] Supports multiple binding methods: The contactless payment interface supports both QR code and NFC account binding. Users can choose the appropriate binding method based on their preferences and device availability. For smartphone users, scanning the QR code quickly binds the account; for users of NFC-enabled phones or smart devices, simply bringing the device close to the sensing area completes the binding. This simple and convenient operation significantly reduces the user barrier to entry.

[0139] Automatic Settlement to User Terminal: After the user completes trash placement and the waste is weighed or counted, the contactless payment interface automatically settles the rebate amount based on the dynamic pricing unit and directly transfers the payment to the user's linked account terminal. This entire process eliminates the need for manual user interaction, achieving a truly contactless payment experience and providing users with an efficient and convenient payment experience, saving them time and effort.

[0140] Ensure payment security and enhance user trust

[0141] Secure and reliable payment technology: The contactless payment interface utilizes advanced payment security technologies, such as encrypted transmission and identity verification, to ensure the security of user account information and payment data during transmission and storage. For example, when users bind their accounts and make payments, the system conducts strict identity verification to prevent account theft or information leakage, ensuring the security of users' funds.

[0142] Transparent settlement process: Users can access their rebate records and settlement status at any time through mobile apps or other devices. The settlement process is transparent and open. This transparency allows users to clearly understand the source and amount of their rebates, strengthens their trust in the payment system, and further increases their enthusiasm for participating in waste sorting and recycling.

[0143] For the rebate control module as a whole

[0144] Improve system operation efficiency and reduce management costs

[0145] Automated Processing: The rebate control module automates the entire process from pricing to payment, reducing manual intervention. In traditional waste sorting and recycling models, rebate calculation and payment often require manual processing, which is inefficient and prone to errors. This module's automated processing enables quick and accurate rebate calculation and payment settlement, significantly improving system operational efficiency and reducing management costs.

[0146] Centralized Data Management: Through a cloud-based price database and a unified payment interface, the rebate control module enables centralized data management. Operators can monitor rebate status, user account information, and other data in real time, facilitating data analysis and decision-making. For example, based on user rebate records and placement behavior, operators can analyze the engagement and demand characteristics of different regions and user types, allowing them to adjust operational strategies in a targeted manner and improve the overall efficiency of the system.

[0147] Promote the development of garbage classification industry and promote the progress of environmental protection

[0148] Business Model Innovation: The rebate control module provides an innovative business model for the waste sorting industry, attracting users to participate in waste sorting and recycling through economic incentives, thereby forming a sustainable industrial ecosystem. This model not only improves the efficiency and quality of waste sorting and recycling, but also drives the development of related industries such as resource recycling and environmental protection equipment manufacturing, promoting the prosperity of the waste sorting industry.

[0149] Significant social benefits: From a societal perspective, the promotion and application of this module will help raise environmental awareness and resource recycling rates across society, reduce environmental pollution caused by landfill and incineration, and promote environmental protection. Furthermore, by enabling users to experience the economic benefits of waste sorting and recycling through rebates, it will inspire more people to participate in environmental protection initiatives and foster positive social trends.

[0150] In a specific embodiment, it further includes: a data management module, wherein:

[0151] The data management module is used for garbage calorific value prediction and self-optimization of sorting strategies.

[0152] In a specific implementation scheme, the data management module includes:

[0153] Garbage calorific value prediction unit, used to predict fluctuations in calorific value supply to power plants using garbage composition data;

[0154] Sorting strategy self-optimization unit is used to dynamically adjust the sorting priority.

[0155] Specifically, the beneficial effects of the data management module include:

[0156] For garbage calorific value prediction unit

[0157] Helping power plants operate stably and ensuring energy supply

[0158] Predict calorific value fluctuations in advance: In the waste incineration power generation scenario, the stability of the calorific value of garbage is crucial to the operation of the power plant. The garbage calorific value prediction unit can predict the fluctuations in the calorific value supply of the power plant in advance based on the garbage composition data. For example, by analyzing the proportion of different types of garbage (such as paper, plastic, kitchen waste, etc.) and their respective combustion calorific values, the overall calorific value of the garbage entering the incinerator can be roughly estimated. When it is predicted that the calorific value may fluctuate greatly, the power plant can take measures in advance, such as adjusting the fuel ratio (appropriately increasing the supplement of high calorific value fuel) or optimizing the combustion parameters (adjusting the air volume, furnace temperature, etc.), so as to avoid the decline in power generation efficiency or even shutdown accidents due to unstable calorific value, and ensure a stable supply of energy.

[0159] Optimizing Power Generation Plans: Accurate calorific value forecasts enable power plants to develop more scientific and rational power generation plans. Based on predicted calorific value trends, power plants can rationally schedule the operating hours and loads of generating equipment, improving equipment utilization. For example, during periods of high calorific value, power generation can be increased appropriately to feed excess energy into the grid; during periods of low calorific value, power generation can be reduced or equipment maintenance can be scheduled, reducing operating costs and improving the power plant's economic efficiency.

[0160] Improve resource utilization efficiency and reduce environmental pollution

[0161] Precisely matching combustion needs: By predicting the calorific value of waste, power plants can more accurately match combustion needs to ensure that the waste is fully burned. Accurate calorific value predictions can avoid the generation of large amounts of harmful gases (such as carbon monoxide and dioxins) due to incomplete combustion caused by excessively high calorific values. They can also prevent the need to add excessive auxiliary fuel due to low calorific values, thereby reducing energy waste and environmental pollution. For example, in batches of waste with low calorific values, an appropriate amount of high-calorific-value auxiliary fuel can be added in advance to bring the calorific value of the mixed fuel into the optimal combustion range, improving combustion efficiency and reducing pollutant emissions.

[0162] Promoting waste sorting and resource recycling: Feedback from the waste calorific value prediction unit can guide front-end waste sorting efforts. When a certain type of waste is found to have a high calorific value and contribute significantly to power generation, efforts can be strengthened to separate and recycle it, improving resource recycling rates. For example, if plastic waste has a high calorific value, through publicity and guidance, residents can be encouraged to sort and dispose of it separately. This can increase the proportion of plastic waste entering the power plant, increasing its energy supply while promoting resource recycling.

[0163] Self-optimizing unit for sorting strategies

[0164] Improve sorting efficiency and reduce operating costs

[0165] Dynamically adapting to changes in garbage: The source and composition of garbage can vary with time, season, region, and other factors. The sorting strategy self-optimization unit can dynamically adjust sorting priorities based on actual conditions to ensure that sorting operations are always carried out efficiently. For example, during holidays, packaging waste (such as cartons and plastic bags) may increase significantly. At this time, the self-optimization unit can increase the sorting priority of this type of waste, allowing sorting equipment and staff to prioritize this waste, avoiding garbage accumulation and improving sorting efficiency.

[0166] Optimize resource allocation: By dynamically adjusting sorting priorities, sorting equipment and human resources can be rationally allocated. For waste types that currently require priority processing, more resources can be concentrated on sorting, while for waste that is not a priority, resource investment can be appropriately reduced. For example, when a certain type of recyclable material has high market demand and a high recycling value, the sorting priority for this type of recyclable material can be increased, and more robotic arms and personnel can be deployed for sorting, thereby improving resource recovery efficiency and reducing overall operating costs.

[0167] Improve classification accuracy to meet diverse needs

[0168] Precisely matching recycling standards: Different recycling companies or processing plants may have different waste sorting standards. The sorting strategy self-optimization unit can dynamically adjust sorting priorities based on the needs of downstream companies, ensuring that sorted waste meets the recycling standards of specific companies. For example, if a recycling company specializes in collecting high-purity plastic bottles, the self-optimization unit can increase the sorting priority of plastic bottles, strengthen the screening and classification of plastic bottles, and improve the purity of the sorted plastic bottles to meet the company's needs, thereby improving the quality of waste recycling and market competitiveness.

[0169] Adapting to changes in policies and regulations: As environmental policies and regulations continue to improve, requirements for waste sorting are becoming increasingly stringent. The sorting strategy self-optimization unit can promptly respond to changes in policies and regulations, adjusting sorting priorities to ensure that waste sorting meets relevant standards. For example, if policies strengthen the management of hazardous waste, the self-optimization unit will increase its sorting priority, increasing its identification and sorting efforts, preventing hazardous waste from entering the wrong disposal channels and reducing environmental risks.

[0170] For the entire data management module

[0171] Achieve data-driven decision-making and improve system intelligence

[0172] Data Integration and Analysis: The data management module integrates multiple data sources, including waste composition and sorting process data. Through in-depth analysis of this data, it provides strong support for waste calorific value prediction and self-optimization of sorting strategies. For example, by analyzing historical waste composition and calorific value data, a calorific value prediction model is established; by analyzing efficiency and accuracy data during the sorting process, sorting strategies are optimized. This data-driven decision-making approach makes the entire waste sorting and processing system more intelligent and scientific.

[0173] Continuous Learning and Improvement: The data management module is capable of continuous learning and improvement. As data accumulates and updates, the calorific value prediction model and sorting strategies can be continuously optimized and refined. For example, when new waste types emerge or waste composition changes, the system can automatically adjust model parameters and sorting strategies to adapt to the new situation, improving system adaptability and stability.

[0174] Promote the coordinated development of industries and advance the progress of environmental protection

[0175] Upstream and Downstream Industry Linkage: The data management module closely connects the front-end (residential placement and collection), mid-end (sorting and processing), and back-end (power plants and recycling companies) of waste sorting, achieving coordinated development of upstream and downstream industries. Through calorific value prediction and self-optimization of sorting strategies, the front-end can better understand the needs of the back-end and adjust classification standards. The back-end can optimize production plans and resource utilization based on the waste information provided by the front-end. For example, based on calorific value prediction results, power plants provide feedback to waste sorting companies on their needs for different types of waste. The waste sorting companies then adjust their sorting strategies based on this feedback to increase the utilization value of the waste.

[0176] Promoting Industry Standardization: The application of the data management module will help promote standardization in the waste sorting and processing industry. Through unified data collection and analysis standards, data from different regions and enterprises can be shared and exchanged, providing a basis for the development of unified standards and norms for the industry. For example, establishing a nationwide waste composition database and calorific value prediction model will provide a reference for waste sorting and processing in various regions, promote the standardization of the entire industry, and advance environmental protection.

[0177] In a specific implementation scheme, the sorting strategy self-optimization unit trains an LSTM model based on historical data and dynamically adjusts the sorting priority.

[0178] Specifically, the sorting strategy self-optimization unit trains the LSTM model based on historical data and dynamically adjusts the sorting priority, which has the following beneficial effects:

[0179] Improve sorting efficiency

[0180] Accurately predict junk traffic and type distribution

[0181] Advantages of the LSTM model: The LSTM (Long Short-Term Memory) model excels at processing and predicting time-series data. In waste sorting scenarios, it can learn from historical sorting data, including information such as the quantity and type of various types of waste within different time periods. For example, at a community waste sorting station, by analyzing daily waste delivery data (morning, noon, and evening) over the past year, the LSTM model can predict the approximate flow and type distribution of various types of waste at different times of the day for a future day.

[0182] Dynamically adjust sorting resources: Based on these predictions, the sorting strategy self-optimization unit can dynamically adjust sorting priorities. For example, if a significant increase in food waste is predicted during the morning hours, the sorting priority for food waste is increased, and more sorting personnel and equipment are deployed to the food waste sorting area in advance to prevent waste accumulation and improve overall sorting efficiency.

[0183] Quickly adapt to changing garbage trends

[0184] Addressing seasonal and cyclical changes: The sources and composition of waste vary with the seasons and cycles. For example, packaging waste (such as cartons and plastic bags) increases significantly during holidays, while food waste generation surges in the summer. The LSTM model can proactively detect these changing trends by learning from seasonal and cyclical patterns in historical data.

[0185] Timely adjustment of sorting strategies: The sorting strategy self-optimization unit adjusts sorting priorities based on model predictions. For example, before the arrival of summer, the sorting priority of food waste is increased, and the number of food waste collection containers and sorting frequency are increased to ensure timely and efficient waste processing.

[0186] Improve classification accuracy

[0187] Learning garbage features and classification relationships

[0188] Multi-dimensional feature analysis: The LSTM model can analyze multiple features of garbage, including appearance (color, shape, size), physical properties (weight, texture), and composition. For example, by learning from a large amount of historical data on plastic bottles, the model can understand the differences in appearance and composition between different brands and materials, as well as their correspondence with different classification standards (recyclable plastic, non-recyclable plastic).

[0189] Precisely guide sorting operations: The sorting strategy self-optimization unit dynamically adjusts sorting priorities based on the model's learning results, guiding sorters or robotic arms to more accurately sort. For example, when a plastic bottle with certain appearance and composition characteristics that meet recyclable plastic standards is detected, its sorting priority is increased to ensure that this type of waste is correctly classified and recycled.

[0190] Reduce human error

[0191] Unified Classification Standards: During the actual sorting process, human factors can lead to inconsistent classification standards, resulting in misjudgments. The LSTM model, by learning from a large amount of historical classification data, can establish unified and accurate classification standards. The sorting strategy self-optimization unit adjusts sorting priorities based on the model's results, making sorting more standardized.

[0192] Assisted sorting decisions: For difficult-to-identify waste, sorters can refer to the model's classification suggestions and priority adjustment information to reduce the possibility of misclassification. For example, for waste that looks similar but has different composition, the model can provide a more accurate classification based on its characteristics. Sorters can then adjust sorting priorities accordingly to improve classification accuracy.

[0193] Reduce operating costs

[0194] Optimize human resource allocation

[0195] Deploy personnel based on demand: Using an LSTM model to predict garbage flow and type distribution, the sorting strategy self-optimization unit can rationally allocate sorting personnel. Staffing can be reduced during periods of low garbage volume or ease of sorting; increased during periods of high garbage volume or increased sorting difficulty. For example, during weekday daytime hours when garbage is relatively light, the number of sorting personnel can be appropriately reduced; while at night, when residents gather to deposit garbage, additional personnel can be added to ensure smooth sorting operations.

[0196] Improving Workforce Efficiency: Dynamically adjusting sorting priorities allows sorters to focus more on the types of waste that need to be processed first, avoiding wasting time on non-critical tasks and thus improving work efficiency. For example, raising the sorting priority of high-calorific-value recyclable waste allows sorters to prioritize this type of waste, which not only increases the value of resource recovery but also improves work output.

[0197] Reduce equipment idleness and loss

[0198] Rational use of sorting equipment: Rationally arrange the use of sorting equipment based on the dynamic adjustment of sorting priorities. For waste types that require priority processing, ensure that the relevant sorting equipment is in an efficient operating state. For waste that is not a priority, the frequency of equipment use can be appropriately reduced or it can be placed in standby mode to reduce the idle time of the equipment. For example, when a certain type of recyclable material has a high sorting priority, increase the operating time of the corresponding sorting equipment (such as conveyor belts and sorting robotic arms) to improve equipment utilization.

[0199] Extending equipment life: Avoiding prolonged high-load operation or unnecessary idling can reduce equipment wear and tear and extend its life. For example, by properly adjusting sorting priorities, equipment can operate more smoothly and orderly, reducing damage caused by frequent starts and stops or overloads, thereby reducing equipment repair and replacement costs.

[0200] Enhance system adaptability and flexibility

[0201] Adapt to different scenario needs

[0202] Multi-scenario data learning: The LSTM model can learn from historical data on waste sorting in different scenarios (such as communities, shopping malls, and office buildings) to understand the characteristics and classification requirements of waste in various scenarios. For example, community waste often consists of a large proportion of kitchen waste, while shopping mall waste often contains more packaging waste and recyclables.

[0203] Personalized sorting strategies: The sorting strategy self-optimization unit dynamically adjusts sorting priorities based on the characteristics of different scenarios and formulates personalized sorting strategies. For example, in community scenarios, it prioritizes kitchen waste and recyclables; in shopping malls, it prioritizes the sorting of packaging waste and valuable recyclables (such as metal products and electronics).

[0204] Quickly respond to policy and market changes

[0205] Fusion of policy and market data: With the continued strengthening of environmental protection policies and changes in market demand for resource recycling, the standards and priorities for waste sorting will also adjust accordingly. The LSTM model can integrate policy and regulatory information (such as new classification standards and recycling targets) and market data (such as price fluctuations of various recyclables) into historical data for analysis.

[0206] Timely Adjustment of Sorting Direction: The sorting strategy self-optimization unit adjusts sorting priorities based on the model's analysis results, ensuring that waste sorting meets policy requirements and market demands. For example, when policies encourage the recycling of a specific recyclable (such as used batteries), its sorting priority is increased to increase recycling volume. When the market price of a recyclable increases significantly, its sorting priority is also increased accordingly to improve the economic benefits of resource recovery.

[0207] exist Figure 2 In the embodiment of the present application, a method for an intelligent garbage sorting robot based on multimodal perception is provided, comprising the following steps:

[0208] Utilize the multimodal sensing subsystem to detect garbage shape in real time, determine material stiffness, and identify components in real time;

[0209] Using a sorting execution module to sort garbage in real time according to the recognition results of the multimodal perception subsystem;

[0210] Use the rebate control module to calculate the rebate amount and make fee payments.

[0211] In the above technical solution, a multimodal sensing subsystem is set up to detect the shape of garbage in real time, determine the material stiffness in real time and identify the components; a sorting execution module is used to sort the garbage in real time according to the recognition results of the multimodal sensing subsystem; and a rebate control module is used to calculate the rebate amount and make fee payments; thereby improving the sorting efficiency and sorting quality of garbage classification.

[0212] In a specific embodiment, it also includes:

[0213] The data management module is used to predict the calorific value of garbage and self-optimize the sorting strategy.

[0214] Those skilled in the art will appreciate that the present application may be implemented as a system, method, or computer program product.

[0215] Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present disclosure may be implemented in the form of a computer program product embodied in one or more computer-readable media, wherein the computer-readable media contains computer-readable program code.

[0216] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0217] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application. Various substitutions and improvements may be made to the present application on this basis, all of which fall within the scope of protection of the present application.

Claims

1. An intelligent garbage sorting robot system based on multimodal perception, characterized in that: include: Multimodal sensing subsystem for real-time detection of garbage morphology, determination of material stiffness, and component identification; A sorting execution module, configured to sort garbage in real time according to the recognition result of the multimodal sensing subsystem; The rebate control module is used to calculate the rebate amount and make fee payments.

2. The intelligent garbage sorting robot system based on multimodal perception according to claim 1 is characterized in that: The multimodal perception subsystem includes: Vision module for real-time detection of garbage forms; The tactile module is used to judge the material stiffness through grasping feedback; The component analysis module is used to identify the type of plastic and metal components using near-infrared spectroscopy.

3. The intelligent garbage sorting robot system based on multimodal perception according to claim 2 is characterized in that: The visual module uses the YOLOv7 algorithm to detect garbage forms in real time.

4. The intelligent garbage sorting robot system based on multimodal perception according to claim 3 is characterized in that: The sorting execution module includes: Sorting compartment, used to direct garbage to different cavities based on identification results; Collaborative robotic arm for grasping objects.

5. The intelligent garbage sorting robot system based on multimodal perception according to claim 4 is characterized in that: The rebate control module includes: Dynamic pricing unit, used to calculate rebate amounts by weight or quantity based on a cloud-based price database; The contactless payment interface is used to support QR code / NFC binding of resident accounts and automatic settlement to the user terminal.

6. The intelligent garbage sorting robot system based on multimodal perception according to claim 5 is characterized in that: Also includes: Data management module, which The data management module is used for garbage calorific value prediction and self-optimization of sorting strategies.

7. The intelligent garbage sorting robot system based on multimodal perception according to claim 6 is characterized in that: The data management module includes: Garbage calorific value prediction unit, used to predict fluctuations in calorific value supply to power plants using garbage composition data; Sorting strategy self-optimization unit is used to dynamically adjust the sorting priority.

8. The intelligent garbage sorting robot system based on multimodal perception according to claim 7 is characterized in that: The sorting strategy self-optimization unit trains the LSTM model based on historical data and dynamically adjusts the sorting priority.

9. An intelligent garbage sorting robot method based on multimodal perception, characterized in that: The following steps are involved: Utilize the multimodal sensing subsystem to detect garbage shape in real time, determine material stiffness, and identify components in real time; Using a sorting execution module to sort garbage in real time according to the recognition results of the multimodal perception subsystem; Use the rebate control module to calculate the rebate amount and make fee payments.

10. The intelligent garbage sorting robot method based on multimodal perception according to claim 9 is characterized in that: Also includes: The data management module is used to predict the calorific value of garbage and self-optimize the sorting strategy.

Images