An intelligent sorting monitoring system based on Internet of Things

Abstract

The application provides an intelligent sorting monitoring system based on the Internet of Things. A multispectral imaging module erected above a material input device captures the spectral characteristics of the articles, and a 3D structured light camera array scans to generate three-dimensional point cloud data. A dynamic task allocation controller receives the feature data set of the multi-modal perception system, and generates a cooperative motion instruction of the grabbing posture and grabbing path of the mechanical arm according to the feature data of the articles. The end of the gripper of the mechanical arm is provided with a variable stiffness flexible gripper, and the surface of the gripper is provided with a tactile sensor, so that the dual mechanical arms dynamically adjust the state in the process of grabbing the materials. The adjustable angle of the slide is set to control the conveying speed of the articles and ensure smooth arrival at the outlet, so that the AGV can be docked on time, and the sorting efficiency is improved. The system realizes efficient, accurate and safe intelligent sorting operation through multi-sensor data fusion, real-time dynamic control and multi-subsystem cooperative optimization.

Description

Technical Field

[0001] This invention relates to the field of logistics sorting technology, and in particular to an intelligent sorting and monitoring system based on the Internet of Things. Background Technology

[0002] Traditional logistics sorting systems generally suffer from low recognition accuracy, poor adaptability, and insufficient collaborative efficiency. Current automated sorting equipment mostly relies on a single vision sensor, making it difficult to accurately distinguish items of similar materials (such as different types of plastic or glass), and lacks precise three-dimensional spatial positioning capabilities, leading to frequent grasping errors. Robotic arms are typically equipped with grippers of fixed rigidity, unable to automatically adjust gripping force according to material characteristics, easily causing damage or detachment when handling fragile or heavy items. The conveying links in the sorting process often use fixed-angle slide designs, unable to dynamically adjust according to the weight and surface characteristics of the materials, resulting in small items flying or large items getting stuck. The connection between AGV transfer systems and sorting exits mainly relies on manually preset programs, resulting in defects such as time asynchrony and large positional deviations, seriously affecting overall sorting efficiency. Furthermore, existing systems lack intelligent learning capabilities, unable to optimize sorting strategies based on historical data, requiring frequent manual intervention when handling complex scenarios (such as stacked items or irregularly shaped parts). These technical deficiencies severely restrict the performance and reliability of sorting systems, necessitating a new sorting solution capable of multi-dimensional perception, intelligent decision-making, and precise collaboration. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide an Internet of Things-based intelligent sorting and monitoring system that overcomes the above problems, and can solve the problems of low sorting efficiency and low reliability of existing sorting systems, so as to achieve efficient, accurate and safe intelligent sorting operations.

[0004] Specifically, the present invention provides an IoT-based intelligent sorting and monitoring system, which includes: A material input device, which integrates a weight sensing belt and thermal imaging modules disposed on both sides of the weight sensing belt, for conveying materials and simultaneously acquiring the weight and temperature data of the materials; A multimodal sensing system includes a multispectral imaging module, a 3D structured light camera array, and a data fusion processor. The spectral feature output terminal of the multispectral imaging module and the point cloud data output terminal of the 3D structured light camera are connected to the input terminal of the data fusion processor to construct a three-dimensional point cloud model of the material and extract the spectral features of the material. A dual-arm collaborative system includes two robotic arms equipped with variable stiffness flexible grippers, a tactile sensor array, and a dynamic task allocation controller. The tactile sensor array is connected to the dynamic task allocation controller, which receives material characteristic data from the multimodal sensing system at its input end and generates collaborative motion commands for the two robotic arms at its output end. The sorting execution device includes a reconfigurable chute system, an AGV docking module, and multiple AGV trolleys. The reconfigurable chute system is signal-connected to the material input device and the multimodal sensing system to adjust the tilt angle of the chute based on the material characteristic data acquired by the material input device and the multimodal sensing system. The inlet end of the chute is used to receive materials from the robotic arm. The AGV docking module has a laser positioning device at the chute outlet that is communicatively connected to the AGV trolleys.

[0005] Optionally, the multispectral imaging module includes: The three-band integrated camera mechanism includes a visible light camera, a near-infrared sensor, and a thermal imager. A spectral feature analysis unit, wherein the spectral feature analysis unit uses a convolutional neural network to extract the feature vector of the material; A 3D point cloud modeling unit connects to a 3D structured light camera signal to enable both to collaboratively model the material.

[0006] Optionally, the dynamic task allocation controller includes: A real-time load monitoring unit is used to collect the joint torque of the robotic arm and the gripping end pose data of the robotic arm. A path optimization algorithm module, which calculates the shortest collision-free path for the two robotic arms to work together based on an improved ant colony algorithm; A stiffness adjustment command generation unit is provided, wherein the tactile sensor is disposed on the gripping end of the mechanical wall; the stiffness adjustment command generation unit adjusts the gripping force of the robotic arm on the material according to the pressure distribution data of the tactile sensor.

[0007] Optionally, the IoT-based intelligent sorting and monitoring system also includes: The edge-cloud collaborative control system includes: An edge layer, comprising a local FPGA processing unit, which processes the raw data stream of the multimodal sensing system in real time to generate feature data of the material; The cloud layer includes a cloud optimization engine and a digital twin simulator; the cloud optimization engine receives the feature data and feeds back a sorting strategy to the edge layer; The digital twin simulator synchronously maps the operating status of the robotic arm, the slide, and the material.

[0008] The collaborative control module is connected to the edge layer and the cloud layer via industrial Ethernet, and the edge layer and the cloud layer are connected to the material input device, the multimodal sensing system, the dual robotic arm collaborative system and the sorting execution device.

[0009] Optionally, the slide rail includes multiple slide rail modules; each slide rail module is provided with an electromagnetic locking buckle at both its front and rear ends; multiple slide rail modules are connected through the electromagnetic locking buckles.

[0010] Optionally, the reconfigurable slide system includes: An angle adjustment mechanism, which is connected to the slide rail, is used to control and adjust the tilt angle of the slide rail; A dynamic tilt adjustment mechanism includes an acceleration sensor and a PID controller. The acceleration sensor is used to detect the vibration, slope, and / or acceleration of the material in the slide in real time. The PID controller works with the tilt adjustment mechanism and the acceleration sensor to calculate a compensation amount based on the real-time detection data of the acceleration sensor, so that the tilt adjustment mechanism can dynamically adjust the tilt angle of the slide.

[0011] Optionally, it also includes an anomaly detection module, including: A vibration acquisition sensor is mounted on the robotic arm and is used to acquire the vibration spectrum of the robotic arm. A vibration analysis unit, connected to the vibration acquisition sensor, is used to acquire and analyze the vibration spectrum; A thermal imaging monitoring unit is used to monitor the temperature distribution of the material input device and / or the drive mechanism of the robotic arm in real time. The fault prediction module is connected to the vibration analysis unit and the thermal imaging monitoring unit. It integrates thermal imaging data and vibration spectrum and uses algorithms to identify different fault modes.

[0012] Optionally, the AGV docking module is used to control the AGV vehicle, and the AGV docking module includes: A laser beacon forms a positioning grating array at the exit end of the slide rail; the positioning grating array is used to guide the AGV vehicle in positioning and calibrating its position. A container demand forecasting unit generates a scheduling instruction for the AGV to go to the exit end of a specific chute based on the material being sorted at the chute position.

[0013] Optionally, the AGV docking module further includes: A collision avoidance system includes multiple detection radars and a feedback unit. Each detection radar is installed on one of the AGVs and is used to detect obstacles within a preset range of the AGV. When an obstacle is detected within the preset range, the AGV stops or decelerates to avoid it. The feedback unit is connected to the robotic arm to provide feedback to the robotic arm on the AGV's stopping or deceleration obstacle avoidance information. When the robotic arm receives the AGV's stopping or deceleration obstacle avoidance information, it replans its grasping path.

[0014] Optionally, the digital twin simulator includes: A physics engine that simulates the rigid body dynamics of the material based on the Bullet engine; The strategy verification module pre-simulates the material sorting process in a virtual environment and evaluates the effectiveness of the strategy. The parameter migration unit synchronizes the optimized control parameters to the slide and the robotic arm.

[0015] In the IoT-based intelligent sorting and monitoring system of this invention, a material input device, a multimodal sensing system, a dual-robotic arm collaborative system, and a sorting execution device are incorporated. A multispectral imaging module mounted above the material input device captures the spectral characteristics of the items, while a 3D structured light camera array scans and generates three-dimensional point cloud data. These two types of data undergo feature fusion processing in a data fusion processor, outputting a feature dataset containing the material, shape, and position of the items. A dynamic task allocation controller receives the feature dataset from the multimodal sensing system and generates coordinated motion commands for the robotic arms' grasping posture and grasping path based on the item's feature data. These commands are then sent to the two robotic arms, which execute the grasping action on the material.

[0016] The robotic arms are equipped with variable stiffness flexible grippers at the gripper ends, and tactile sensors are placed on the gripper surfaces. This allows the dual robotic arms to dynamically adjust their state during material handling, ensuring the material is smoothly grasped and placed onto the chute, thus guaranteeing material safety and preventing accidents. The adjustable angle of the chute controls the conveying speed of the items and ensures a smooth arrival at the exit, allowing materials to be safely and steadily transported down the chute, ensuring both efficiency and safety in material sorting. The AGV cart needs to accurately reach the chute exit to receive the materials. If the chute angle is not properly adjusted, the arrival time of the materials at the chute exit may be inconsistent, causing the AGV cart to wait or miss items. Therefore, setting the appropriate chute angle ensures that items arrive at the exit at a consistent speed, enabling the AGV to dock on time and improving sorting efficiency.

[0017] In summary, this system achieves efficient, accurate, and safe intelligent sorting operations through multi-sensor data fusion, real-time dynamic control, and multi-subsystem collaborative optimization.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic structural diagram of an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention. Figure 2 This is a schematic structural diagram of a multispectral imaging module in an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention. Figure 3 This is a schematic structural diagram of a dynamic task allocation controller in an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention. Figure 4 This is a schematic structural diagram of an anomaly detection module in an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention. Figure 5 This is a schematic structural diagram of a reconfigurable chute system in an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention. Figure 6 This is a schematic structural diagram of a chute in an IoT-based intelligent sorting and monitoring system according to an embodiment of the present invention.

[0020] In the diagram: 100, Material input device; 110, Weight sensing belt; 120, Thermal imaging module; 200, Multimodal sensing system; 210, Multispectral imaging module; 211, Three-band integrated camera mechanism; 212, Spectral feature analysis unit; 213, 3D point cloud modeling unit; 220, 3D structured light camera array; 230, Data fusion processor; 300, Dual robotic arm collaborative system; 310, Robotic arm; 320, Tactile sensor array; 330, Dynamic task allocation controller; 331, Real-time load monitoring unit; 332, Path optimization algorithm module; 33 3. Stiffness adjustment command generation unit; 400. Sorting execution device; 410. Reconfigurable slide system; 411. Angle adjustment mechanism; 412. Dynamic tilt angle adjustment mechanism; 420. AGV docking module; 430. AGV trolley; 500. Edge-cloud collaborative control system; 510. Edge layer; 520. Cloud layer; 530. Collaborative control module; 600. Anomaly detection module; 610. Vibration acquisition sensor; 620. Vibration analysis unit; 630. Thermal imaging monitoring unit; 640. Fault prediction module; 710. Slide module; 720. Electromagnetic latch. Detailed Implementation

[0021] The following reference Figures 1 to 6 In describing the IoT-based intelligent sorting and monitoring system of this embodiment of the invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature, that is, include one or more of that feature. In the description of this invention, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. When a feature "includes or includes" one or more of the features it encompasses, unless otherwise specifically described, this indicates that other features are not excluded and may be further included.

[0022] In the description of this embodiment, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0023] Figure 1 This is a schematic diagram of an IoT-based intelligent sorting and monitoring system, such as... Figure 1As shown, and refer to Figures 2 to 6 This invention provides an IoT-based intelligent sorting and monitoring system, which includes a material input device 100, a multimodal sensing system 200, a dual robotic arm collaborative system 300, and a sorting execution device 400.

[0024] The material input device 100 integrates a weight sensing belt 110 and thermal imaging modules 120 disposed on both sides of the weight sensing belt 110, for conveying materials and simultaneously acquiring the weight and temperature data of the materials.

[0025] The multimodal sensing system 200 includes a multispectral imaging module 210, a 3D structured light camera array 220, and a data fusion processor 230. The spectral feature output terminal of the multispectral imaging module 210 and the point cloud data output terminal of the 3D structured light camera are connected to the input terminal of the data fusion processor 230 to construct a three-dimensional point cloud model of the material and extract the spectral features of the material.

[0026] The dual-arm collaborative system 300 includes two robotic arms 310 equipped with variable stiffness flexible grippers, a tactile sensor array 320, and a dynamic task allocation controller 330. The tactile sensor array 320 is connected to the dynamic task allocation controller 330. The dynamic task allocation controller 330 receives material characteristic data from the multimodal sensing system 200 at its input terminal and generates collaborative motion commands for the two robotic arms 310 at its output terminal.

[0027] The sorting execution device 400 includes a reconfigurable chute system 410, an AGV docking module 420, and multiple AGV carts 430. The reconfigurable chute system 410 is signal-connected to the material input device 100 and the multimodal sensing system 200 to adjust the tilt angle of the chute based on the material characteristic data acquired by the material input device 100 and the multimodal sensing system 200. The inlet end of the chute is used to receive materials from the robotic arm 310, and the laser positioning device at the chute outlet of the AGV docking module 420 is communicatively connected to the AGV carts 430.

[0028] Specifically, the thermal imaging module 120 in the material input device 100 should be connected to the multimodal sensing system 200. The former provides the latter with temperature and weight data of the material on the weight sensing belt 110, thereby transmitting the data after data fusion processor 230 of the multimodal sensing system 200 to the dual robotic arm collaborative system 300 and the sorting execution device 400. Furthermore, since the multimodal sensing system 200 is located above the material input device 100, the multispectral imaging module 210 mounted above the material input device 100 captures the spectral features of the item, and the 3D structured light camera array 220 scans and generates three-dimensional point cloud data. The two types of data undergo feature fusion processing in the data fusion processor 230, outputting a feature dataset containing the item's material, shape, and location.

[0029] Furthermore, the dynamic task allocation controller 330 receives the feature dataset (feature data of the material, shape and position of the object, and temperature and weight data) from the multimodal perception system 200. Based on the feature data of the object, the dynamic task allocation controller 330 generates cooperative motion instructions for the gripping posture and gripping path of the robotic arm 310, and sends the instructions to the two robotic arms 310, which then perform the gripping action on the material.

[0030] Furthermore, the robotic arm 310 is a six-degree-of-freedom robotic arm 310, and the gripper end of the robotic arm 310 is equipped with a variable stiffness flexible gripper. The surface of the gripper is provided with a tactile sensor. The tactile sensor is used to monitor the state of the material and the gripping force of the gripper on the material during the gripping process, and feeds this data back to the dynamic task allocation controller 330 in real time. The dynamic task allocation controller 330 then transmits the data to the robotic arm 310, so that the dual robotic arms 310 dynamically adjust their state during the material gripping process to ensure that the material is smoothly gripped onto the slide, thereby ensuring the safety of the material and preventing accidents.

[0031] Furthermore, adjusting the chute angle primarily affects the speed and direction of the items moving along the chute. When the chute is tilted, gravity causes the items to slide down the slope. The larger the angle, the faster the descent, but an excessively large angle may cause the items to roll over or be damaged; if the angle is too small, the items may not slide smoothly. Therefore, adjusting the angle is to control the conveying speed of the items and ensure a smooth arrival at the exit, thereby enabling the materials to be safely and smoothly conveyed down the chute, ensuring the efficiency and safety of material sorting. It should be noted that the principle of changing the chute inclination angle is as follows: when the chute inclination angle θ increases, the acceleration of the item sliding down the chute is a = g*sinθ−μ*g*cosθ (μ is the coefficient of friction). By adjusting θ, the sliding speed of the items can be precisely controlled (avoiding high-speed collisions or stagnation).

[0032] Furthermore, the AGV 430 needs to accurately reach the exit end of the chute to receive materials. If the chute angle is not properly adjusted, the time it takes for materials to reach the chute exit may be unstable, causing the AGV 430 to wait or miss items. Therefore, ensuring the chute angle is appropriate guarantees that items arrive at the exit at a consistent speed, enabling the AGV to dock on time and improving sorting efficiency.

[0033] It should be noted that the coordination logic between the slide angle and the AGV trolley 430 is as follows: AGV arrival time for 430: TAGV = D / v (where D is the travel distance and v is the speed) Item sliding time

[0034] The control system adjusts θ to make Tslide≈TAGV±δt (δt≤1s).

[0035] In summary, this system achieves efficient (1200 pieces / hour), accurate (error <1mm), and safe (damage rate <0.1%) intelligent sorting operations through multi-sensor data fusion, real-time dynamic control, and multi-subsystem collaborative optimization.

[0036] During operation, the material input device 100 collects the weight and temperature data of the material in real time through the integrated weight sensing belt 110 and thermal imaging module 120, and transmits these basic physical characteristics to the multimodal sensing system 200. At the same time, the multispectral imaging module 210 and the 3D structured light camera array 220 mounted above the material input device 100 capture the spectral features and three-dimensional geometric information of the material, respectively. The data fusion processor 230 intelligently fuses these multi-dimensional sensing data (including weight, temperature, material spectral features and three-dimensional point cloud model) to generate a structured dataset containing complete features of the material.

[0037] After receiving this feature data, the dynamic task allocation controller 330 generates optimized collaborative operation instructions for the robotic arms 310 through intelligent algorithms. This includes precise gripping posture planning and motion path planning, controlling the variable stiffness flexible grippers on the two six-degree-of-freedom robotic arms 310 to perform adaptive gripping operations. Tactile sensors integrated into the gripper surfaces monitor the gripping force and material status in real time, forming a closed-loop feedback control system to ensure that all types of materials (from fragile items to heavy objects) can be safely and stably transferred to the adjustable-angle slide system. The slide system dynamically adjusts its tilt angle based on the material's weight, coefficient of friction, and other characteristics, precisely controlling the speed and trajectory of gravity-driven descent to ensure the material reaches the exit position in optimal condition. Simultaneously, the AGV scheduling system works in conjunction with the slide control system, using laser positioning to ensure that the AGV 430 docks precisely and on time at the slide exit, completing the entire sorting process.

[0038] In some embodiments of the present invention, such as Figure 2 As shown, the multispectral imaging module 210 includes a three-band integrated camera mechanism 211, a spectral feature analysis unit 212, and a three-dimensional point cloud modeling unit 213. The three-band integrated camera mechanism 211 includes a visible light camera, a near-infrared sensor, and a thermal imager. The spectral feature analysis unit 212 uses a convolutional neural network to extract the feature vector of the material. The three-dimensional point cloud modeling unit 213 connects the signals from the 3D structured light camera to enable both to collaboratively model the material.

[0039] Specifically, irregularly shaped materials may require adjustment of the slide angle to prevent them from getting stuck or rolling over. Material spectral characteristics relate to the surface properties of the object, such as the coefficient of friction, which directly affects the sliding speed and stability. The roles of 3D point clouds and material spectral characteristics in slide adjustment include: 1. Shape Analysis: Determine the geometric features of the object using a point cloud model to predict its sliding behavior.

[0040] 2. Material analysis: The surface friction coefficient is obtained through spectral characteristics, which affects the setting of the slide angle.

[0041] 3. Comprehensive parameter adjustment: Combining weight, shape, and material data, the optimal tilt angle is dynamically calculated to ensure smooth transport of items.

[0042] Furthermore, the visible light camera in the three-band integrated camera mechanism 211 acquires RGB images (resolution ≥4K) of the material surface to identify visual features such as color, texture, and printed markings, achieving basic classification (e.g., distinguishing between red and blue packaging boxes) with a positioning accuracy of ±0.5mm; the near-infrared sensor identifies material composition based on the absorption / reflection characteristics of materials at specific wavelengths; and the thermal imager detects the temperature distribution on the material surface to identify abnormal temperature rises (e.g., lithium battery overheating ≥60℃) and isolate hazardous materials. Therefore, three-band data fusion can simultaneously acquire the material's "appearance + composition + temperature" full-dimensional features, solving the blind spots of single optical modal recognition.

[0043] Furthermore, the spectral feature analysis unit 212 can obtain material information, surface condition, etc. by analyzing the material's reflection / absorption characteristics in different spectral bands. It uses a convolutional neural network (CNN) to automatically extract spectral feature vectors, replacing the traditional manual threshold judgment, and is suitable for high-speed sorting scenarios.

[0044] In some embodiments of the present invention, such as Figure 1 and Figure 3As shown, the dynamic task allocation controller 330 includes a real-time load monitoring unit 331, a path optimization algorithm module 332, and a stiffness adjustment command generation unit 333. The real-time load monitoring unit 331 collects the joint torque of the robotic arm 310 and the gripping end pose data of the robotic arm 310. The path optimization algorithm module 332 calculates the shortest collision-free path for the two robotic arms 310 to work collaboratively based on an improved ant colony algorithm. The tactile sensor is installed on the gripping end of the robotic arm. The stiffness adjustment command generation unit 333 adjusts the gripping force of the robotic arm 310 on the material based on the pressure distribution data from the tactile sensor.

[0045] In some embodiments of the present invention, such as Figure 1 As shown, the IoT-based intelligent sorting and monitoring system also includes an edge-cloud collaborative control system 500, comprising an edge layer 510, a cloud layer 520, and a collaborative control module 530. The edge layer 510 includes a local FPGA processing unit that processes the raw data stream from the multimodal sensing system 200 in real time to generate characteristic data of the material. The cloud layer 520 includes a cloud optimization engine and a digital twin simulator. The cloud optimization engine receives the characteristic data and feeds back sorting strategies to the edge layer 510.

[0046] The digital twin simulator synchronously maps the operating states of the robotic arm 310, the chute, and the material. The collaborative control module 530 is connected to the edge layer 510 and the cloud layer 520 via industrial Ethernet, and the edge layer 510 and the cloud layer 520 are connected to the material input device 100, the multimodal sensing system 200, the dual robotic arm collaborative system 300, and the sorting execution device 400.

[0047] Specifically, the sorting strategy includes path planning, grabbing order, slide angle adjustment, AGV scheduling instructions, etc. The cloud optimization engine is one part of it, located in the cloud, and is responsible for big data analysis and strategy optimization.

[0048] Furthermore, digital twins construct a "parallel world" in a virtual environment by synchronizing physical device data (such as the position of robotic arm 310 and material status) in real time. When sorting strategies need to be optimized, real-time simulation is used: new strategies (such as adjusting the path of robotic arm 310) are first run in the virtual model to predict results such as collisions and efficiency; safety verification: if the simulation passes (e.g., efficiency increases by 20% and there is no risk of collision), the strategy is encrypted and sent to the physical device; seamless switching: edge devices load new parameters within milliseconds, without the need for downtime debugging, and production continues. The effect: traditional solutions require several hours of downtime for trial and error. Therefore, this solution achieves zero downtime for strategy updates through a closed loop of "virtual trial and error → real-world execution," ensuring continuous production capacity.

[0049] Furthermore, the edge layer 510 uploads feature data (including material material, location, and weight) to the cloud layer 520. The cloud layer 520 sends optimized sorting strategy parameters (including robotic arm 310 path, slide tilt angle, and AGV scheduling instructions) to the edge layer 510. After the digital twin simulator pre-tests the feasibility of the strategy, it triggers a seamless strategy switch.

[0050] During operation, the edge-cloud collaborative control system begins with the local FPGA processing unit performing real-time analysis of the raw data stream from the multimodal sensing system 200. Hardware-accelerated algorithms extract the material's spectral characteristics, 3D coordinates, and weight information. Simultaneously, the edge controller directly drives the robotic arm 310 to perform basic grasping actions and adjust the chute angle based on its local rule base, ensuring continuous sorting operations. The structured feature data, after encryption, is uploaded to the cloud optimization engine. The engine, combining cross-node historical data, uses a federated learning framework to train a sorting strategy model, generating an optimization scheme that includes the robotic arm 310's collaborative path, the chute's dynamic adjustment formula, and the AGV scheduling sequence. This optimized scheme is then verified by a digital twin simulator using a physics engine. Once the strategy is validated, the cloud sends parameter packages to the edge, and the robotic arm 310 improves its grasping efficiency based on the new path planning.

[0051] The digital twin synchronously maps the real-time status of the physical system and continuously monitors the effect of strategy execution. If the trajectory deviation of the robotic arm 310 is detected to be greater than 5mm or the slide is congested, the strategy circuit breaker mechanism is immediately triggered to roll back to the stable version. At the same time, the abnormal data is fed back to the cloud to start adaptive optimization iteration, forming a closed-loop control link of "perception-decision-execution-evolution".

[0052] In some embodiments of the present invention, such as Figure 6 As shown, the slide rail includes multiple slide rail modules 710, and each slide rail module 710 is provided with an electromagnetic latch 720 at its front and rear ends. The multiple slide rail modules 710 are connected through the electromagnetic latches 720.

[0053] Specifically, the modular chute unit is connected to other chute units or bases via electromagnetic latches 720, allowing for the rapid assembly of chute lengths according to sorting requirements. The quick-connect mechanism of the electromagnetic latches 720 likely involves electromagnetic engagement, locking when energized and releasing when de-energized, facilitating replacement or layout adjustments. This quick-connect design improves system flexibility and maintenance efficiency; for example, it allows for rapid replacement of chute modules 710 when handling materials of different sizes, reducing downtime.

[0054] In some embodiments of the present invention, such as Figure 5As shown, the reconfigurable slide system 410 includes an angle adjustment mechanism 411 and a dynamic tilt adjustment mechanism 412. The angle adjustment mechanism 411 is connected to the slide and is used to control and adjust the tilt angle of the slide. The dynamic tilt adjustment mechanism 412 includes an acceleration sensor and a PID controller; the acceleration sensor is used to detect the vibration, slope, and / or acceleration of the material in the slide in real time; the PID controller works with the angle adjustment mechanism 411 and the acceleration sensor to calculate a compensation amount based on the real-time detection data of the acceleration sensor, so that the angle adjustment mechanism 411 dynamically adjusts the tilt angle of the slide.

[0055] Specifically, the six-axis accelerometer in the dynamic tilt adjustment mechanism 412 is used to monitor the real-time vibration and tilt status of the slide. The PID controller then adjusts the movement of the electric push rod based on this data to maintain the set tilt angle. In particular, when the sensor detects vibration or angle deviation, the PID controller calculates the compensation amount, drives the push rod to extend or retract, and dynamically adjusts the slide angle to ensure that the material slides down smoothly.

[0056] In some embodiments of the present invention, as shown in 4, the IoT-based intelligent sorting and monitoring system further includes an anomaly detection module 600, which includes a vibration acquisition sensor 610, a vibration analysis unit 620, a thermal imaging monitoring unit 630, and a fault prediction module 640.

[0057] The vibration acquisition sensor 610 is mounted on the robotic arm 310 and is used to acquire the vibration spectrum of the robotic arm 310. The vibration analysis unit 620 is connected to the vibration acquisition sensor 610 and is used to acquire and analyze the vibration spectrum. The thermal imaging monitoring unit 630 is used to monitor the temperature distribution of the material input device 100 and / or the drive mechanism of the robotic arm 310 in real time. The fault prediction module 640 is connected to the vibration analysis unit 620 and the thermal imaging monitoring unit 630, and fuses thermal imaging data and vibration spectrum to identify different fault modes through algorithm analysis.

[0058] Specifically, the fault prediction module 640 needs to integrate temperature and vibration data, and may use machine learning models or rule engines to identify different fault modes. For example, high temperature accompanied by vibration at a specific frequency may indicate bearing failure, while localized overheating may indicate an electrical problem. Classification and early warning may be based on these combined characteristics to categorize faults into mechanical, electrical, overheating, etc., and trigger different levels of alarms.

[0059] Furthermore, the fault prediction module 640 performs classification and early warning by integrating temperature data from the thermal imaging monitoring unit 630 and spectral characteristics from the vibration analysis unit 620. The thermal imaging unit captures the surface temperature field of the equipment in real time and identifies local overheating (e.g., bearing temperature rise > 70℃); the vibration sensor collects acceleration signals, and the vibration analysis unit 620 extracts characteristic frequencies (e.g., bearing fault characteristic frequency 1kHz ± 5%). The fault prediction module 640 integrates temperature gradient, vibration spectrum, and historical data, and uses a machine learning model (e.g., random forest) to classify fault types: abnormal temperature is mainly classified as overheating, abnormal vibration is mainly classified as mechanical wear, and both are classified as serious faults. The warning level is determined according to confidence level (e.g., >90% triggers emergency shutdown). The module communicates with each unit in real time via an industrial bus to ensure spatiotemporal alignment of temperature and vibration data, improving diagnostic accuracy.

[0060] In some embodiments of the present invention, the AGV docking module 420 is used to control the AGV trolley 430, and the AGV docking module 420 includes a laser beacon that forms a positioning grating array at the exit end of the slide rail. The positioning grating array is used to guide the AGV trolley 430 in positioning and calibrating its position.

[0061] Specifically, the primary function of laser beacons is to provide precise positioning information. The AGV (Automated Guided Vehicle) 430 needs to accurately reach the sorting exit location, and laser beacons can help the AGV identify and locate target points by emitting specific patterns of gratings or beams. For example, laser beacons may form a positioning grating array around the sorting exit, and the AGV adjusts its position by detecting the position of these gratings to ensure precise docking.

[0062] Furthermore, laser beacons can also be used to calibrate the position of AGVs in real time, especially in dynamic environments where the exit position of the sorting system may change due to adjustments in the chute angle. Real-time feedback from laser beacons can help AGVs adjust their paths promptly, avoiding collisions or misalignments.

[0063] A container demand forecasting unit generates a scheduling instruction for the AGV 430 to the exit end of a specific chute based on the material being sorted at the chute position.

[0064] In some embodiments of the present invention, the AGV docking module 420 further includes an anti-collision system, which includes multiple detection radars and a feedback unit. Each detection radar is disposed on one of the AGV vehicles 430. The detection radar is used to detect obstacles within a preset range of the AGV vehicle 430, so that when an obstacle is detected within the preset range, the AGV vehicle 430 stops or decelerates to avoid the obstacle. The feedback unit is connected to the robotic arm 310 to provide feedback to the robotic arm 310 on the stopping or deceleration of the AGV vehicle 430 to avoid the obstacle. When the robotic arm 310 receives the stopping or deceleration information of the AGV vehicle 430, it replans the grasping path.

[0065] Specifically, the core function of the collision avoidance system is to ensure that equipment does not collide with each other during operation, thus guaranteeing safety and efficiency. It uses technologies such as UWB radar to monitor the surrounding environment in real time, detecting obstacles or moving objects. When a potential collision risk is detected, the system triggers emergency braking or path adjustment. For example, the AGV slows down and stops when approaching an obstacle, and the robotic arm 310 replans its path. This reduces equipment damage and production interruptions, improving system reliability. Simultaneously, optimizing the path avoids unnecessary detours, improving overall efficiency. Combined with sensor data fusion, the system can adapt to complex environmental changes, ensuring continuous and safe operation.

[0066] In some embodiments of the present invention, the digital twin simulator includes a physics engine, a strategy verification module, and a parameter migration unit. The physics engine simulates the rigid body dynamics of the material based on the Bullet engine. The strategy verification module pre-simulates the material sorting process in a virtual environment and evaluates the effectiveness of the strategy. The parameter migration unit synchronizes the optimized control parameters to the chute and the robotic arm 310.

[0067] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. An intelligent sorting and monitoring system based on the Internet of Things, characterized in that, include: A material input device, which integrates a weight sensing belt and thermal imaging modules disposed on both sides of the weight sensing belt, for conveying materials and simultaneously acquiring the weight and temperature data of the materials; A multimodal sensing system includes a multispectral imaging module, a 3D structured light camera array, and a data fusion processor. The spectral feature output terminal of the multispectral imaging module and the point cloud data output terminal of the 3D structured light camera are connected to the input terminal of the data fusion processor to construct a three-dimensional point cloud model of the material and extract the spectral features of the material. A dual-arm collaborative system includes two robotic arms equipped with variable stiffness flexible grippers, a tactile sensor array, and a dynamic task allocation controller. The tactile sensor array is connected to the dynamic task allocation controller, which receives material characteristic data from the multimodal sensing system at its input end and generates collaborative motion commands for the two robotic arms at its output end. The sorting execution device includes a reconfigurable chute system, an AGV docking module, and multiple AGV carts. The reconfigurable chute system is signal-connected to the material input device and the multimodal sensing system to adjust the chute's tilt angle based on material characteristic data acquired by the material input device and the multimodal sensing system. The chute's inlet end is used to receive materials from the robotic arm. The AGV docking module includes a laser positioning device located at the chute's outlet, which is communicatively connected to the AGV carts. The slide rail includes multiple slide rail modules; each slide rail module is equipped with an electromagnetic locking buckle at both its front and rear ends; multiple slide rail modules are connected through the electromagnetic locking buckles. The reconfigurable slide system includes: An angle adjustment mechanism, which is connected to the slide rail, is used to control and adjust the tilt angle of the slide rail; A dynamic tilt adjustment mechanism includes an acceleration sensor and a PID controller. The acceleration sensor is used to detect the vibration, slope, and / or acceleration of the material in the slide in real time. The PID controller is connected to the tilt adjustment mechanism and the acceleration sensor to calculate a compensation amount based on the real-time detection data of the acceleration sensor, so that the tilt adjustment mechanism can dynamically adjust the tilt angle of the slide.

2. The IoT-based intelligent sorting and monitoring system according to claim 1, characterized in that, The multispectral imaging module includes: The three-band integrated camera mechanism includes a visible light camera, a near-infrared sensor, and a thermal imager. A spectral feature analysis unit, wherein the spectral feature analysis unit uses a convolutional neural network to extract the feature vector of the material; A 3D point cloud modeling unit is connected to a 3D structured light camera signal, and the two work together to model the material.

3. The IoT-based intelligent sorting and monitoring system according to claim 1, characterized in that, The dynamic task allocation controller includes: A real-time load monitoring unit is used to collect the joint torque of the robotic arm and the gripping end pose data of the robotic arm. A path optimization algorithm module, which calculates the shortest collision-free path for the two robotic arms to work together based on an improved ant colony algorithm; A stiffness adjustment command generation unit adjusts the gripping force of the robotic arm on the material based on the pressure distribution data of the tactile sensor.

4. The IoT-based intelligent sorting and monitoring system according to claim 1, characterized in that, Also includes: The edge-cloud collaborative control system includes: An edge layer, comprising a local FPGA processing unit, which processes the raw data stream of the multimodal sensing system in real time to generate feature data of the material; The cloud layer includes a cloud optimization engine and a digital twin simulator; the cloud optimization engine receives the feature data and feeds back a sorting strategy to the edge layer; The digital twin simulator synchronously maps the operating status of the robotic arm, the slide, and the material; The collaborative control module is connected to the edge layer and the cloud layer via industrial Ethernet, and the edge layer and the cloud layer are connected to the material input device, the multimodal sensing system, the dual robotic arm collaborative system and the sorting execution device.

5. The IoT-based intelligent sorting and monitoring system according to claim 1, characterized in that, It also includes an anomaly detection module, including: A vibration acquisition sensor is mounted on the robotic arm and is used to acquire the vibration spectrum of the robotic arm. A vibration analysis unit, connected to the vibration acquisition sensor, is used to acquire and analyze the vibration spectrum; A thermal imaging monitoring unit is used to monitor the temperature distribution of the material input device and / or the drive mechanism of the robotic arm in real time. The fault prediction module is connected to the vibration analysis unit and the thermal imaging monitoring unit. It integrates thermal imaging data and vibration spectrum and uses algorithms to identify different fault modes.

6. The IoT-based intelligent sorting and monitoring system according to claim 1, characterized in that, The AGV docking module is used to control the AGV vehicle. The AGV docking module includes: A laser beacon forms a positioning grating array at the exit end of the slide rail; the positioning grating array is used to guide the AGV vehicle in positioning and calibrating its position. The container demand forecasting unit generates a scheduling instruction for the AGV trolley to go to the exit end of the chute based on the location of the material to be sorted at the chute.

7. The IoT-based intelligent sorting and monitoring system according to claim 6, characterized in that, The AGV docking module also includes: A collision avoidance system includes multiple detection radars and a feedback unit. Each detection radar is installed on one of the AGVs and is used to detect obstacles within a preset range of the AGV. When an obstacle is detected within the preset range, the AGV stops or decelerates to avoid the obstacle. The feedback unit is connected to the robotic arm to provide feedback to the robotic arm on the AGV's stopping or deceleration obstacle avoidance information. When the robotic arm receives the AGV's stopping or deceleration obstacle avoidance information, it replans its grasping path.

8. The IoT-based intelligent sorting and monitoring system according to claim 4, characterized in that, The digital twin simulator includes: A physics engine that simulates the rigid body dynamics of the material based on the Bullet engine; The strategy verification module pre-simulates the material sorting process in a virtual environment and evaluates the effectiveness of the strategy. The parameter migration unit synchronizes the optimized control parameters to the slide and the robotic arm.

Images