AI vision-based construction waste intelligent control sorting system and sorting equipment

Abstract

The application relates to the technical field of solid waste treatment, and discloses an intelligent control sorting system and sorting equipment for construction waste based on AI vision, which integrates material conveying, visual sensing, multi-stage driving sorting and a central main control module; a main controller constructs a window load entropy model based on visual data, reversely adjusts the frequency of a front-end vibrating material uniformizing device to realize full-line load balancing, and effectively prevents the phenomenon of overload congestion or idling of rear-end mechanical arms; a special electro-hydraulic mixed drive gripper is arranged at the sorting end, which can intelligently switch between servo electric power soft grabbing and hydraulic rigid heavy load locking modes according to the physical properties of materials; in addition, the system calculates visual touch feature residual errors by using physical feedback in the grabbing process, and iteratively corrects the weight parameters of the visual recognition algorithm online. The application effectively solves the problems that the traditional sorting line has large flux fluctuation and complex rigid and flexible materials are difficult to be considered, and significantly improves the overall sorting efficiency and purity of the system.

Description

Technical Field

[0001] This invention relates to the field of solid waste treatment technology, specifically to an AI vision-based intelligent control and sorting system and equipment for construction waste. Background Technology

[0002] With the acceleration of urbanization and the continuous advancement of infrastructure construction, the amount of construction and demolition waste generated has increased dramatically. Traditional landfill disposal methods, due to their large land occupation and environmental pollution risks, are gradually shifting towards resource utilization. In the resource-based treatment process of construction waste, efficient sorting of different components such as concrete blocks, waste bricks, wood, plastics, and metals is a key step in achieving recycling. Because manual sorting suffers from high labor intensity, harsh working environments, and high efficiency due to worker fatigue, intelligent sorting equipment based on machine vision and automated actuators has gradually become a hot topic in industry research and application.

[0003] Existing intelligent sorting systems for construction waste typically consist of a belt conveyor, a vision recognition unit, and actuators. Their workflow is generally set as follows: a front-end feeding device feeds material into the conveyor belt at a fixed frequency; the vision unit captures images and identifies the material's location and type; subsequently, the control system drives the actuators to operate at specific times based on the recognition results, separating the target material from the mixed flow. These systems primarily rely on a pre-set algorithm model for unidirectional open-loop control, meaning that the execution action is directly triggered by visual signals, and the feeding speed is usually set to a constant value to maintain continuous operation of the production line.

[0004] While existing technologies have achieved some degree of automation in sorting operations, several shortcomings remain: First, the current system lacks a load linkage mechanism between the front-end feeding and the back-end sorting execution. The material distribution on the conveyor belt is often random and uneven. When materials arrive densely, a constant feeding speed can cause the instantaneous processing workload to exceed the robotic arm's movement limits, resulting in severe missed grabbing. Conversely, when materials are sparse, the robotic arm's computing power and motion resources are idle and wasted, leading to a mismatch between the system's overall throughput and energy efficiency. Second, construction waste has a complex composition and vastly different physical properties, including heavy and rigid concrete blocks as well as lightweight and fragile wood or flexible woven fabrics. The existing single-drive mode struggles to simultaneously ensure highly reliable locking of heavy materials and rapid, flexible grabbing of lightweight materials, resulting in heavy objects easily slipping and falling, and lightweight objects easily breaking and pulverizing. Finally, the identification method that relies solely on visual texture to infer the material and density of materials has limitations. Because construction waste is often covered with dust or has oil stains, there is often a non-linear deviation between visual features and physical properties. Existing technologies lack an online correction mechanism based on physical grasping feedback, which makes it impossible for the system to correct the grasping parameters according to the actual working conditions. The recognition accuracy and grasping stability under long-term operation are difficult to guarantee. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an AI vision-based intelligent control and sorting system and equipment for construction waste. This system solves the problems of low efficiency caused by the mismatch between feeding and sorting load in existing construction waste sorting systems, as well as the difficulty of using a single drive method to handle both light and heavy, rigid and flexible materials.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The first aspect of the present invention provides an intelligent control and sorting system for construction waste based on AI vision. The system includes a material conveying module, a vision perception module, a multi-stage drive sorting module, and a central control module connected via an industrial fieldbus. The material conveying module is equipped with a vibration equalization device, and the end of the multi-stage drive sorting module is provided with an electro-hydraulic hybrid drive gripper.

[0008] The central control module is configured to execute the following control logic: construct a sliding time window based on the data collected by the visual perception module, calculate the window load entropy representing the ratio of the current grasping task density to the theoretical processing capacity of the system; compare the window load entropy with a preset threshold, and generate a frequency adjustment command to reversely adjust the vibration frequency of the vibrating material distribution device, thereby changing the material distribution density;

[0009] Simultaneously, the driving mode of the electro-hydraulic hybrid drive gripper is switched according to the physical properties of the target material, and visual prediction data and physical gripping feedback data are compared during operation to calculate visual-touch feature residuals in order to correct the weight parameters of the visual recognition algorithm in real time.

[0010] Preferably, the specific steps for the central control module to calculate the window load entropy include:

[0011] The mask of the target material is extracted using a semantic segmentation algorithm, and the occlusion dispersion is calculated based on depth data.

[0012] By combining texture features with related databases to obtain estimated density, the estimated mass of the target material is calculated.

[0013] Using a nonlinear weighted model, the grasp confidence index of each target material is calculated based on semantic segmentation confidence, occlusion dispersion, and density adaptability factor;

[0014] The calculation logic of the density adaptability factor adopts a piecewise function: the factor is 1 when the estimated mass is less than or equal to the safe load threshold; the factor decreases linearly when the estimated mass is between the safe load threshold and the rated load limit; and the factor is forced to zero and marked as ungraspable when the estimated mass exceeds the rated load limit.

[0015] The window load entropy is obtained by summing the products of the grasp confidence index and the recovery value weight of all valid target materials within the sliding time window, and dividing by the theoretical maximum grasp quality throughput of the system after correction by the equipment state decay factor.

[0016] Preferably, the specific logic of the central main control module in reverse adjusting the vibration equalization device is as follows:

[0017] When the window load entropy is higher than the target load threshold, a frequency reduction command is generated to reduce the feeding speed of the vibrating material equalization device;

[0018] When the window load entropy is lower than the target load threshold, a frequency upsampling command is generated to increase the feeding speed of the vibrating material equalization device;

[0019] The frequency adjustment command is generated based on a proportional control algorithm, and the calculated target vibration frequency is limited to a preset physical safety range after amplitude limiting processing.

[0020] Preferably, the central control module is further configured to perform spatiotemporal flux dynamic programming:

[0021] Based on the combined speed of the material conveying module and the multi-stage drive sorting module, the shortest interception time and remaining dwell time of the target material are calculated, and unreachable targets are eliminated.

[0022] Calculate the operational efficiency ratio of the reachable target material. This operational efficiency ratio is the product of the grasp confidence index of the target material and the economic value weight, divided by the sum of the robotic arm movement time and the gripper action response time.

[0023] Within a finite time window, the optimal crawling sequence is generated with the objective function of maximizing total efficiency gains.

[0024] Preferably, the electro-hydraulic hybrid drive gripper adopts a structure in which a servo motor and a hydraulic cylinder are connected in parallel to drive the same linkage mechanism;

[0025] The hydraulic circuit of the electro-hydraulic hybrid drive gripper integrates a normally open large-diameter bypass unloading valve.

[0026] The normally open large-diameter bypass unloading valve is configured to remain open in non-hydraulic drive mode, simultaneously connecting the rodless chamber interface and the rod chamber interface of the hydraulic cylinder to the return oil tank, so that the piston rod of the hydraulic cylinder is in a low-resistance floating state with dual chambers open to the atmosphere.

[0027] Preferably, the specific logic for the central main control module to switch the electro-hydraulic hybrid drive gripper drive mode is as follows:

[0028] When a flexible material is detected, the electric drive mode is selected, the servo motor is controlled to work in torque mode, and the normally open large-diameter bypass unloading valve is kept open.

[0029] When a rigid material is detected, the hydraulic drive mode is selected. After the gripper is detected to be in contact with the material, the normally open large-diameter bypass unloading valve is closed and the hydraulic pump station is started. The hydraulic oil is used to push the linkage mechanism to lock.

[0030] The detection logic for contacting materials is as follows: when the end of the electro-hydraulic hybrid drive gripper enters the spatial enclosure of the target material, and the difference between the real-time measured current of the servo motor and the theoretical current of the no-load dynamic feedforward model exceeds a preset contact threshold, it is determined to be contact.

[0031] Preferably, the specific steps for the central control module to calculate the visual-tactile feature residuals include:

[0032] During the grabbing and lifting phase, the vertical axis current component of the servo drive or the feedback from the force sensor is monitored, and the actual physical mass is calculated after subtracting the unloaded gravity model.

[0033] By combining the actual grasping thickness and visual projection area, the equivalent physical volume is calculated based on the columnar approximation model, and then the actual physical density is obtained.

[0034] The visually estimated thickness and density obtained by the visual perception module are compared with the actual grasping thickness and the actual physical density, respectively, and the normalized deviation is calculated. The weighted sum is then used to obtain the visual-touch feature residual.

[0035] Preferably, the specific method for correcting the visual recognition algorithm parameters includes:

[0036] When the visual-touch feature residual exceeds the tolerance threshold, a feature suppression mask is generated. This mask is used to reduce the grasp confidence index weight of materials with the same texture identifier in subsequent operations.

[0037] A weighted moving average algorithm is used to iteratively update the density mapping value corresponding to the texture identifier in the associated database using the actual physical density measured in this task.

[0038] Preferably, the visual perception module includes a linear color camera and a three-dimensional laser contour scanner. The trigger end of the visual perception module is connected to an incremental encoder installed on the active roller end of the material conveying module, and a hardware trigger mode is used to realize the spatial registration of two-dimensional texture images and three-dimensional depth point cloud data.

[0039] A second aspect of the present invention provides a construction waste sorting device, the device comprising an AI vision-based intelligent control sorting system for construction waste as described in any of the preceding claims.

[0040] This invention provides an AI-based vision-based intelligent control and sorting system and equipment for construction waste. It offers the following advantages:

[0041] 1. This invention quantifies the material grasping task density within the current visual range by constructing a window load entropy model, and based on this, constructs a reverse closed-loop control from the back-end computing layer to the front-end feeding layer. The main controller can dynamically adjust the frequency of the vibrating material leveling device according to the real-time load, automatically slowing down the sparse material flow when the material is dense and accelerating the feeding when the material is sparse. This proactive intervention mechanism eliminates the overload and missed grasping or idle phenomena of the robotic arm caused by traditional fixed-frequency feeding, ensuring that the sorting system is always in the optimal operating throughput range.

[0042] 2. This invention employs an electro-hydraulic graded hybrid drive strategy, solving the problem that a single drive method cannot adequately address the characteristics of both rigid and flexible materials in construction waste. For lightweight, flexible materials, the torque mode of the servo motor enables rapid yet gentle gripping, preventing material breakage. For heavy, rigid materials, the high power density of the hydraulic system provides a large locking force to prevent falling. Simultaneously, the normally open bypass unloading valve integrated into the hydraulic circuit ensures low-resistance servoing in non-hydraulic mode, guaranteeing both high-frequency response under light loads and reliable clamping under heavy loads, significantly reducing the drop and breakage rates during operation.

[0043] 3. This invention establishes a visual-tactile rheological mapping and adaptive correction mechanism, utilizing the physical features fed back by the robotic arm during the grasping process to verify the estimated parameters of the visual recognition system. The system calculates the visual-tactile feature residuals in real time and iteratively updates the density mapping values ​​in the material database. This mechanism effectively overcomes the systematic bias caused by surface contamination or changes in lighting when relying solely on visual texture to judge material properties, enabling the sorting system to have online learning capabilities. As operational data accumulates, the accuracy of judging material properties under different working conditions gradually improves. Attached Figure Description

[0044] Figure 1 This is a diagram showing the overall architecture of the intelligent construction waste sorting system based on visual-touch fusion and load balancing of the present invention.

[0045] Figure 2 This is a perspective view of the sorting equipment of the present invention;

[0046] Figure 3 This is a flowchart of the multidimensional visual feature extraction and grasping confidence calculation process of the present invention;

[0047] Figure 4 This is a block diagram of the front-end reverse control logic based on window load entropy of the present invention.

[0048] Figure 5This is a flowchart of the spatiotemporal flux dynamic planning and path topology generation of the present invention;

[0049] Figure 6 This is a flowchart of the electro-hydraulic graded mixing drive control logic of the present invention;

[0050] Figure 7 This is a flowchart of the view-tactile rheotropic mapping and adaptive correction logic of the present invention;

[0051] In the formula: 10, material conveying module; 20, vision perception module; 30, multi-level drive sorting module; 40, central control module. Detailed Implementation

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] See attached document Figure 1 and attached Figure 2 , Figure 1 This diagram illustrates the overall architecture of a smart construction waste sorting system based on visual-touch fusion and load balancing, according to an embodiment of the present invention. The system mainly consists of a material conveying module, a visual perception module, a multi-level drive sorting module, and a central control module. The modules interact via a high-speed industrial fieldbus, forming a closed-loop control topology between the physical and information layers.

[0054] The material conveying module is located at the front of the production line and includes a vibrating material leveling device and a belt conveyor arranged in sequence. The vibrating material leveling device is equipped with a variable frequency drive eccentric vibrating motor. The control terminal of this vibrating motor is electrically connected to the analog output interface of the central main control module to receive frequency adjustment commands to change the material drop density. A high-precision incremental encoder is installed at the drive roller end of the belt conveyor. This encoder outputs pulse signals in real time reflecting the speed and displacement of the conveyor belt, providing a unified clock reference for the system's spatiotemporal synchronization.

[0055] The visual perception module, positioned directly above the belt conveyor, primarily comprises a high-frequency linear color camera, a 3D laser contour scanner, and a high-brightness shadowless light source. The trigger terminal of the visual perception module is directly connected to the incremental encoder of the belt conveyor, employing a hardware trigger mode for image acquisition to ensure strict spatial registration between the generated 2D texture image and the 3D depth point cloud data. The visual perception module transmits the acquired raw image data to the central control module via a gigabit Ethernet interface. The specific optical imaging principles of the linear camera and laser scanner are well-known technologies in the field and will not be elaborated upon here.

[0056] The multi-stage drive sorting module is located downstream of the vision perception module, and its main structure is a multi-axis Cartesian coordinate gantry robot. An electro-hydraulic hybrid drive gripper is installed at the end flange of the gantry robot, which integrates a servo motor drive module and a micro-hydraulic force amplification module. To address the hydraulic locking and damping issues caused by the volume difference between the rodless and rod chambers during the follow-up motion of a single-rod hydraulic cylinder, a normally open large-diameter bypass unloading valve is integrated into the hydraulic circuit. This valve is open in non-hydraulic drive mode, simultaneously connecting the rodless and rod chamber interfaces of the hydraulic cylinder to the return oil tank, ensuring that the piston rod is in a low-resistance floating state with both chambers open to the atmosphere when following the motor's movement. The servo drive of the gantry robot and the controller of the hybrid drive gripper communicate with the central main control module via an EtherCAT real-time Ethernet bus to execute path planning instructions and gripping action instructions.

[0057] The central control module is the core of the entire sorting system for computation and decision-making, typically employing an embedded industrial computer or a high-performance edge computing unit. The main controller runs a real-time operating system, responsible for receiving visual data, processing encoder signals, monitoring feedback from various sensors (such as current, pressure, and temperature), and calculating sorting strategies based on preset control algorithms. Specifically, the central control module balances the system load by inversely adjusting the frequency of the vibration equalization device, optimizes the gripping timing by planning the motion trajectory of the gantry robot, and adapts to the physical properties of different materials by switching the drive modes of the hybrid drive grippers.

[0058] The system architecture, through the interconnection of physical hardware, supports the implementation of subsequent specific methods and steps such as multi-dimensional visual feature extraction, front-end reverse regulation, spatiotemporal flux planning, and electro-hydraulic graded control.

[0059] See attached document Figure 3 , Figure 3 A flowchart illustrating the multi-dimensional visual feature extraction and grasping confidence calculation process according to an embodiment of the present invention is shown. The visual perception module is connected to the main controller via a high-speed industrial bus to establish the multi-dimensional feature space of the material in real time and quantitatively evaluate the grasping feasibility.

[0060] The visual perception module is equipped with a hardware trigger interface to synchronize with the encoder signal of the conveyor. When the material enters the detection area with the conveyor, the line scan camera component acquires high-resolution color texture images in a line scanning manner, while the 3D laser scanning component simultaneously acquires depth map data. The main controller uses a pre-calibrated hand-eye transformation matrix to uniformly register the pixel coordinate system of the color texture image with the spatial coordinate system of the depth map, ensuring a strict spatial correspondence between texture information and geometric information.

[0061] The main controller performs the following processing steps on the collected multidimensional data to build a crawling confidence model:

[0062] The main controller uses a semantic segmentation algorithm to process the registered image, identify independent connected regions within the field of view, and generate a mask and category label for each target material. Based on the depth data within the mask coverage area, the main controller performs 3D reconstruction, calculates the geometric center coordinates and minimum bounding box size of the target material.

[0063] To address the significant density variations in construction waste sorting scenarios, the main controller does not directly use visual volume as the basis for load determination. Instead, it introduces density estimation logic. The main controller integrates the depth pixel values ​​within the target material's mask area to obtain the target volume. Simultaneously, it extracts the texture feature descriptor of the target surface, uses this descriptor as a key, and searches a pre-defined texture-material-density association database to obtain the estimated material category and estimated density corresponding to the texture. Based on the target volume and estimated density, the main controller calculates the estimated mass of the target material.

[0064] The main controller uses the gradient information from the depth map to calculate the normal vector of each sampling point on the surface of the target material, and calculates the dispersion of the normal vector direction to generate occlusion dispersion. This index is used to characterize the roughness of the material surface and whether it is covered by other materials; when the occlusion dispersion value is large, it indicates that the normal direction of the object surface is messy, there is serious stacking or the shape is extremely irregular, making it unsuitable for gripping operations.

[0065] Based on the aforementioned characteristic parameters, the main controller calculates the grasping confidence index for each target material using a nonlinear weighted model. This index quantifies the probability that the gantry robot arm will successfully grasp the material without causing equipment overload. The calculation formula is as follows:

[0066] ;

[0067] In the formula; This indicates the confidence index for data capture. The confidence score represents the semantic segmentation score, and its value range is a normalized interval. It is used to characterize the certainty of the visual recognition algorithm in judging the target contour. This represents the occlusion dispersion calculated in the preceding steps; This represents the preset weighting coefficient, which is calibrated according to the specific needs of the sorting task; This represents the density adaptability factor. To avoid incorrectly lowering the grasp confidence level due to material approaching its rated load, the calculation logic of this factor is modified to a piecewise function:

[0068] ;

[0069] In the formula; This represents the estimated quality calculated in the preceding steps; Indicates the rated load limit of the gantry robot and hybrid drive gripper; This represents the preset safe load factor, which is 0.9 in this embodiment. This formula ensures the factor is 1 within the safe load range, linearly decays only when approaching the limit load, and is forcibly reset to zero when exceeding the limit. It should be noted that the system has an absolute veto logic: when... When the calculation result is 0, regardless of Based on the final score, the main controller forcibly marks the target material as ungraspable to prevent overload safety accidents. The main controller binds the calculated grasp confidence index to the target material and stores it in the dynamic task queue as the basis for subsequent path planning and drive mode decisions.

[0070] See attached document Figure 4 , Figure 4 A block diagram of a front-end reverse control logic based on window load entropy according to an embodiment of the present invention is shown. The main controller achieves adaptive and proactive intervention in material throughput by establishing a reverse feedback link from the back-end computing layer to the front-end physical feeding layer.

[0071] The main controller internally operates a sliding time window algorithm to dynamically monitor the instantaneous workload on the conveyor. It reads the encoder pulse signals from the conveyor servo motors in real time and constructs a pulse-count-based first-in-first-out queue in memory. The number of pulses covered by this queue corresponds to a fixed physical length on the conveyor, which is set as the sum of the gantry robot's maximum operating radius and the visual recognition buffer distance. As the conveyor operates, new material data is enqueued, and material data that has flowed out of the operating range is dequeued. The main controller updates the set of materials to be processed in the current window in real time accordingly.

[0072] Based on the crawl confidence data generated by the aforementioned steps, the main controller performs the following steps to achieve load balancing:

[0073] The main controller iterates through all target materials within the current sliding time window, filtering out valid targets with a grasping confidence index higher than a preset threshold. The main controller then retrieves the corresponding recycling value weight from a preset recycling value database based on the target material's category label. Subsequently, the main controller calculates the degree to which these valid targets consume the gantry robot's computing power and motion resources, generating a window load entropy. This index characterizes the ratio between the grasping task density within the current window and the system's theoretical maximum processing capacity. The formula for calculating the window load entropy is as follows:

[0074] ;

[0075] In the formula; This represents the window load entropy calculated at the current moment; Indicates the first in the window The confidence index for capturing a target material; Indicates the first The recycling value weight of each target material; This represents the theoretical maximum grasping mass throughput of the system per unit time. This constant is determined by the maximum acceleration of the gantry robot and the average grasping cycle. This represents the equipment status degradation factor. The main controller reads the hydraulic oil temperature of the hybrid drive gripper and the winding temperature of the servo motor via the industrial bus, and generates this factor through a normalized mapping function. This represents the upper limit of the set decay factor saturation; in this embodiment, 0.9 is selected to prevent calculation divergence caused by the denominator approaching zero. When the equipment condition deteriorates... As the load increases, the system's theoretical processing capacity decreases, which in turn increases the calculated load entropy value, triggering a more conservative feeding strategy.

[0076] The main controller compares the calculated window load entropy with the preset target load threshold and uses a proportional control algorithm to generate a frequency adjustment command for the vibrating feed homogenizer. This step adjusts the feed density through physical means to maintain the load entropy within the system's optimal processing range in subsequent time windows. The specific calculation logic for frequency adjustment is as follows:

[0077] ;

[0078] In the formula; This indicates the calculated target vibration frequency of the vibrating material homogenizer at the next moment; This indicates the actual feedback frequency of the vibrating material equalizer at the current moment; This represents the proportional gain coefficient, used to adjust the system's response sensitivity; This represents the target optimal load entropy threshold of the system.

[0079] It is worth noting that the calculated target vibration frequency It needs to be limited to [the specified range]. Within the physical safety zone, prevent the conveyor from stopping or reversing due to calculation divergence.

[0080] The main controller sends the calculated target vibration frequency to the frequency converter of the vibrating uniform material device through an industrial communication protocol.

[0081] When the window load entropy is significantly higher than the target optimal load entropy threshold, it indicates that the high-value grabbable material on the current conveyor is too dense, exceeding the instantaneous processing capacity of the gantry robot. At this time, the main controller sends a frequency reduction command. The vibration equalization device responds to the command by reducing the vibration frequency, thereby increasing the physical distance between the material falling onto the conveyor and reducing the material distribution density per unit length.

[0082] When the window load entropy is significantly lower than the target optimal load entropy threshold, it indicates that the system is underloaded. At this time, the main controller sends an up-frequency command. The vibrating material leveling device increases the feeding speed, increases the material throughput per unit time, and ensures that the sorting production line is operating at full capacity.

[0083] Through the above steps, this invention constructs a closed-loop control system that reversely determines the front-end feeding speed based on the back-end processing capability, effectively avoiding overload and missed grabbing of the robotic arm under high throughput conditions and idle computing power under low throughput conditions.

[0084] See attached document Figure 5 , Figure 5 A flowchart illustrating spatiotemporal throughput dynamic planning and path topology generation according to an embodiment of the present invention is shown. The main controller constructs a nonlinear dynamic scheduling model based on the real-time spatiotemporal state and value density of the materials, enabling real-time optimization of the sorting task sequence.

[0085] The main controller maintains a dynamic task pool that maps all valid targets within the conveyor sorting area. By reading real-time position feedback from the conveyor servo drive, the main controller periodically calculates the conveyor belt's displacement increment per unit time and uses this increment to update the global coordinates of each target material in the task pool, ensuring the robotic arm's target coordinates are always synchronized with its physical position. The main controller executes the following processing steps:

[0086] Because the material moves at high speed with the conveyor, the main controller needs to calculate the predicted position of the target material at the moment of future grabbing. Based on the linear velocity of the conveyor and the maximum combined velocity of the gantry robot, the main controller calculates the shortest interception time required for the robot arm to catch up with and contact the target material from its current position. Simultaneously, the main controller calculates the remaining dwell time of the target material as it moves from its current position to the boundary of the sorting area. If the shortest interception time is greater than the remaining dwell time, it indicates that the target material will move out of the working range before the robot arm arrives. The main controller marks the target material as unreachable and removes it from the task pool.

[0087] The main controller calculates the single-operation efficiency ratio for all reachable target materials in the task pool. This calculation incorporates the time cost of the robotic arm's actions. The main controller uses a trapezoidal velocity planning algorithm to estimate the total time it takes for the robotic arm to move from its current posture to the predicted target position and finally to the corresponding unloading point. The formula for calculating the operation efficiency ratio is as follows:

[0088] ;

[0089] In the formula; This represents the calculated efficiency ratio of a single operation; This represents the grasp confidence index calculated from the visual perception process; This indicates the material economic value weight corresponding to the target material; This represents the total inverse kinematics time required for the robotic arm's end effector to move from its current position to the target predicted position and finally reach the designated unloading point. This represents the inherent mechanical response time constant required for the hybrid drive gripper to complete the closing, locking, and releasing actions.

[0090] The main controller constructs a dynamic programming model in the time domain based on the calculated job efficiency ratio. This model generates a target capture sequence that maximizes total efficiency gains within a finite time window. The main controller employs a search strategy with time window constraints, prioritizing targets with high job efficiency ratios for inclusion in the sequence and verifying whether adding a target would render subsequent targets in the sequence unreachable due to time delays. The objective function for path planning is defined as follows:

[0091] ;

[0092] In the formula; This represents the total performance gain of the generated sequence; This represents the total number of tasks in the sequence; Indicates the first in the sequence The efficiency ratio of each task; This indicates that the robotic arm has reached the next stage after completing the previous task. The moment of each task; Indicates the first The critical moment when a task is removed from the sorting area; This is an indicator function; it takes the value 1 when the time constraint is met, and 0 otherwise.

[0093] Through this computational logic, the main controller can identify and eliminate tasks that, although high in individual value, cause the loss of multiple medium-value targets due to their remote location or excessive time consumption, thereby ensuring the maximization of global throughput value.

[0094] The sorting system operates in a continuous flow, with new materials constantly entering the detection field of view. The main controller refreshes the optimal grasping sequence at high frequency. When a new high-efficiency target enters the task pool, the main controller triggers an interrupt mechanism to re-evaluate the currently executing sequence. If the insertion of the new target can improve the overall efficiency gain within the remaining time window, the main controller immediately reconstructs the path topology, inserts the new task, and removes old tasks that have become unreachable due to time constraints. This mechanism ensures that the robotic arm always prioritizes the most cost-effective combination of targets within the current field of view.

[0095] See attached document Figure 6 , Figure 6 A flowchart of the electro-hydraulic graded hybrid drive control logic according to an embodiment of the present invention is shown. The hybrid drive gripper adopts a mechanical architecture in which a servo motor and a hydraulic cylinder drive the same linkage mechanism in parallel. The servo motor drives the linkage mechanism through a ball screw assembly to achieve rapid opening and closing of the fingers. The hydraulic cylinder, as an auxiliary force-amplifying element, is hinged at the lever arm of the linkage mechanism. Furthermore, the hybrid drive gripper integrates a miniature hydraulic pump station, eliminating the need for external long pipeline oil supply. The main controller dynamically switches the drive strategy based on the physical properties of the target material to balance non-destructive gripping of flexible materials with highly reliable clamping of rigid heavy objects.

[0096] The main controller performs the following steps to implement hierarchical adaptive control:

[0097] As the robotic arm moves to the grasping position, the main controller consults a pre-set database of material mechanical properties based on the material category label determined by the visual perception system and the estimated density. If the target material belongs to a pre-set set of flexible materials (e.g., plastic, wood, fabric), the main controller selects the electric drive mode; if the target material belongs to a pre-set set of rigid materials (e.g., concrete, metal, brick), the main controller selects the hydraulic drive mode.

[0098] Regardless of the selected mode, the hybrid drive gripper is initially driven independently by a servo motor. To avoid viscous damping in the hydraulic system during rapid movement, the main controller ensures the normally open bypass unloading valve is in a de-energized open state (or sends a maintain-open command). At this time, both the rodless and rod chambers of the hydraulic cylinder are connected to the oil tank, resulting in a low-resistance floating state with both chambers open to the atmosphere. The servo motor drives the fingers to close with high acceleration, shortening the idle stroke time. During this process, the main controller monitors the real-time current of the servo motor and compares it with the theoretical current of the pre-built no-load dynamic feedforward model to calculate the current residual. Simultaneously, the main controller introduces position constraint determination; contact detection logic is activated only when the coordinates of the robotic arm's end effector enter the three-dimensional bounding box of the target material. When the constraint conditions are met and the current residual exceeds the preset contact threshold, it is determined that the gripper fingers have contacted the material surface, and subsequent control logic is immediately triggered. The contact detection conditions are as follows:

[0099] ;

[0100] In the formula; This indicates the real-time measured current of the servo motor. This represents the theoretical no-load current calculated based on the current motion state (velocity, acceleration); This indicates the preset contact detection threshold, which must be greater than the maximum dynamic error caused by nonlinear friction and inertia during the motor's acceleration and deceleration under no-load conditions.

[0101] If the current mode is determined to be electric drive, the main controller switches the servo motor's control loop from the speed loop to the current loop (i.e., torque mode). The main controller sets a target reference current based on the material's fragility, maintaining a constant electromagnetic torque in the servo motor, thereby applying a controllable, flexible clamping force to the material surface via the linkage mechanism. At this time, the bypass unloading valve remains open, and the hydraulic cylinder continues to be in an unloaded, floating state.

[0102] If the current mode is determined to be hydraulic drive, the main controller, upon detecting a contact signal, controls the servo motor to maintain the current output torque to preserve the contact state. Simultaneously, it sends a closing command to the large-diameter bypass unloading valve and activates the micro hydraulic pump station and solenoid directional valve. High-pressure oil enters the rodless chamber of the hydraulic cylinder, further tightening the linkage mechanism. Due to the power density characteristics of the hydraulic system, the hybrid drive gripper can apply high-strength physical locking force without increasing the motor load, ensuring that heavy, rigid materials do not slip or fall off during high-speed handling.

[0103] Once the robotic arm reaches the unloading point, the main controller sends a release command. In hydraulic mode, the main controller prioritizes depressurizing the hydraulic valves. Only after the pressure sensor reading returns to zero does the servo motor reverse to drive the fingers to open. In electric mode, the servo motor directly reverses to reset. This strategy utilizes the high-frequency response of the servo motor for most of the stroke, activating the hydraulic system only for the instantaneous contact with the rigid object, thus reducing heat generation and wear on hydraulic components.

[0104] See attached document Figure 7 , Figure 7 A flowchart illustrating the visual-tactile rheological mapping and adaptive correction logic according to an embodiment of the present invention is shown. The main controller calculates the physical residual by comparing the predicted data from the visual perception module with the physical feedback data from the hybrid drive gripper in actual operation, and corrects the weight parameters of the visual recognition algorithm accordingly, forming a closed-loop learning mechanism.

[0105] The main controller performs the following steps to implement adaptive correction:

[0106] After the hybrid drive gripper performs the grasping action and contacts the material, the main controller does not immediately record the steady-state value. Instead, it records the correspondence between the encoder position change and the current torque change of the servo motor at a high-frequency sampling rate. The main controller calculates the slope of the stress-strain curve during the gripping process to extract the dynamic stiffness characteristics of the material. Once the gripper reaches the steady-state gripping force, the main controller continues to monitor the position creep over a small time interval to characterize the plastic deformation properties of the material.

[0107] Subsequently, during the vertical lifting phase of the robotic arm, the main controller monitors the current component of the vertical axis of the servo drive or the feedback value of the end force sensor, and calculates the actual physical mass of the grasped material by subtracting the unloaded gravity model.

[0108] Finally, the main controller combines the actual gripping thickness, actual physical mass, and steady-state current value to generate a tactile physical feature vector that includes geometric dimensions, mechanical stiffness, plastic rheological parameters, and physical density.

[0109] The main controller calls the predicted feature vector generated from the visual data in the previous steps and aligns and compares it with the tactile physical feature vector. Due to factors such as light reflection, occlusion, or inconsistencies between surface texture and internal material in visual measurements, there will be a deviation between the two. The main controller uses a normalized bias algorithm to calculate the visual-tactile feature residual, which quantifies the degree of distortion in visual perception relative to physical measurement. The calculation formula is as follows:

[0110] ;

[0111] In the formula; This represents the calculated visual-touch feature residuals; This represents the visually estimated thickness obtained from a 3D laser scanning component. This indicates the actual gripping thickness fed back by the gripper encoder; This represents the estimated density derived from a lookup table of visual texture features; The physical density is represented by dividing the physical mass measured above by the equivalent physical volume of the object (this volume is calculated by multiplying the actual grasping thickness by the visual projection area based on the columnar approximation model). This represents the preset normalized weighting coefficient, used to balance the influence weights of geometric errors and physical property errors.

[0112] The main controller compares the calculated visual-touch feature residuals with a preset tolerance threshold. When the residual exceeds the tolerance threshold, it indicates a significant mismatch between the surface texture of the currently grasped material and its physical properties. In this case, the main controller uses the Sigmoid activation function to generate a feature suppression mask for that specific texture ID. This mask is a numerical coefficient between 0 and 1; the larger the residual, the closer the coefficient is to 0, and it is used to suppress the weights of such unreliable visual features in subsequent calculations.

[0113] The main controller feeds back the generated feature suppression mask to the parameter layer of the vision perception module. In subsequent operation cycles, when the vision perception module identifies materials with the same texture ID again, the main controller uses the feature suppression mask to reduce the weight of the grasp confidence index.

[0114] Furthermore, to address the systematic biases present in the texture-material-density lookup table, the main controller employs a weighted moving average algorithm to iteratively update the density mapping value corresponding to the texture ID. The update logic is as follows:

[0115] ;

[0116] In the formula; This represents the updated density mapping value in the material lookup table; This indicates a lookup of an existing density mapping value in the table; This represents the learning rate coefficient, which is set to 0.05 in this embodiment. It is used to control the step size of parameter iteration and avoid system oscillation caused by single measurement errors. This represents the actual physical density measured in this operation.

[0117] Through this physical residual feedback mechanism, the system can automatically correct the judgment logic of material properties under specific working conditions during use, so that the accuracy of the material database gradually converges with the increase of the number of operations, reducing the risk of grasping and dropping or equipment overload caused by visual misjudgment.

[0118] Specific application examples are described below:

[0119] This embodiment is applied to a construction and renovation waste recycling center with an annual processing capacity of 300,000 tons. The sorting objects are mainly demolition mixed materials, including concrete blocks, red bricks, wood, plastic pipes, and textiles.

[0120] Hardware configuration parameters:

[0121] Material conveying module: Belt conveyor with a bandwidth of 1200mm and a rated operating speed of 1.5m / s. Vibrating material distribution device: A linear vibrating feeder driven by dual motors, with a rated power of 2×1.5kW and a frequency converter speed regulation range of 0-60Hz.

[0122] Visual perception module: Employs a 4K linear CMOS industrial camera with a line frequency of 28kHz; coupled with a line laser contour sensor, achieving a Z-axis repeatability of 20µm. The industrial computer is equipped with an NVIDIA RTX 4090 computing card for image inference.

[0123] Multi-stage drive sorting module: Three-axis gantry robot, X-axis travel 3000mm, maximum combined acceleration 15m / s². End-effector hybrid drive gripper with maximum opening stroke of 300mm, servo motor rated torque 2.4Nm, micro hydraulic pump station rated pressure 14MPa, hydraulic cylinder diameter 32mm.

[0124] Central control module: Based on the Beckhoff TwinCAT3 real-time control platform, it controls the servo and IO modules of each axis through the EtherCAT bus, with a communication cycle of 1ms.

[0125] Implementation process overview:

[0126] During system operation, the main controller first calculates the distribution density (window load entropy) of the material on the conveyor belt using a sliding time window algorithm. When it detects that large pieces of concrete are densely packed, causing the load entropy to exceed 0.85, the main controller sends a command to the vibrating material homogenizer to reduce the vibration frequency from 50Hz to 38Hz, thus creating a sparser material flow.

[0127] After the visual system identifies the target, if it determines that it is lightweight wood (estimated density 0.6 g / cm³), 3 The gripper uses a servo motor in torque mode (set current 3A) to grip the material. If the material is determined to be a heavy concrete block (estimated density 2.4g / cm³, estimated mass 15kg), the gripper closes the bypass unloading valve and starts the hydraulic pump the moment it contacts the material, applying a clamping force of 8000N.

[0128] During the operation, the system continuously records the motor current feedback and the actual gripping thickness, and corrects the density parameters in the material database.

[0129] Experimental verification and effect comparison:

[0130] To verify the effectiveness of this solution, a comparative test was conducted on the aforementioned production line for 48 hours. The test material was standard-component mixed construction and renovation waste.

[0131] Experimental group setup:

[0132] Control group (Group A): Traditional fixed-frequency feeding (fixed 50Hz) is used, the gripper uses only a single pneumatic drive (clamping force is not variable), and there is no visual or tactile feedback correction function.

[0133] Experimental Group (Group B): The present invention is adopted, including variable frequency feeding based on load entropy, electro-hydraulic hybrid drive and adaptive correction of visual-thyrorheological parameters.

[0134] Experimental data recording:

[0135] The following data was collected from the on-site SCADA system and high-precision electronic belt scale. The data retains the randomness and fluctuation of the on-site working conditions.

[0136] Table 1: Comparison of System Throughput and Load Stability Test Data

[0137]

[0138] Table 2: Statistics on the sorting accuracy and drop rate of materials of different materials

[0139]

[0140] Table 3: Adaptive Correction Process Data for Visual-Touch Fusion (for a specific texture ID: C&D-Mix-04)

[0141]

[0142] Based on the technical solution proposed in this invention, the specific application embodiments described above, and the test data, the following conclusions are drawn:

[0143] 1. Load balancing significantly improves system throughput stability.

[0144] According to the data in Table 1, after introducing a front-end reverse control mechanism based on window load entropy, the average feeding throughput of the experimental group increased from 39.8 t / h in the control group to 48.1 t / h, an increase of approximately 20.8%. More importantly, the number of times the robotic arm stopped due to overload in the experimental group decreased significantly from 18 times to 1 time, and the idle waiting time percentage decreased from 15.45% to 3.9%. The data shows that by calculating the load entropy in real time and adjusting the frequency of the vibrating material leveling device in reverse, peak shaving and valley filling are effectively achieved, solving the problem of "sometimes congested and overloaded, sometimes idle" caused by traditional fixed-frequency feeding, and ensuring that the system always operates within the optimal efficiency range.

[0145] 2. Electro-hydraulic hybrid graded drive effectively solves the problem of balancing rigid and flexible materials.

[0146] According to the data in Table 2, for heavy, rigid concrete blocks, the drop rate of the experimental group was only 0.8%, far lower than the 14.2% of the control group. This is attributed to the rigid locking force provided by the hydraulic drive mode under high load conditions, overcoming the problem of insufficient gripping force of single pneumatic or electric grippers. Meanwhile, for fragile waste wood, the material breakage rate of the experimental group decreased from 12.3% to 1.4%. This verifies the superiority of the servo motor torque mode (current loop control) in flexible gripping scenarios. Furthermore, the design of the bypass unloading valve ensures low-resistance follow-up in non-hydraulic mode, without increasing the system's response delay.

[0147] 3. The visual-touch fusion closed-loop correction mechanism improves recognition accuracy.

[0148] According to the data in Table 3, as the number of iterations increases, the residual between the visually estimated density and the physically measured density shows a convergence trend (decreasing from 0.23 to 0.02). The corrected database density parameter gradually approaches the physical true value (converging from 1.80 g / cm³ to 2.22 g / cm³), thereby increasing the grasping confidence score from 0.65 to 0.94. This indicates that the system possesses online learning capabilities through the visual-tactile rheological mapping and feedback update logic described in steps S610 to S640. The system can utilize the tactile feedback data (mass, thickness, force feedback) from physical grasping to correct the inherent bias of judging materials solely based on visual texture, which is reflected in Table 2 as a significant decrease in the visual recognition misclassification rate (concrete misclassification rate decreased from 8.5% to 3.1%).

[0149] In summary, this embodiment verifies the technical effectiveness of the intelligent construction waste sorting system based on visual-touch fusion and load balancing in improving operational efficiency, reducing equipment failure rate, and increasing sorting purity. This solution has clear engineering application value.

[0150] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI vision-based intelligent control and sorting system for construction waste, characterized in that: The system includes a material conveying module, a vision sensing module, a multi-stage drive sorting module, and a central control module connected via an industrial fieldbus; the material conveying module is equipped with a vibration equalization device, and the end of the multi-stage drive sorting module is equipped with an electro-hydraulic hybrid drive gripper. The central control module is configured to execute the following control logic: construct a sliding time window based on the data collected by the visual perception module, calculate the window load entropy representing the ratio of the current grasping task density to the theoretical processing capacity of the system; compare the window load entropy with a preset threshold, and generate a frequency adjustment command to reversely adjust the vibration frequency of the vibrating material distribution device, thereby changing the material distribution density; Simultaneously, the driving mode of the electro-hydraulic hybrid drive gripper is switched according to the physical properties of the target material, and visual prediction data and physical gripping feedback data are compared during operation to calculate visual-touch feature residuals in order to correct the weight parameters of the visual recognition algorithm in real time.

2. The intelligent control and sorting system for construction waste based on AI vision according to claim 1, characterized in that, The specific steps by which the central control module calculates the window load entropy include: The mask of the target material is extracted using a semantic segmentation algorithm, and the occlusion dispersion is calculated based on depth data. By combining texture features with related databases to obtain estimated density, the estimated mass of the target material is calculated. Using a nonlinear weighted model, the grasp confidence index of each target material is calculated based on semantic segmentation confidence, occlusion dispersion, and density adaptability factor; The calculation logic of the density adaptability factor adopts a piecewise function: the factor is 1 when the estimated mass is less than or equal to the safe load threshold; the factor decreases linearly when the estimated mass is between the safe load threshold and the rated load limit; and the factor is forced to zero and marked as ungraspable when the estimated mass exceeds the rated load limit. The window load entropy is obtained by summing the products of the grasp confidence index and the recovery value weight of all valid target materials within the sliding time window, and dividing by the theoretical maximum grasp quality throughput of the system after correction by the equipment state decay factor.

3. The intelligent control and sorting system for construction waste based on AI vision according to claim 1, characterized in that, The specific logic of the central main control module in reverse adjusting the vibration uniform material distribution device is as follows: When the window load entropy is higher than the target load threshold, a frequency reduction command is generated to reduce the feeding speed of the vibrating material equalization device; When the window load entropy is lower than the target load threshold, a frequency upsampling command is generated to increase the feeding speed of the vibrating material equalization device; The frequency adjustment command is generated based on a proportional control algorithm, and the calculated target vibration frequency is limited to a preset physical safety range after amplitude limiting processing.

4. The intelligent control and sorting system for construction waste based on AI vision according to claim 1, characterized in that, The central control module is also configured to perform spatiotemporal flux dynamic programming: Based on the combined speed of the material conveying module and the multi-stage drive sorting module, the shortest interception time and remaining dwell time of the target material are calculated, and unreachable targets are eliminated. Calculate the operational efficiency ratio of the reachable target material. This operational efficiency ratio is the product of the grasp confidence index of the target material and the economic value weight, divided by the sum of the robotic arm movement time and the gripper action response time. Within a finite time window, the optimal crawling sequence is generated with the objective function of maximizing total efficiency gains.

5. The intelligent control and sorting system for construction waste based on AI vision according to claim 1, characterized in that, The electro-hydraulic hybrid drive gripper adopts a structure in which a servo motor and a hydraulic cylinder are connected in parallel to drive the same linkage mechanism. The hydraulic circuit of the electro-hydraulic hybrid drive gripper integrates a normally open large-diameter bypass unloading valve. The normally open large-diameter bypass unloading valve is configured to remain open in non-hydraulic drive mode, simultaneously connecting the rodless chamber interface and the rod chamber interface of the hydraulic cylinder to the return oil tank, so that the piston rod of the hydraulic cylinder is in a low-resistance floating state with dual chambers open to the atmosphere.

6. The intelligent control and sorting system for construction waste based on AI vision according to claim 5, characterized in that, The specific logic for the central control module to switch the electro-hydraulic hybrid drive gripper drive mode is as follows: When a flexible material is detected, the electric drive mode is selected, the servo motor is controlled to work in torque mode, and the normally open large-diameter bypass unloading valve is kept open. When a rigid material is detected, the hydraulic drive mode is selected. After the gripper is detected to be in contact with the material, the normally open large-diameter bypass unloading valve is closed and the hydraulic pump station is started. The hydraulic oil is used to push the linkage mechanism to lock. The detection logic for contacting materials is as follows: when the end of the electro-hydraulic hybrid drive gripper enters the spatial enclosure of the target material, and the difference between the real-time measured current of the servo motor and the theoretical current of the no-load dynamic feedforward model exceeds a preset contact threshold, it is determined to be contact.

7. The intelligent control and sorting system for construction waste based on AI vision according to claim 1, characterized in that, The specific steps for the central control module to calculate the visual-touch feature residuals include: During the grabbing and lifting phase, the vertical axis current component of the servo drive or the feedback from the force sensor is monitored, and the actual physical mass is calculated after subtracting the unloaded gravity model. By combining the actual grasping thickness and visual projection area, the equivalent physical volume is calculated based on the columnar approximation model, and then the actual physical density is obtained. The visually estimated thickness and density obtained by the visual perception module are compared with the actual grasping thickness and the actual physical density, respectively, and the normalized deviation is calculated. The weighted sum is then used to obtain the visual-touch feature residual.

8. The AI ​​vision-based intelligent control and sorting system for construction waste according to claim 7, characterized in that, The specific methods for correcting the parameters of the visual recognition algorithm include: When the visual-touch feature residual exceeds the tolerance threshold, a feature suppression mask is generated. This mask is used to reduce the grasp confidence index weight of materials with the same texture identifier in subsequent operations. A weighted moving average algorithm is used to iteratively update the density mapping value corresponding to the texture identifier in the associated database using the actual physical density measured in this task.

9. The intelligent control and sorting system for construction waste based on AI vision according to any one of claims 1 to 8, characterized in that, The visual perception module includes a linear color camera and a 3D laser contour scanner. The trigger end of the visual perception module is connected to an incremental encoder installed on the active roller end of the material conveying module. The spatial registration of the two-dimensional texture image and the three-dimensional depth point cloud data is realized by using a hardware trigger mode.

10. A construction waste sorting device, characterized in that, The device includes the AI ​​vision-based intelligent control and sorting system for construction waste as described in any one of claims 1 to 9.

Images