Intelligent sorting and resourceful treatment system for industrial solid wastes

Abstract

The invention discloses an intelligent sorting and resourceful treatment system for industrial solid wastes, which is characterized in that rich textures of RGB (Red, Green and Blue) images and accurate geometric information of depth point clouds are complementarily fused, and topological relation modeling is carried out on unstructured and seriously deformed solid wastes in combination with a graph convolutional network; and then the decoupled macro and micro physical characteristics are mapped into material rigidity parameters through a multi-physical field network so as to guide grabbing force planning under impedance control. Thus, the recognition success rate under the stacking working condition can be remarkably increased, self-adaptive flexible grabbing can be completed when rigid and flexible mixed materials are treated, object damage caused by overload clamping or sliding caused by dynamic swing is effectively eradicated, and the stability and operation efficiency of production line resourceful treatment are greatly improved.

Description

Technical Field

[0001] This application relates to the field of intelligent sorting, and more specifically, to an intelligent sorting and resource recovery system for industrial solid waste. Background Technology

[0002] With the accelerating pace of global industrialization, the amount of industrial solid waste generated continues to rise, and its composition is becoming increasingly complex. Efficient and precise automated sorting and resource recovery of this waste has become a core approach to achieving a circular economy transformation, alleviating environmental pressure, and improving the secondary utilization rate of raw materials. Constructing intelligent and unmanned sorting solutions can not only significantly reduce the health risks and labor costs associated with manual sorting, but is also an inevitable requirement for upgrading traditional environmental engineering towards digital and refined industrial manufacturing.

[0003] Existing intelligent sorting systems typically rely on vision or depth sensors for object recognition and localization, achieving a certain degree of grasping control based on predefined rules. However, in complex industrial scenarios, especially when dealing with heavily stacked and deformed industrial solid waste, the instance segmentation accuracy of existing solutions drops significantly, making it difficult to accurately extract the complete contour and spatial position of target objects, leading to subsequent grasping failures or damage to the objects. This problem severely restricts the reliability and adaptability of the system on real production lines, making it difficult for existing methods to achieve stable, compliant, and adaptive grasping operations in solid waste flows that are a mixture of rigid and flexible materials with varying shapes.

[0004] Therefore, there is a need for an optimized intelligent sorting and resource recovery system for industrial solid waste. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an intelligent sorting and resource recovery system for industrial solid waste.

[0006] According to one aspect of this application, an intelligent sorting and resource utilization system for industrial solid waste is provided, comprising: The multimodal perception and data preprocessing module is used to perform spatiotemporal registration, region of interest extraction, and macro enhancement on the raw image stream and depth point cloud stream of waste collected from industrial conveyor belts to obtain a target object dataset containing aligned image patches, depth patches, and 3D position information. The physical feature decoupling and extraction module is used to decouple and extract the surface micro-features and macro-geometric features of the target object dataset to obtain physical property feature vectors. The material property stiffness prediction module is used to perform stiffness prediction on the physical property feature vector based on a multiphysics mapping network to obtain the material property parameter set. The gripping force planning module is used to estimate the mass of the object based on the target object dataset and combine it with the material property parameter set to plan the gripping force in order to obtain impedance control configuration information. The compliant gripping dynamic execution module is used to set parameters for the robotic arm controller using impedance control configuration information to drive the robotic arm to move to the three-dimensional position indicated by the target object dataset. During the execution of the gripping closing action, the motor output torque is dynamically adjusted according to the real-time position deviation and impedance control configuration information to complete adaptive compliant gripping.

[0007] Compared with existing technologies, this application provides an intelligent sorting and resource recovery system for industrial solid waste. It utilizes the rich texture of RGB images and the precise geometric information of depth point clouds for complementary fusion, and combines this with graph convolutional networks to model the topological relationships of unstructured and severely deformed solid waste. Furthermore, it maps the decoupled macro- and micro-physical features into material stiffness parameters through a multiphysics network to guide the planning of gripping forces under impedance control. This not only significantly improves the recognition success rate under stacked conditions but also enables adaptive and compliant gripping when handling mixed rigid and flexible materials. It effectively prevents damage to objects caused by clamping overload or slippage due to dynamic swaying, greatly enhancing the stability and operational efficiency of the production line's resource recovery process. Attached Figure Description

[0008] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 This is a block diagram of an intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application; Figure 2 This is a schematic diagram of the data flow of an intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application; Figure 3 This is a block diagram of the physical feature decoupling and extraction module in the intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application; Figure 4 This is a block diagram of the grasping force planning module in the intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application. Detailed Implementation

[0010] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0011] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0012] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0013] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0014] The technical solution of this application proposes an intelligent sorting and resource utilization system for industrial solid waste. Figure 1 This is a block diagram of an intelligent sorting and resource recovery system for industrial solid waste according to an embodiment of this application. Figure 2 This is a schematic diagram of the data flow in an intelligent sorting and resource recovery system for industrial solid waste according to an embodiment of this application. Figure 1 and Figure 2As shown, the industrial solid waste intelligent sorting and resource utilization system 300 according to an embodiment of this application includes: a multimodal perception and data preprocessing module 310, used to perform spatiotemporal registration, region of interest extraction, and macro enhancement on the original image stream and depth point cloud stream of waste collected from an industrial conveyor belt to obtain a target object dataset containing aligned image patches, depth patches, and three-dimensional position information; a physical feature decoupling extraction module 320, used to decouple and extract surface micro-features and macro-geometric features from the target object dataset to obtain physical property feature vectors; and a material property stiffness prediction module 330, used to predict the material properties stiffness of the target object. The material property feature vectors are used to perform stiffness prediction based on a multiphysics mapping network to obtain a set of material property parameters; the grasping force planning module 340 is used to estimate the mass of the object based on the target object dataset and combine the material property parameter set to perform grasping force planning to obtain impedance control configuration information; the compliant grasping dynamic execution module 350 is used to set parameters for the robotic arm controller using the impedance control configuration information to drive the robotic arm to move to the three-dimensional position indicated by the target object dataset, and dynamically adjust the motor output torque according to the real-time position deviation and impedance control configuration information during the execution of the grasping closing action to complete adaptive compliant grasping.

[0015] Specifically, the multimodal perception and data preprocessing module 310 is used to perform spatiotemporal registration, region of interest extraction, and macro enhancement on the raw image stream and depth point cloud stream of waste collected from the industrial conveyor belt to obtain a target object dataset containing aligned image patches, depth patches, and 3D position information. It should be understood that in the complex environment of industrial solid waste sorting, the waste on the conveyor belt is of various types, heavily stacked, and extremely irregular in shape. The raw image stream only contains texture information and is highly susceptible to interference from lighting and shadows; while the raw depth point cloud stream provides spatial geometric information, it often suffers from pixel-level misalignment with the image and inherent noise. To provide high-precision input for subsequent feature decoupling extraction and material stiffness prediction, a structured target object dataset is constructed in the technical solution of this application. Specifically, spatiotemporal registration eliminates pose deviations between sensors, filtering out invalid background interference by extracting regions of interest, and then using macro enhancement to highlight microscopic texture details. This ensures that the system can obtain consistent three-dimensional position information and enhanced image features under conditions of heavy stacking occlusion and severe deformation, laying a reliable perceptual foundation for achieving high-precision intelligent sorting.

[0016] In practice, firstly, based on system calibration parameters, the pixels in the depth frame are projected onto the coordinate system of the original image frame to obtain full-frame aligned RGB-D data. This step aims to achieve spatial alignment between the RGB color image and the depth point cloud data at the pixel level, eliminating parallax problems caused by differences in the physical positions of the camera and depth sensor. Specifically, using system calibration parameters obtained beforehand through a calibration board, these parameters mainly include the intrinsic and extrinsic matrices (rotation and translation matrices) of the color and depth cameras, as well as lens distortion coefficients. For each pixel in the depth frame, its two-dimensional coordinates and depth value are known. Through a coordinate transformation model, its three-dimensional coordinates are transformed to the coordinate system of the color camera and projected onto the pixel plane of the color image. In this process, the coordinate transformation model transforms the three-dimensional point coordinates in the depth camera coordinate system to the color camera coordinate system through the extrinsic matrix, and then projects them to its pixel coordinate system through a series of matrix operations. Finally, by performing the above transformations and interpolations (such as bilinear interpolation) on all depth pixels, full-frame aligned RGB-D data that is spatially fully registered with the original RGB image is generated.

[0017] Next, depth-threshold-based background culling and instance segmentation are performed on the full-frame aligned RGB-D data to obtain the original list of regions of interest. This step aims to separate the waste targets to be sorted from the complex scene and remove background interference such as conveyor belts and supports. Specifically, a depth threshold range is set, which corresponds to a specific workspace height above the conveyor belt surface. For each pixel in the full-frame aligned RGB-D data, it is determined whether its depth value is within the preset depth threshold range. Pixels that meet the condition are retained as foreground, while the rest are considered background and culled. Based on the depth threshold processing, an instance segmentation algorithm (e.g., based on connected component analysis or deep learning models such as Mask R-CNN) is further used to cluster and segment the foreground region to identify individual waste instances. Each instance is represented by its pixel mask, and its bounding box is extracted. All this segmented instance information (including masks and bounding boxes) is organized into a list, i.e., the original list of regions of interest.

[0018] Furthermore, texture detail enhancement processing is performed on the image patches in the original region of interest list, and the enhanced image patches, corresponding depth patches, and 3D centroid coordinates are structurally encapsulated to obtain the target object dataset. This step aims to improve the clarity of the micro-texture of the image patches, facilitating subsequent feature extraction, and effectively organizing all relevant information. Specifically, first, the original region of interest list is traversed. For each instance in the list, the corresponding image patch is cropped from the full-frame aligned RGB image based on its bounding box information. Second, texture detail enhancement processing is performed on this RGB image patch, for example, using the Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm or a deep learning-based super-resolution reconstruction method to enhance the contrast and clarity of its surface micro-texture. Simultaneously, based on the same bounding box, the corresponding depth patch is cropped from the full-frame aligned depth data. Then, based on the depth patch and pixel mask of this instance, its centroid coordinates in 3D space are calculated. In a specific example of this application, the centroid coordinates in 3D space can be obtained by summing and averaging the 3D coordinates corresponding to all valid pixels within the instance mask. Finally, the three key information elements—the image patch after texture detail enhancement, the corresponding depth patch, and the calculated 3D centroid coordinates—are bound and encapsulated using data structures (such as dictionaries or custom class instances in Python) to form a complete target object data entry. The collection of data entries from all instances constitutes the final target object dataset.

[0019] Specifically, the physical feature decoupling extraction module 320 is used to decouple and extract surface micro-features and macro-geometric features from the target object dataset to obtain physical attribute feature vectors. It should be understood that during the sorting process of industrial solid waste, materials are often in a complex, unsteady state, such as severe compression deformation or heavy stacking. Since the physical properties of an object, such as stiffness and mass, are deeply correlated with its macro-geometric contours (e.g., overall curvature) and surface micro-textures (e.g., wrinkle degree, roughness), mixing these features makes it difficult to achieve accurate mapping of material properties. Therefore, in the technical solution of this application, by decoupling and extracting macro-geometric features and micro-surface features, solid waste is profiled from different spatial scales. This allows the system to effectively distinguish objects such as wads of waste paper and thin metal sheets, which may be similar in geometry but differ greatly in micro-texture and physical stiffness, providing key feature criteria for ultimately achieving adaptive compliant grasping.

[0020] Figure 3 This is a block diagram of the physical feature decoupling and extraction module in an intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application. Figure 3As shown, the physical feature decoupling extraction module 320 includes: a macroscopic geometric curvature feature extraction unit 321, used to extract macroscopic geometric curvature features based on depth gradient from aligned depth patches in the target object dataset to obtain macroscopic geometric curvature features; a microscopic texture wrinkle feature extraction unit 322, used to extract enhanced image patches from the target object dataset and convert them into grayscale images, and extract microscopic texture wrinkle features based on frequency domain filtering from the grayscale images to obtain microscopic surface wrinkle features; and a physical attribute feature unit 323, used to perform physical attribute feature extraction on the macroscopic geometric curvature features and microscopic surface wrinkle features to obtain a physical attribute feature vector.

[0021] Specifically, the macroscopic geometric curvature feature extraction unit 321 is used to extract macroscopic geometric curvature features based on depth gradients from aligned depth patches in the target object dataset to obtain macroscopic geometric curvature features. That is, it utilizes the three-dimensional geometric information contained in the depth patches to extract curvature features reflecting changes in the object's macroscopic shape. These features are directly related to the object's physical properties such as stiffness and compressive strength. Specifically, it uses discrete difference operators to solve for the first-order gradient and second-order partial derivatives in the depth map space and constructs a curvature calculation model. For coordinates in the depth map... The pixel, whose surface has average curvature It is used to characterize macroscopic geometric features, and this process can be expressed by the formula: in, It represents the average curvature, used to describe the direction and intensity of the evolution of the macroscopic geometry of an object's surface at a specific point; This indicates the depth map in the horizontal direction ( The first-order partial derivative of the depth value (direction) reflects the rate of change of the depth value along the horizontal direction; This indicates the depth map in the vertical direction ( The first-order partial derivative of the depth value (direction) reflects the rate of change of the depth value along the vertical direction; The second-order partial derivative of the depth map in the horizontal direction reflects the rate of change of the horizontal gradient and is used to describe the unevenness of the surface. This represents the second-order partial derivative of the depth map in the vertical direction, reflecting the rate of change of the vertical gradient; The second-order mixed partial derivative of the depth map describes the degree of surface distortion along the diagonal direction. This formula quantitatively calculates the curvature of an object's surface by combining the first and second derivative information of the depth map. In the technical solution of this application, this formula is used to process aligned depth map patches to obtain macroscopic geometric curvature features reflecting the macroscopic contour information of solid waste.

[0022] Specifically, the micro-texture wrinkle feature extraction unit 322 is used to extract enhanced image patches from the target object dataset and convert them into grayscale images. The grayscale images are then subjected to frequency domain filtering-based micro-texture wrinkle feature extraction to obtain micro-surface wrinkle features. This step aims to separate features reflecting surface micro-textures (such as wrinkles and unevenness) from the macro-enhanced image patches. These features are closely related to physical properties such as the friction coefficient and surface roughness of the object. Specifically, firstly, a two-dimensional fast Fourier transform is performed on the grayscale image, transforming it from the spatial domain to the frequency domain, to utilize the spatial repeatability and frequency of change of the image signal to identify detailed information. Secondly, in the frequency domain, a high-pass filter is applied to filter out low-frequency components representing smooth backgrounds or illumination changes, retaining high-frequency components representing micro-textures and wrinkles. Finally, an inverse Fourier transform is performed on the filtered frequency domain image, transforming it back to the spatial domain to obtain an enhanced micro-texture image, from which feature vectors are extracted to obtain micro-surface wrinkle features.

[0023] Specifically, the physical attribute feature unit 323 is used to extract physical attribute features from macroscopic geometric curvature features and microscopic surface wrinkle features to obtain a physical attribute feature vector. This step aims to fuse the separately extracted macroscopic geometric features and microscopic texture features to form a unified physical attribute feature representation. Specifically, the extracted macroscopic geometric curvature features and microscopic surface wrinkle features are dimensionally concatenated and fused, that is, through the concatenation operation of feature vectors, features of different dimensions and scales are mapped to a unified semantic vector space, ultimately constructing a complete physical attribute feature vector.

[0024] Specifically, the material property stiffness prediction module 330 is used to perform stiffness prediction on the physical property feature vector based on a multiphysics mapping network to obtain a set of material property parameters. It should be understood that industrial solid waste sorting lines face extremely complex physical scenarios. Traditional single-vision recognition schemes can only obtain the shape coordinates of objects but cannot predict their physical feedback. When handling materials with a mixture of rigidity and flexibility, such as waste paper balls, woven bags, and aluminum cans, due to the uncertainty of the material properties of the objects, robotic arms often adopt a uniform motion logic. This can lead to control decoupling between the applied clamping force and the effective friction coefficient because the object's stiffness is ignored. Therefore, in the technical solution of this application, stiffness prediction based on a multiphysics mapping network is used to transform the macroscopic geometric abrupt changes and microscopic surface wrinkles observed visually into intrinsic mechanical parameters, thereby identifying the object's hardness (stiffness) and surface slip characteristics (friction coefficient). This provides core data support for subsequently avoiding plastic collapse of the object structure, utilizing the frictional gain effect of soft objects, and establishing a dynamic closed-loop model of clamping force and deformation.

[0025] In practice, the physical attribute feature vector is first input into the fully connected backbone layer of the visual stiffness mapping neural network model to obtain a high-dimensional latent semantic feature vector. This step aims to perform high-level feature abstraction and dimensionality transformation on the input physical attribute feature vector to extract the deep semantic information most relevant to the physical attribute prediction task. Specifically, the physical attribute feature vector is fed into a pre-trained visual stiffness mapping neural network model. The core of this model is a fully connected backbone layer, typically composed of multiple cascaded fully connected layers, each possibly followed by an activation function (such as ReLU) and a normalization layer (such as BatchNorm). During this process, the physical attribute feature vector undergoes layer-by-layer nonlinear transformations within this backbone layer, ultimately yielding a higher-dimensional or more refined latent semantic feature vector at the output of the backbone layer. This vector encodes deep patterns in the input features that are highly correlated with physical attributes such as stiffness and friction.

[0026] Next, the latent semantic feature vector is input into the stiffness prediction branch and the friction prediction branch respectively to obtain the normalized stiffness coefficient estimate and the surface friction coefficient estimate. That is, by utilizing the common features extracted from the shared backbone network, regression predictions of two key physical properties are performed simultaneously, achieving multi-task learning and improving the model's efficiency and accuracy. Specifically, the high-dimensional latent semantic feature vector obtained in the previous step is used as the input to two independent branches. The stiffness prediction branch typically consists of one or more fully connected layers, with its last layer using activation functions such as the Sigmoid function to restrict the output to the range of 0 to 1, producing a normalized stiffness coefficient estimate. Similarly, the friction prediction branch also consists of a similar network structure, and its output layer may use the Sigmoid function or a linear activation function (if the range of the friction coefficient is known and finite) to produce a surface friction coefficient estimate. The stiffness prediction branch and the friction prediction branch perform matrix multiplication with the latent semantic feature vector through their respective network weight matrices, and after passing through the activation function, are finally mapped to a scalar output value, namely the normalized stiffness coefficient estimate and the surface friction coefficient estimate.

[0027] Furthermore, the normalized stiffness coefficient estimate and the surface friction coefficient estimate are structurally combined to obtain the material property parameter set. Specifically, the normalized stiffness coefficient estimate output from the stiffness prediction branch and the surface friction coefficient estimate output from the friction prediction branch are encapsulated using data structures (e.g., dictionaries, named tuples, or custom data classes in Python). This structured combination constitutes the material property parameter set, which, as a whole, contains the key physical parameters required for gripping force planning.

[0028] Specifically, the grasping force planning module 340 is used to estimate the mass of the object based on the target object dataset and combine it with the material property parameter set to plan the grasping force to obtain impedance control configuration information. It should be understood that industrial solid waste streams contain a large amount of materials with varying stiffness and density, exhibiting a mixture of rigid and flexible components. Traditional grasping methods often employ fixed clamping forces or simple static linear safety factors. This approach ignores the nonlinear coupling relationship between the applied clamping force and the effective friction coefficient established through the object's stiffness, easily leading to excessive pressure on low-stiffness or semi-rigid objects (such as wads of waste paper or woven bags), causing structural damage, or causing material slippage due to neglecting acceleration compensation during high-speed lifting. Through this step, the system can dynamically calculate the optimal balanced force based on the object's three-dimensional dimensions and predicted material stiffness, preventing slippage and avoiding crushing of the object, and convert this force into impedance control commands executable by the robotic arm's bottom layer, thereby achieving an organic closed loop of perception and control.

[0029] Figure 4 This is a block diagram of the grasping force planning module in an intelligent sorting and resource utilization system for industrial solid waste according to an embodiment of this application. Figure 4 As shown, the gripping force planning module 340 includes: an object mass estimation unit 341, used to estimate the object mass based on volume and density from the depth tiles in the target object dataset and the normalized stiffness coefficient estimates in the material attribute parameter set to obtain the object's physical state parameters; a target clamping force command determination unit 342, used to determine the target clamping force command to prevent slippage based on the object's physical state parameters and the material attribute parameter set; and a variable impedance parameter dynamic mapping unit 343, used to perform variable impedance parameter dynamic mapping based on stiffness prediction values ​​on the target clamping force command based on the normalized stiffness coefficient estimates in the material attribute parameter set to obtain impedance control configuration information.

[0030] Specifically, the object mass estimation unit 341 is used to estimate the object mass based on volume and density from the depth tiles and normalized stiffness coefficient estimates in the material attribute parameter set of the target object dataset to obtain the object's physical state parameters. Specifically, the system extracts the object's 3D point cloud information from the target object dataset and estimates the object's volume by calculating the space occupied by the point cloud. Simultaneously, the system uses the normalized stiffness coefficient estimates obtained from the material attribute prediction network, combined with a preset material density comparison relationship, to infer the object's approximate density. Finally, by calculating the product of volume and density, the system obtains the object's physical state parameters, which include information such as the object's mass and volume, providing a foundation for subsequent mechanical calculations.

[0031] Specifically, the target clamping force command determination unit 342 is used to determine the target clamping force command to prevent slippage based on the object's physical state parameters and material property parameter set. Specifically, some attempts simplify the gripping force calculation to the product of the object's weight and a fixed safety factor based on a static linear assumption. This approach essentially ignores the nonlinear coupling relationship between the applied clamping force and the effective friction coefficient established through the object's inherent stiffness. However, in actual working conditions, when handling low-stiffness or semi-rigid solid waste objects (such as wads of waste paper, woven bags, and aluminum cans), applying a larger clamping force significantly increases the microscopic and macroscopic contact area of ​​the object's surface (i.e., evolving from point contact to surface contact), thereby nonlinearly increasing the effective friction coefficient. This is known as the friction gain effect. Therefore, this approach treats the friction coefficient as a constant and fails to utilize the gain brought by this deformation. Consequently, when facing low-stiffness materials such as wads of waste paper or thin-walled aluminum cans, the system cannot effectively utilize the friction gain effect generated by the increased contact area due to force deformation. Furthermore, due to the lack of explicit structural yield boundary constraints, relying solely on empirical safety factors cannot avoid physical critical points. For semi-rigid thin-walled objects such as aluminum cans, once the clamping force exceeds its structural collapse threshold, the object will instantly undergo plastic collapse, leading to complete failure of the gripping mechanism. Therefore, this method fails to establish a dynamic closed-loop mathematical model encompassing clamping force, deformation, friction, and structural strength, and cannot effectively prevent physical damage to the object's structure while ensuring gripping stability.

[0032] Therefore, in the technical solution of this application, an improved solution is proposed, which aims to use the stiffness of an object to predict the change of its contact area under force, and then solve for the optimal balance force that can both ensure that the object does not slip and avoid the collapse of the object structure.

[0033] In this process, firstly, based on the normalized stiffness coefficient estimates from the object's volume and material property parameters (physical state parameters), mechanical constraint parameters are determined. This step aims to set a physical upper limit for the gripping force to prevent crushing the object, while simultaneously quantifying the improvement in frictional performance of soft objects after being subjected to force. Specifically, using the input object volume... and predicted stiffness Based on the statistical laws of materials mechanics, the maximum compressive force that an object can withstand before irreversible failure is estimated. Simultaneously, define the sensitivity factor. To describe the rate at which the effective coefficient of friction changes with pressure (the lower the stiffness, the higher the sensitivity), this process can be expressed by the formula: in, The yield strength threshold of the object structure. The reference stress constant is For the volume of the object, The normalized stiffness coefficient, The friction gain sensitivity factor, , These are the empirical fitting coefficients. This step is the first to introduce crush prevention as an explicit mathematical constraint into the control loop, and through... The factor gives the system the ability to recognize the physical phenomenon that soft objects become tighter and less likely to slip when gripped, thus providing boundary conditions and gain models for subsequent optimization and ensuring that the system does not blindly increase the clamping force and damage the object.

[0034] Secondly, based on the normalized stiffness coefficient estimates from the material property parameter set and the object mass from the object's physical state parameters, a viscoelastic oscillation factor is introduced to perform nonlinear correction calculations on the inertial force generated solely by gravity and lifting acceleration to obtain the dynamic load requirements. Since soft solid waste is not only affected by gravity during high-speed lifting, but also experiences whiplash effects or hysteresis deformation due to its own flexibility, generating additional peeling torques, a viscoelastic oscillation factor is further introduced to perform nonlinear correction calculations on the inertial force generated solely by gravity and lifting acceleration. Specifically, a viscoelastic oscillation factor is introduced. To correct the inertial force calculated solely from gravity and acceleration, the lower the stiffness, the easier it is for the object to swing. Therefore, the minimum tangential holding force required to maintain the grip is calculated. This will also increase accordingly, and the process can be expressed by the formula: in, To minimize tangential holding force, To estimate the quality, It is the acceleration due to gravity. To maximize acceleration, It is the viscoelastic oscillation factor. The gain coefficient for swing amplitude. This is the stiffness attenuation constant. This step accurately characterizes the mechanical behavior of flexible objects during dynamic motion, avoiding slippage accidents caused by neglecting dynamic oscillations. Ultimately, it provides the system with a load requirement value that more closely matches actual working conditions, ensuring the reliability of the grasping process.

[0035] Furthermore, based on the estimated surface friction coefficient from the mechanical constraint parameters, dynamic load requirements, and material property parameter set, the target clamping force command is determined. This step aims to find the optimal solution using the previously established gain model. Specifically, a deformation-enhanced friction model is constructed, i.e., the effective friction coefficient... It is a function of the applied force F To prevent slippage, the following conditions must be met. This is a nonlinear equation with respect to F, obtained by solving for the smallest positive real root of the equation and relating it to the yield threshold. By performing a truncation comparison, the final target clamping force is obtained. This process can be expressed by the following formula: in, For the ultimate goal of clamping force, Based on the coefficient of friction, The dimensionless loading factor The operation aims to minimize the value. Ultimately, it achieves dual optimization of energy consumption and stability, ensuring a high success rate in grasping while minimizing the risk of physical damage to objects, thus solving the adaptability problem of traditional linear calculation methods in rigid-flexible hybrid scenarios.

[0036] Through the aforementioned improvement mechanism, this technical solution achieves significant technical effects. Firstly, adaptive safety is greatly enhanced by explicitly introducing a structural yield threshold. The system possesses physical awareness when dealing with fragile or semi-rigid objects, effectively preventing crushing caused by clamping overload. Secondly, energy efficiency and equipment lifespan are optimized. Utilizing a deformation-friction coupling model, the algorithm intelligently identifies the frictional gain characteristics of soft objects, avoiding the blind application of excessive redundant clamping forces, thereby reducing motor energy consumption and minimizing wear on the mechanical end effector. Finally, dynamic stability is significantly enhanced by introducing a viscoelastic oscillation factor. The solution addresses the problem of flexible solid waste slipping and falling during high-speed transportation, ensuring reliable grasping even at full production speed.

[0037] Specifically, the variable impedance parameter dynamic mapping unit 343 is used to dynamically map the target clamping force command based on the normalized stiffness coefficient estimate from the material property parameter set, thereby obtaining impedance control configuration information. Specifically, the calculated target clamping force and the predicted object stiffness are mapped to a set of parameters for the robotic arm impedance controller, typically including virtual stiffness, virtual damping, and desired force. For soft objects (low stiffness coefficient), the system configures lower virtual stiffness and higher damping to make the end effector more compliant, absorbing energy during contact and avoiding excessive impact; for hard objects (high stiffness coefficient), higher virtual stiffness is configured to ensure positioning accuracy and stability during gripping. The resulting impedance control configuration information serves as a bridge connecting high-level planning and low-level actuators.

[0038] Specifically, the compliant gripping dynamic execution module 350 is used to set parameters for the robotic arm controller using impedance control configuration information to drive the robotic arm to move to the three-dimensional position indicated by the target object dataset. During the gripping closing action, it dynamically adjusts the motor output torque based on real-time position deviation and impedance control configuration information to achieve adaptive compliant gripping. It should be understood that industrial solid waste exhibits extremely high diversity, with significant differences in material, shape, and physical stiffness. Traditional rigid gripping often generates excessive clamping force when dealing with objects such as wads of waste paper, woven bags, or semi-rigid aluminum cans due to fixed position control, leading to object damage or equipment overload; or the inability to sense contact force causes objects to slip during high-speed movement. Through this step, the system can establish a dynamic relationship between the force and displacement at the end of the robotic arm, endowing the robotic arm with force characteristics similar to a spring-damped system through impedance control, thereby achieving automatic torque compensation and adaptive adjustment at the moment of gripping, ensuring the efficiency and safety of the sorting process.

[0039] In practice, firstly, inverse kinematics calculations are performed based on the 3D centroid coordinates of the target object dataset to determine the target joint angles. Then, parameters from the impedance control configuration information are written into the robotic arm's underlying servo control registers to initialize the impedance control mode. Specifically, the system reads the 3D centroid coordinates of the object to be grasped from the target object dataset. These coordinates define the target Cartesian space position that the robotic arm's end effector needs to move to. Subsequently, using an inverse kinematics algorithm, this target position and orientation (usually preset to be perpendicular to the conveyor belt) are converted into a set of target angle values ​​that each joint of the robotic arm needs to achieve. Simultaneously, the system writes the impedance control configuration information (usually including parameters such as virtual stiffness, virtual damping, and desired force) output by the grasping force planning module into specific control registers of the robotic arm joint motor servo drivers. This write operation switches the controller's operating mode from standard position control to impedance control mode, preparing the underlying hardware for subsequent compliant interaction.

[0040] Next, the current joint states and end effector Cartesian states of the robotic arm are acquired in the real-time control loop. The Jacobian matrix of the robotic arm is calculated, and the required driving torque for each joint is calculated based on the impedance control configuration information and the deviation between the current state and the desired state. Specifically, within each control cycle, the system reads the current joint angles and joint velocities of the robotic arm through encoders. Using the forward kinematics model, the actual position of the end effector in Cartesian space can be calculated based on the current joint angle. Simultaneously, the Jacobian matrix of the robotic arm in the current configuration is calculated, which establishes the mapping relationship between joint space velocity and Cartesian space velocity. Then, the system calculates the position deviation (i.e., the difference between the current position and the desired position) and velocity deviation (i.e., the difference between the current velocity and the desired velocity, the desired velocity is usually zero during the positioning phase) of the end effector. In Cartesian space, according to the impedance model, the desired end effector force is determined by the deviation and impedance parameters. Then, using the transpose of the Jacobian matrix, the force in Cartesian space is mapped to joint space, and the equivalent torque required by each joint is calculated. This calculation process ensures that the robotic arm responds to contact with the environment: when a positional deviation occurs (such as deformation of a contact object), a corresponding compensating force is generated.

[0041] Then, the driving torque is sent to the servo driver to execute the compliant gripping action. Specifically, the system sends the torque command for each joint calculated in the previous step to the corresponding joint servo driver via a real-time communication bus (such as EtherCAT). After receiving the torque command, the driver directly controls the motor to output the corresponding current (torque), driving the robotic arm to move. Throughout the entire gripping and closing action, from the moment the end effector contacts the object to the moment it fully closes and grips the object, the calculation of the driving torque is continuously running in every control cycle. Therefore, when the gripper contacts the object and encounters resistance, the resulting actual positional deviation is immediately detected, and the output torque is dynamically adjusted accordingly. This allows the gripper to compliantly adapt to the shape of the object and apply the planned gripping force, rather than rigidly impacting the object or executing a rigid trajectory. In this way, the entire adaptive compliant gripping process is ultimately achieved.

[0042] As described above, the intelligent sorting and resource utilization system 300 for industrial solid waste according to the embodiments of this application can be implemented in various wireless terminals, such as servers with intelligent sorting and resource utilization algorithms for industrial solid waste. In one possible implementation, the intelligent sorting and resource utilization system 300 for industrial solid waste according to the embodiments of this application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the intelligent sorting and resource utilization system 300 for industrial solid waste can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the intelligent sorting and resource utilization system 300 for industrial solid waste can also be one of many hardware modules of the wireless terminal.

[0043] Alternatively, in another example, the intelligent sorting and resource utilization system 300 for industrial solid waste and the wireless terminal can also be separate devices, and the intelligent sorting and resource utilization system 300 for industrial solid waste can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0044] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An intelligent sorting and resource utilization system for industrial solid waste, characterized in that, include: The multimodal perception and data preprocessing module is used to perform spatiotemporal registration, region of interest extraction, and macro enhancement on the raw image stream and depth point cloud stream of waste collected from industrial conveyor belts to obtain a target object dataset containing aligned image patches, depth patches, and 3D position information. The physical feature decoupling and extraction module is used to decouple and extract the surface micro-features and macro-geometric features of the target object dataset to obtain physical property feature vectors. The material property stiffness prediction module is used to perform stiffness prediction on the physical property feature vector based on a multiphysics mapping network to obtain the material property parameter set. The gripping force planning module is used to estimate the mass of the object based on the target object dataset and combine it with the material property parameter set to plan the gripping force in order to obtain impedance control configuration information. The compliant gripping dynamic execution module is used to set parameters for the robotic arm controller using impedance control configuration information to drive the robotic arm to move to the three-dimensional position indicated by the target object dataset. During the execution of the gripping closing action, the motor output torque is dynamically adjusted according to the real-time position deviation and impedance control configuration information to complete adaptive compliant gripping.

2. The intelligent sorting and resource utilization system for industrial solid waste according to claim 1, characterized in that, The multimodal sensing and data preprocessing module is used for: Based on the system calibration parameters, the pixels in the depth frame are projected onto the coordinate system of the original image frame to obtain full-frame aligned RGB-D data; Perform depth-threshold-based background culling and instance segmentation on the full-frame aligned RGB-D data to obtain the original list of regions of interest; The image patches in the original region of interest list are subjected to texture detail enhancement processing, and the enhanced image patches, corresponding depth patches, and 3D centroid coordinates are structured and encapsulated to obtain the target object dataset.

3. The intelligent sorting and resource utilization system for industrial solid waste according to claim 1, characterized in that, The physical feature decoupling and extraction module includes: The macroscopic geometric curvature feature extraction unit is used to extract macroscopic geometric curvature features based on depth gradient from the aligned depth tiles in the target object dataset to obtain macroscopic geometric curvature features. The micro-texture wrinkle feature extraction unit is used to extract enhanced image patches from the target object dataset and convert them into grayscale images. The grayscale images are then subjected to frequency domain filtering-based micro-texture wrinkle feature extraction to obtain micro-surface wrinkle features. The physical property feature unit is used to perform physical property feature analysis on macroscopic geometric curvature features and microscopic surface wrinkle features to obtain physical property feature vectors.

4. The intelligent sorting and resource utilization system for industrial solid waste according to claim 1, characterized in that, The material property stiffness prediction module is used for: The physical property feature vectors are input into the fully connected backbone layer of the visual stiffness mapping neural network model to obtain high-dimensional latent semantic feature vectors. The latent semantic feature vectors are input into the stiffness prediction branch and the friction prediction branch respectively to obtain the normalized stiffness coefficient estimate and the surface friction coefficient estimate. The normalized stiffness coefficient estimate and the surface friction coefficient estimate are structurally combined to obtain the material property parameter set.

5. The intelligent sorting and resource utilization system for industrial solid waste according to claim 1, characterized in that, The grasping force planning module includes: The object mass estimation unit is used to estimate the object mass based on volume and density from the normalized stiffness coefficient estimates in the depth tiles and material property parameter sets in the target object dataset to obtain the object's physical state parameters. The target clamping force command determination unit is used to determine the target clamping force command to prevent slippage based on the object's physical state parameters and material property parameter set. The variable impedance parameter dynamic mapping unit is used to perform variable impedance parameter dynamic mapping based on the normalized stiffness coefficient estimate in the material property parameter set to obtain impedance control configuration information by performing variable impedance parameter dynamic mapping on the target clamping force command based on the stiffness prediction value.

6. The intelligent sorting and resource utilization system for industrial solid waste according to claim 1, characterized in that, The compliant capture dynamic execution module is used for: Inverse kinematics calculations are performed based on the three-dimensional centroid coordinates in the target object dataset to determine the target joint angles, and the parameters in the impedance control configuration information are written into the underlying servo control register of the robotic arm to initialize the impedance control mode. The current joint state and end effector Cartesian state of the robotic arm are collected in the real-time control loop. The Jacobian matrix of the robotic arm is calculated. Based on the impedance control configuration information and the deviation between the current state and the desired state, the required driving torque of each joint is calculated. The driving torque is sent to the servo driver to perform a compliant gripping motion.

7. The intelligent sorting and resource utilization system for industrial solid waste according to claim 5, characterized in that, The target clamping force command determination unit is used for: Based on the normalized stiffness coefficient estimates from the set of object volume and material property parameters in the object's physical state parameters, the mechanical constraint parameters are determined. Based on the normalized stiffness coefficient estimate in the material property parameter set and the mass of the object in the physical state parameters, a viscoelastic oscillation factor is introduced to perform nonlinear correction calculation on the inertial force generated solely by gravity and lifting acceleration in order to obtain the dynamic load requirement. The target clamping force command is determined based on the estimated surface friction coefficient values ​​from the mechanical constraint parameters, dynamic load requirements, and material property parameter sets.

Images