Mechanical automatic material grabbing equipment and method

Abstract

The invention provides a mechanical automatic material grabbing device and method, and the method comprises the steps: screening out a plurality of finger tip adjustment parameters of a manipulator end effector through a grabbing space of a target material, and converting all the finger tip adjustment parameters into compliance control parameters of the manipulator end effector in a closing process; based on the contact signal, determining a trend vector when the target material slides relative to the finger tip surface of the end effector of the manipulator, and determining the stability of the end effector of the manipulator in the sliding state of the target material according to the trend vector; dynamically predicting the closed path of the grabbing process through the compliance control parameters and the stability to obtain the grabbing stability deviation of the manipulator end effector; and when the grabbing stability deviation is larger than a preset stability threshold value, the closing speed and the contact pressure of all the finger tips of the mechanical arm are adjusted. By the adoption of the scheme, self-adaptive compliant control of the manipulator based on local features can be achieved in a complex grabbing environment, so that stable carrying of materials in the mechanical automatic grabbing process is guaranteed.

Description

Technical Field

[0001] This application relates to the field of gripping control technology, and more specifically, to a mechanical automated material gripping device and method. Background Technology

[0002] Mechanical automated material handling is an automated process that uses robotic arms to pick up, transfer, and place materials, with sensing, motion planning, and end effector control at its core. Its goal is to achieve highly stable and adaptable intelligent handling in complex and changing production environments. While the development of 3D sensing, compliant control, and tactile sensing technologies has enabled automated equipment to achieve a high success rate in handling some regular objects, it still struggles to achieve high-precision path planning and real-time stable control when dealing with irregularly shaped materials, materials with significant differences in texture, or materials with complex surface structures.

[0003] However, existing mechanical gripping processes typically rely on preset gripping postures or rigid control strategies based on simple force feedback, making it difficult to adapt to the local geometric features of material surfaces and dynamic slippage phenomena during the gripping process. Especially when gripping complex-shaped or easily slippery materials, existing methods cannot perceive changes in the contact state between the material and the end effector fingertips of the robot in real time, leading to slippage or uneven gripping force in the initial stages of gripping, thus reducing gripping stability and success rate. Furthermore, existing technologies lack dynamic prediction and adaptive adjustment mechanisms for closed paths, often resulting in significant gripping stability deviations or even material drop, affecting overall operational efficiency. Therefore, how to achieve adaptive compliant control of the robot based on local features in complex gripping environments to ensure stable material handling during automated mechanical gripping has become a challenge for the industry. Summary of the Invention

[0004] This application provides a mechanical automated material grasping device and method, which can realize adaptive compliant control of the robot based on local features in complex grasping environments.

[0005] In a first aspect, this application provides a mechanically automated material handling method, comprising the following steps: Acquire 3D point cloud data of the target material, and then generate the grasping space for the target material; Based on the local curvature distribution of the grasping space, multiple fingertip adjustment parameters of the robotic end effector are selected, and all fingertip adjustment parameters are converted into compliant control parameters of the robotic end effector during the closing process. The contact signal of the robot end effector when it initially grasps the target material is collected. Based on the interaction characteristics of force gradient and friction fluctuation in the contact signal, the tendency vector when the target material slides relative to the fingertip surface of the robot end effector is determined. The stability of the robot end effector in the sliding state of the target material is determined by the tendency vector. The compliant control parameters and the stability are used to dynamically predict the closed path of the grasping process, thereby obtaining the grasping stability deviation of the robot end effector. When the grasping stability deviation is greater than the preset stability threshold, the closing speed and contact pressure of each fingertip of the robot end effector are adjusted.

[0006] In conjunction with the first aspect, in one possible implementation, selecting multiple fingertip adjustment parameters of the robotic arm end effector based on the local curvature distribution of the grasping space specifically includes: For each candidate grasping region in the grasping space, determine the local curvature distribution characteristics of its surface; Preliminary adjustment parameters of the robotic arm end effector fingertip are matched for each candidate grasping region by all local curvature distribution features; Feasibility screening and optimization were performed on all the preliminary adjustment parameters to obtain multiple fingertip adjustment parameters for the robot's end effector.

[0007] In conjunction with the first aspect, in one possible implementation, converting all fingertip adjustment parameters into compliant control parameters for the robotic end effector during the closing process specifically includes: Based on all fingertip adjustment parameters, key variables related to contact mechanics were extracted. Based on the extracted key variables, determine the real-time position / force adjustment rules required for each fingertip of the robot end effector during the closing process; The compliant control parameters of the robot end effector during the closing process are determined by the real-time position / force adjustment rules.

[0008] In conjunction with the first aspect, in one possible implementation, determining the tendency vector when the target material slips relative to the surface of the manipulator's end effector finger, based on the interaction characteristics of the force gradient and frictional fluctuations in the contact signal, specifically includes: The time series of the normal force and tangential force on the fingertip are extracted from the contact signal to determine the force gradient and friction fluctuation. The coupling relationship and phase difference between the force gradient and the friction fluctuation in the time domain are analyzed to identify the interaction mode characterizing slip initiation. The projection vector of the sliding direction onto the fingertip contact plane is determined based on the interaction mode of the sliding initiation. The direction and amplitude of the projection vector are normalized to obtain the tendency vector when the target material slides relative to the surface of the end effector finger of the robot.

[0009] In conjunction with the first aspect, in one possible implementation, determining the stability of the robot end effector in the target material slippage state by the approach vector specifically includes: The local stability evaluation value of the individual stability of each fingertip of the robot end effector is determined by the tending vector; The local stability evaluation values ​​of all fingertips are weighted and fused, and the stability of the robot end effector under the sliding state of the target material is determined by combining the balance of force distribution of each fingertips.

[0010] In conjunction with the first aspect, in one possible implementation, the closed path of the grasping process is dynamically predicted through the compliance control parameters and the stability, thereby obtaining the grasping stability deviation of the robot end effector, specifically including: A predictive model for simulating the dynamic behavior of the grasping closure process is constructed based on the compliance control parameters and the stability. The prediction model is used to simulate the movement of the fingertip from its current position to a fully closed state along the planned closed path. The grasping stability deviation of the robot's end effector is determined based on the final state predicted by simulation.

[0011] In conjunction with the first aspect, in one possible implementation, the target material is an irregularly shaped, slippery, or soft material.

[0012] Secondly, this application provides a mechanically automated material handling device, comprising: The acquisition module is used to acquire the three-dimensional point cloud data of the target material, and then generate the grasping space of the target material. The processing module is used to filter out multiple fingertip adjustment parameters of the robotic arm end effector based on the local curvature distribution of the grasping space, and convert all fingertip adjustment parameters into compliant control parameters of the robotic arm end effector during the closing process; The processing module is also used to collect the contact signal when the robot end effector initially grasps the target material, and determine the tendency vector when the target material slides relative to the fingertip surface of the robot end effector based on the interaction characteristics of the force gradient and friction fluctuation in the contact signal, and determine the stability of the robot end effector in the sliding state of the target material by the tendency vector. The processing module is also used to dynamically predict the closed path of the grasping process through the compliance control parameters and the stability, thereby obtaining the grasping stability deviation of the robot end effector. The execution module is used to adjust the closing speed and contact pressure of each fingertip of the robot end effector when the grasping stability deviation is greater than a preset stability threshold.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described mechanical automated material handling method.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to perform the aforementioned automated material handling method.

[0015] The technical solution provided in this application has the following beneficial effects: This application's solution, firstly, selects multiple fingertip adjustment parameters for the robotic arm's end effector based on the local curvature distribution of the grasping space. This enables the robotic arm to adapt to irregular material surfaces in complex environments, allowing it to enter the grasping state with a more suitable contact point at the initial closing stage, thereby reducing the risk of excessive pressure, accidental collision, or slippage. Secondly, converting all fingertip adjustment parameters into compliant control parameters for the robotic arm's end effector during the closing process allows the robotic arm to have adjustable elastic response and dynamic clearance capabilities during the closing process, establishing stable contact on complex material surfaces in a gentler manner, reducing the risk of slippage, falling, or material damage. Then, the stability of the robotic arm's end effector in the target material's slippage state is determined by the tendency vector, enabling early detection of slippage risks. This allows the robotic arm to take compensatory measures at the initial slippage stage, enhancing its ability to suppress dynamic disturbances in complex grasping environments, thereby improving the stability of the final handling process. Finally, through the compliant control parameters and the stability... The system dynamically predicts the closed path of the grasping process to obtain the grasping stability deviation of the robot's end effector. This allows for overall control over the contact state, clamping force distribution, and fingertip movement trends during material grasping, ensuring the final path better conforms to the material surface changes, improving grasping quality, and guaranteeing material stability and force balance during handling. Furthermore, when the grasping stability deviation exceeds a preset stability threshold, the closing speed and contact pressure of each fingertip of the robot's end effector are adjusted. This enables timely compensation when slippage risk increases or posture deviation grows, forming a closed-loop self-adjusting process for the grasping action. It dynamically corrects the grasping state based on actual material characteristics without human intervention, reducing slippage, deviation, and damage, ensuring stable handling in complex grasping environments, and achieving a high degree of adaptability and stability in the overall material grasping process. In summary, this solution enables adaptive compliant control of the robot based on local features in complex grasping environments, ensuring stable material handling during automated mechanical grasping processes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an exemplary flowchart of a mechanically automated material handling method according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating the determination of fingertip adjustment parameters according to some embodiments of this application; Figure 3This is an exemplary flowchart illustrating the determination of stability according to some embodiments of this application; Figure 4 This is a structural schematic diagram of a mechanically automated material handling device according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing a mechanically automated material handling method according to some embodiments of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a mechanically automated material handling method according to some embodiments of this application. The mechanically automated material handling method mainly includes the following steps: In step 101, the three-dimensional point cloud data of the target material is acquired, and then the grasping space of the target material is generated.

[0020] In practice, acquiring the 3D point cloud data of the target material and generating its grasping space can be achieved in the following way: A 3D vision sensor, such as a structured light 3D scanner, mounted on the robot's base or worktable, can scan the target material to obtain raw 3D point cloud data. The raw point cloud is then denoised and registered to construct a complete 3D point cloud model of the target material. Next, the surface curvature is calculated based on this model, and region segmentation is performed to obtain multiple independent sub-surface regions. Finally, based on the preset contact dimensions of the end effector's fingertips, regions that satisfy the area and shape requirements are selected. The constrained sub-regions are used as candidate grasping regions. Finally, the set of three-dimensional poses, boundary information, and surface normals of all candidate grasping regions is used as the grasping space of the target material. The grasping space is a parameterized set containing position, orientation, and region identifiers in its data structure, which can be directly used for grasping planning. As a preferred embodiment, the calibration of the sensor and the robot arm's base coordinate system can be completed before scanning, so that the point cloud data and the generated grasping space are directly located in the arm coordinate system. In other embodiments, a binocular stereo vision system can also be used to acquire point cloud data, which is not limited in this application.

[0021] It should be noted that the three-dimensional point cloud data in this application refers to the set of spatial coordinates that characterize the geometric shape of the surface of the target material; the grasping space in this application refers to the set of all candidate surface regions that the robot end effector can stably grasp, wherein the candidate grasping region represents a sub-surface region that meets the basic contact size requirements of the robot end effector and is the basic unit constituting the grasping space.

[0022] In step 102, multiple fingertip adjustment parameters of the robotic end effector are selected based on the local curvature distribution of the grasping space, and all fingertip adjustment parameters are converted into compliant control parameters of the robotic end effector during the closing process.

[0023] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart of determining fingertip adjustment parameters in some embodiments of this application. In this embodiment, the selection of multiple fingertip adjustment parameters of the robotic arm end effector based on the local curvature distribution of the grasping space can be achieved by the following steps: First, in step 1021, for each candidate grasping region in the grasping space, the local curvature distribution characteristics of its surface are determined; Secondly, in step 1022, preliminary adjustment parameters of the end effector fingertip of the robot are matched for each candidate grasping region by all local curvature distribution features; Finally, in step 1023, all preliminary adjustment parameters are screened for feasibility and optimized to obtain multiple fingertip adjustment parameters for the robot end effector.

[0024] In specific implementation, determining the local curvature distribution characteristics of the surface of each candidate grasping region in the grasping space can be achieved in the following way: For each candidate grasping region in the grasping space, firstly, extract the point cloud subset corresponding to the candidate grasping region from the three-dimensional point cloud model of the target material; then, calculate the normal vector of each point in the subset, and estimate the Gaussian curvature and mean curvature of each point based on the change of the normal vector using principal component analysis or surface fitting method; then, statistically analyze the curvature values ​​of all points to obtain a feature vector describing the curvature distribution of the candidate grasping region. The feature vector includes at least the mean curvature, standard deviation, and histogram of the curvature values ​​within a preset interval, which serve as the local curvature distribution characteristics of the surface of the candidate grasping region; wherein, as a preferred embodiment, the gradient change of the surface curvature of the candidate grasping region can be further calculated to capture edge or concave / convex features; in other embodiments, a curvature estimation network based on deep learning can also be used to directly regress the curvature distribution characteristics from the point cloud, which is not limited in this application.

[0025] In specific implementation, matching the initial adjustment parameters of the robotic end effector fingertip to each candidate grasping region based on all local curvature distribution features can be achieved in the following way: First, a curvature feature-adjustment parameter mapping database can be pre-established. This database is obtained through simulation or experimental calibration and stores combinations of fingertip adjustment parameters corresponding to different curvature distribution features. The adjustment parameters include at least the fingertip approach posture angle, pre-grasp opening width, initial contact force setting value, and finger surface compliance stiffness reference value. Then, the local curvature distribution feature vector of each candidate grasping region is input into the database. Through feature similarity matching, such as Euclidean distance or cosine similarity, the closest mapping entry is retrieved, thereby assigning a set of initial adjustment parameters of the robotic end effector fingertip to each candidate grasping region. The matching process can be completed online in real time or pre-calculated and stored offline. As a preferred embodiment, when the curvature distribution features are complex, support vector regression or neural network models can be used as mapping functions to directly predict the adjustment parameters based on the input features. This is not limited here.

[0026] In practical implementation, all preliminary adjustment parameters undergo feasibility screening and optimization. The multiple fingertip adjustment parameters of the robot's end effector can be implemented in the following way: First, based on the robot's kinematic model and workspace constraints, check whether the fingertip poses specified in the preliminary adjustment parameters matched to each candidate grasping area are reachable, and eliminate poses that would cause arm singularities or collisions; then, for parameters that pass the reachability check, use a grasping stability simulation model, such as static analysis based on force closure or friction cone constraints, to evaluate their theoretical grasping stability, and eliminate those with stability scores lower than the pre-defined values. A set of threshold parameters is defined. Finally, a multi-objective optimization algorithm, such as particle swarm optimization or gradient descent, can be used to fine-tune the remaining parameter set. The optimization objectives include maximizing grasping stability, minimizing grasping torque deviation, and satisfying the fingertip force balance condition. The optimized sets of fingertip adjustment parameters are output as the final screening results, which can be used as multiple fingertip adjustment parameters for the robot's end effector. The optimization process can be combined with real-time computing resources to ensure response speed. In other embodiments, a hard screening can also be performed directly based on kinematic and stability rules through a rule engine, which is not limited here.

[0027] It should be noted that the local curvature distribution feature in this application refers to the numerical representation of the surface unevenness and geometric shape changes of a candidate grasping area; the preliminary adjustment parameters of the fingertip in this application refer to the initial suggested values ​​of a set of robot end effector control parameters obtained from the initial matching; the fingertip adjustment parameters in this application refer to a set of multiple sets of optimized and feasible robot end effector fingertip control parameters, which are used as the final control instruction set to guide the robot end effector to perform adaptive, high-success-rate grasping actions.

[0028] In some embodiments, converting all fingertip adjustment parameters into compliant control parameters for the robot end effector during the closing process can be achieved through the following steps: Based on all fingertip adjustment parameters, key variables related to contact mechanics were extracted. Based on the extracted key variables, determine the real-time position / force adjustment rules required for each fingertip of the robot end effector during the closing process; The compliant control parameters of the robot end effector during the closing process are determined by the real-time position / force adjustment rules.

[0029] In specific implementation, the key variables related to contact mechanics can be extracted from all fingertip adjustment parameters in the following way: each set of fingertip adjustment parameters is read sequentially, and the variables that directly determine the contact mechanics behavior between the fingertip and the material are parsed and extracted from each set of fingertip adjustment parameters as key variables related to contact mechanics. These key variables include at least: the expected initial contact force setting value defined by the fingertip adjustment parameters, the fingertip approach posture angle used to describe the optimal orientation when the fingertip approaches the material, and the finger surface compliance stiffness reference value as a compliance control benchmark. The extraction process can be completed by the parameter parsing module, which locates and outputs a list of values ​​of the above variables according to the preset parameter structure template for use in the next step of calculation. In a preferred embodiment, the pre-grabbing opening width can also be extracted as an auxiliary variable for generating the motion trajectory.

[0030] In specific implementation, the real-time position / force adjustment rules required for each fingertip of the robotic arm end effector during the closing process, based on the extracted key variables, can be achieved in the following way: the expected initial contact force setting value and the finger surface compliance stiffness reference value from the extracted key variables can be input into a preset impedance control model; this model takes the deviation between the target contact force and the current contact force as input, and calculates in real time the fingertip position or velocity correction amount required to eliminate this force deviation through the stiffness and damping relationship defined in the model. The force deviation-pose adjustment amount mapping relationship formed by this calculation process constitutes the real-time position / force adjustment rules required for each fingertip of the robotic arm end effector during the closing process; specifically, for each fingertip, this rule clarifies how to adjust its position or velocity to maintain stable and compliant contact when the actual contact force deviates from the expected value during the grasping and closing process; wherein, the parameters of the impedance control model can be adaptively adjusted according to the fingertip approach posture angle to adapt to different contact geometries; other methods can also be used to determine this in other embodiments, which are not limited here.

[0031] In specific implementation, the compliance control parameters of the robot end effector during the closing process can be determined by the real-time position / force adjustment rules in the following way: the real-time position / force adjustment rules are parameterized, and their core dynamic relationship is expressed as a set of stiffness coefficient matrices and damping coefficient matrices that can directly drive the servo controller; specifically, based on the proportional and differential relationship between force deviation and pose adjustment in the adjustment rules, the stiffness coefficient and damping coefficient values ​​of each finger joint or Cartesian space direction are directly mapped or calculated through fitting; finally, the stiffness and damping coefficients of all finger joints are integrated and synchronized in time to generate a set of unified compliance control parameters covering the entire closing action phase of the robot end effector; these compliance control parameters are directly sent to the robot's underlying servo ring to achieve active compliance and anti-disturbance capability throughout the closing process; in a preferred embodiment, the global damping coefficient can be uniformly scaled according to the estimated weight of the material to suppress oscillation; other methods can also be used to determine this in other embodiments, which are not limited here.

[0032] It should be noted that the key variables in this application refer to the physical quantities that affect the contact state and interaction force between the fingertip and the target material; the real-time position / force adjustment rules in this application refer to the closed-loop control logic of how each fingertip dynamically adjusts its motion posture or speed according to the real-time contact force changes during the grasping and closing phase; and the compliance control parameters in this application refer to the set of stiffness and damping coefficients that can be directly configured to the underlying servo controller of the robot end effector, which are used to directly drive the robot end effector to achieve the required active compliance behavior.

[0033] In step 103, the contact signal of the robot end effector when it initially grasps the target material is collected. Based on the interaction characteristics of force gradient and friction fluctuation in the contact signal, the tendency vector when the target material slides relative to the fingertip surface of the robot end effector is determined. The stability of the robot end effector in the sliding state of the target material is determined by the tendency vector.

[0034] In specific implementation, the contact signal during the initial grasping of the target material by the end effector of the robotic arm can be acquired in the following way: A six-dimensional force / torque sensor installed on the wrist or fingertips of the end effector can be used to acquire the contact signal during the initial grasping stage. When the fingertips close according to the compliance control parameters and make initial contact with the material surface, the sensor measures the three-dimensional force and three-dimensional torque components of each fingertip in real time, forming the original contact signal. This contact signal is then filtered and noise-reduced, and converted to a unified fingertip contact point coordinate system, outputting a time-synchronized force and torque signal. The initial grasping stage is defined as a short time window from the first detection of effective contact force. As a preferred embodiment, a low-pass filter can be used for noise reduction. In other embodiments, a tactile sensor array can be used to directly measure the pressure distribution. Other methods can also be used for acquisition in other embodiments, and are not limited here.

[0035] It should be noted that the contact signal in this application refers to the timing data reflecting the real-time interaction force and torque between the fingertip and the material during the brief period when the end effector of the robot arm makes initial contact with the target material.

[0036] In some embodiments, determining the tendency vector when the target material slips relative to the surface of the manipulator's end effector finger, based on the interaction characteristics of the force gradient and friction fluctuations in the contact signal, can be achieved through the following steps: The time series of the normal force and tangential force on the fingertip are extracted from the contact signal to determine the force gradient and friction fluctuation. The coupling relationship and phase difference between the force gradient and the friction fluctuation in the time domain are analyzed to identify the interaction mode characterizing slip initiation. The projection vector of the sliding direction onto the fingertip contact plane is determined based on the interaction mode of the sliding initiation. The direction and amplitude of the projection vector are normalized to obtain the tendency vector when the target material slides relative to the surface of the end effector finger of the robot.

[0037] In specific implementation, the time series of the normal force and tangential force experienced by the fingertip are extracted from the contact signal to determine the force gradient and friction fluctuation. This can be achieved as follows: the contact signal of each fingertip in the contact point coordinate system can be decomposed into a normal force component along the surface normal direction and a tangential force component along two orthogonal directions in the contact plane; then, the amplitudes of the resultant or component forces of the normal force and tangential force are sampled to form their respective time series; then, the first derivative of each of these two time series is calculated using the central difference method to obtain the force variation description. The normal force gradient and the tangential force gradient, which are of varying speeds, together constitute the force gradient. Simultaneously, the tangential force time series is bandpass filtered, for example, with a passband of 10Hz to 200Hz, to extract its high-frequency fluctuation components. The root mean square value of these components within the sliding time window is then calculated as the quantized frictional fluctuation characteristic. In a preferred embodiment, wavelet transform can be used to extract the energy of the tangential force signal in a specific frequency band as the frictional fluctuation. In other embodiments, spectral analysis can be directly used to determine the fluctuation characteristics; this application does not limit this approach.

[0038] In specific implementation, analyzing the coupling relationship and phase difference between the force gradient and the friction fluctuation in the time domain, and then identifying the interaction mode characterizing slip initiation, can be achieved in the following way: First, the time series of the force gradient and the friction fluctuation are time-aligned; then, the cross-correlation function between the two is calculated, the time delay corresponding to the cross-correlation peak is found as the phase difference, and the correlation coefficient between the two within the sliding window is calculated to quantify the coupling strength; next, a discrimination condition is set, for example: when the tangential force gradient is continuously negative and the amplitude of the friction fluctuation exceeds a preset threshold, and the correlation coefficient between the two is negative and the absolute value is greater than the set threshold, it is determined that an interaction mode characterizing slip initiation has occurred. The preset threshold can be set by experimental calibration and statistical methods, and adjusted proportionally according to the friction coefficient between the fingertip surface material and the target material. Alternatively, an online adaptive method can be used to dynamically update the threshold with recent historical data statistics. In addition, as a preferred embodiment, a classifier based on a hidden Markov model can be used to identify the joint feature sequence; in other embodiments, common modes can also be extracted and analyzed through empirical mode decomposition, which is not limited in this application.

[0039] In specific implementation, determining the projection vector of the sliding direction on the fingertip contact plane based on the interaction mode of sliding initiation can be achieved in the following way: when the interaction mode of sliding initiation is identified, the tangential force vector in the contact plane at that moment is obtained; based on the direction of the descent of the tangential force gradient and the main oscillation direction of friction fluctuation in the interaction mode, the main direction of the sliding trend is determined. For example, the initial direction of the sliding direction projection vector is taken as the weighted composite direction of the negative direction of the tangential force gradient and the main direction of friction fluctuation, and its initial amplitude is set to be proportional to the product of the magnitude of the tangential force gradient and the amplitude of friction fluctuation. The two-dimensional vector composed of the main direction of the sliding trend and its corresponding initial amplitude is the projection vector of the sliding direction on the fingertip contact plane. In a preferred embodiment, the instantaneous change direction of the tangential force vector can be directly used as the sliding direction. In other embodiments, the direction can also be determined by predicting the change trend of the tangential force through a filter. This application does not limit this.

[0040] In specific implementation, the direction and amplitude of the projection vector are normalized to obtain the tendency vector when the target material slides relative to the surface of the end effector finger of the robot. This can be achieved in the following way: First, the amplitude of the projection vector is divided by a normalized reference value, such as the current maximum static friction force estimate, to obtain the relative amplitude, which is then restricted to the interval [0, 1]. At the same time, the direction of the projection vector is converted into a unit vector in the contact plane. Finally, the normalized relative amplitude is multiplied by the unit direction vector to obtain the tendency vector when the target material slides relative to the surface of the end effector finger of the robot. This tendency vector is a two-dimensional vector, whose direction represents the direction of the sliding trend and whose amplitude represents the intensity of the sliding trend. In a preferred embodiment, the normalized reference value can be dynamically updated according to the real-time normal force and the preset friction coefficient. In other embodiments, a nonlinear function can also be used to smoothly normalize the amplitude, which is not limited here.

[0041] It should be noted that, in this application, the force gradient refers to the first derivative describing the rate of change of the normal and tangential forces on the fingertip over time, and is used as an indicator to characterize the dynamic evolution of the contact state; the frictional fluctuation in this application refers to the characteristic signal in the contact signal that characterizes the microscopic slippage or stick-slip oscillation phenomenon of the contact interface; the slip initiation interaction mode in this application refers to a characteristic state identified by analyzing the coupling relationship (such as strong negative correlation) and phase difference between the force gradient and the frictional fluctuation in the time domain; the projection vector in this application refers to a two-dimensional vector composed of the main direction of the slip trend and its corresponding initial amplitude; the slip trend vector in this application refers to a quantitative parameter with direction and intensity information that describes the slip trend in a standardized dimensionless manner in the local coordinate system of the fingertip.

[0042] In some embodiments, reference Figure 3As shown, this figure is an exemplary flowchart of determining stability in some embodiments of this application. In this embodiment, determining the stability of the robot end effector in the target material slippage state by the approach vector can be achieved by the following steps: The local stability evaluation value of the individual stability of each fingertip of the robot end effector is determined by the tending vector; The local stability evaluation values ​​of all fingertips are weighted and fused, and the stability of the robot end effector under the sliding state of the target material is determined by combining the balance of force distribution of each fingertips.

[0043] In specific implementation, the local stability evaluation value of the individual stability of each fingertip of the robot end effector, determined by the trend vector, can be achieved in the following way: For each fingertip of the robot end effector, its corresponding trend vector is read, and the normalized amplitude representing the slippage tendency intensity in the trend vector is extracted; then, the normalized amplitude is input into a preset stability mapping function, which is configured to map the normalized amplitude to an evaluation value within a predetermined range, such as 0 to 1 or 0 to 100; wherein, the mapping function is designed such that the weaker the slippage tendency intensity (i.e., the smaller the normalized amplitude), the higher the output evaluation value, so as to represent a higher degree of local stability; specifically, a linear decay function or an exponential decay function can be used to implement this mapping, thereby obtaining the local stability evaluation value of the individual stability of each fingertip of the robot end effector; as a preferred embodiment, the mapping function can be calibrated offline based on historical grasping data of the fingertip surface material; in other embodiments, a lookup table method can also be used to directly obtain the evaluation value from the normalized amplitude, which is not limited in this application.

[0044] In practical implementation, the stability evaluation values ​​of the local stability of all fingertips are weighted and fused, and the stability of the robot end effector under the sliding state of the target material is determined by combining the uniformity of the force distribution of each fingertip. This can be achieved in the following way: First, assign a weight coefficient to each fingertip. This weight coefficient can be based on the importance of the fingertip in the grasping task, for example, the main load-bearing fingertip has a higher weight, or it can be preset based on its historical reliability data; then, calculate the weighted average of the local stability evaluation values ​​of all fingertips as the preliminary overall stability; next, analyze the real-time contact force distribution of each fingertip during the grasping process, and calculate its comparison with the ideal uniform distribution or the preset minimum value. The degree of deviation in force distribution can be measured, for example, by calculating the standard deviation between the force applied to each fingertip and the average force. Finally, the initial overall stability is reduced based on this degree of deviation. For example, the greater the deviation, the smaller the reduction factor, with a minimum not lower than a lower limit. The reduced value is then used as the final stability of the robot end effector in the target material sliding state. In a preferred embodiment, the reduction factor can be determined based on the degree of deviation using a piecewise linear function or a lookup table. In other embodiments, the degree of force distribution deviation can also be calculated as an independent penalty term combined with a weighted average, which is not limited here.

[0045] It should be noted that the local stability evaluation value in this application refers to a quantitative score reflecting the ability of the end effector fingertip to resist slippage and maintain local contact stability during the gripping process; the balance of fingertip force distribution in this application refers to the degree of deviation of the contact force applied or borne by each fingertip from the ideal or optimal distribution state in a multi-finger gripping state, which is used as a key indicator to measure the synergy and coordination effect of gripping forces; the stability in this application refers to a comprehensive quantitative index of the overall ability and robustness of the end effector of the robot to resist the slippage of the target material in the current gripping configuration.

[0046] In step 104, the closed path of the grasping process is dynamically predicted by the compliance control parameters and the stability, thereby obtaining the grasping stability deviation of the robot end effector.

[0047] In some embodiments, the grasping stability deviation of the robotic arm end effector can be obtained by dynamically predicting the closed path of the grasping process using the compliance control parameters and the stability, through the following steps: A predictive model for simulating the dynamic behavior of the grasping closure process is constructed based on the compliance control parameters and the stability. The prediction model is used to simulate the movement of the fingertip from its current position to a fully closed state along the planned closed path. The grasping stability deviation of the robot's end effector is determined based on the final state predicted by simulation.

[0048] In specific implementation, a predictive model for simulating the dynamic behavior of the gripping closure process can be constructed based on the compliance control parameters and the stability. This can be achieved by using the joint or Cartesian space stiffness matrix and damping matrix from the compliance control parameters as core parameters describing the dynamic characteristics of the interaction between the fingertip and the target material, thus constructing a simplified multibody dynamics model. Simultaneously, the stability is introduced into the model as a dynamic constraint or model attenuation factor to reflect the influence of the overall anti-slip capability of the current gripping system on the stability of the closure process. Specifically, this model simplifies each fingertip into a single entity controlled by the stiffness and damping parameters. A mass-spring-damping system is connected to the virtual material, and the target material is regarded as a rigid body with equivalent mass and inertia. The model's input includes the current position, velocity, and contact force state of each fingertip, as well as the planned closed path. The model's output is a predicted sequence of the fingertip state and material pose over a future period, ultimately yielding a predictive model for simulating the dynamic behavior of the grasping and closing process. In a preferred embodiment, to balance computational speed and accuracy, a linear time-varying system or state-space equations can be used to describe the model. In other embodiments, a data-based surrogate model can also be used for prediction, and this application does not limit this.

[0049] In specific implementation, simulating the motion of the fingertip from its current position to a fully closed state along the planned closed path using the prediction model can be achieved in the following way: The actual position, velocity, and contact force of each fingertip collected at the current moment are used as the initial state of the prediction model; then, iterating forward with a fixed, short time step, such as 1 millisecond to 10 milliseconds, in each iteration, the model calculates and updates the predicted position, predicted velocity, and predicted contact force of each fingertip in the next step based on the expected motion input of the fingertip along the planned closed path, the current dynamic parameters, and the introduced stability constraints. The predicted pose of the material is updated synchronously; this iterative process continues until the simulated fingertip position reaches the preset fully closed state or the maximum number of prediction steps is reached, thereby obtaining a complete state prediction sequence from the current state to the end of the closure process; wherein, the planned closure path is given by the initial grasping plan, which is usually a set of time trajectories of fingertip positions; as a preferred embodiment, the desired motion input can be finely adjusted in real time according to the predicted contact force during the simulation process to simulate the effect of compliant control; in other embodiments, numerical integration methods such as the Runge-Kutta method can also be used to improve the simulation accuracy, and this application does not limit this.

[0050] In specific implementation, determining the grasping stability deviation of the robot end effector based on the final state predicted by simulation can be achieved in the following way: First, extract key indicators from the final state predicted by simulation, which may include: the predicted final contact force of each fingertip, the predicted offset of the final material pose relative to the desired pose, and the trend of the stability throughout the prediction sequence, such as whether it continues to decay; then, calculate the deviations of these indicators from the ideal state, for example: calculate the uniformity of the predicted contact force distribution, such as the variance, as the force distribution deviation; calculate the magnitude of the material pose offset as the pose deviation; and calculate the maximum decay of the stability as the stability decay deviation; finally, sum these three deviation components according to preset weights to obtain a single scalar value, and use this value as the grasping stability deviation of the robot end effector; wherein, as a preferred embodiment, the preset weights can be calibrated offline or adaptively adjusted online according to different material types and the importance of the grasping task; in other embodiments, a multi-indicator fusion method, such as calculating the deviation based on fuzzy logic, can also be used, and this application does not limit this.

[0051] It should be noted that the predictive model in this application refers to a model used to simulate and extrapolate the subsequent dynamics of the grasping closure process in the digital domain. It is used to prospectively extrapolate the possible results if the original closure path is continued under the current control parameters and stability conditions. The final state predicted in the simulation in this application refers to the set of system state prediction data about the moment when the fingertip reaches complete closure. The grasping stability deviation in this application refers to the scalar value of the comprehensive difference between the expected final result and the ideal stable state if the end effector of the robot continues to perform the grasping closure action in the current state.

[0052] In step 105, when the grasping stability deviation is greater than the preset stability threshold, the closing speed and contact pressure of each fingertip of the robot end effector are adjusted.

[0053] In specific implementation, when the grasping stability deviation is greater than the preset stability threshold, the closing speed and contact pressure of each fingertip of the robot end effector can be adjusted in the following way: First, the grasping stability deviation is compared with the preset stability threshold. If the grasping stability deviation is greater than the preset stability threshold, an adjustment mechanism is triggered. This adjustment mechanism can calculate a comprehensive adjustment gain based on the magnitude of the grasping stability deviation and the stability and compliance control parameters used to generate the deviation. Then, based on this adjustment gain, the planned closing speed of all fingertips is reduced proportionally to allow more time for force adjustment and stabilization. Simultaneously, for each fingertip, based on its stability... The local stability evaluation values ​​contributed in the calculation are adjusted differentially. For example, for fingertips with lower local stability evaluation values, i.e., poorer local stability, the set value of their contact pressure is increased by a higher proportion; for fingertips with higher local stability evaluation values, their pressure set value is maintained or only slightly adjusted. Finally, the adjusted closing speed and contact pressure set values ​​of each fingertip are combined with the original compliance control parameters to generate a new composite command and send it to the robot's underlying controller for execution. The preset stability threshold can be pre-calibrated using experimental data of different materials and grasping tasks. As a preferred embodiment, the adjustment process can adopt the proportional-integral-derivative (PID) control concept, dynamically adjusting the gain based on the rate of change of the grasping stability deviation. In other embodiments, the adjustment amount can also be determined based on fuzzy rules or by querying a preset adjustment table, which is not limited here.

[0054] It should be noted that the stability threshold in this application refers to the decision benchmark for judging whether the expected stability of the current grasping process is within an acceptable range. When the grasping stability deviation of the robot end effector exceeds this threshold, it triggers the active adjustment of the control parameters of the robot end effector, thus constituting the core judgment node in the entire adaptive grasping control logic.

[0055] In another aspect, in some embodiments, this application provides a mechanically automated material handling device, see reference. Figure 4 The figure is a schematic diagram of the structure of a mechanically automated material handling device 400 according to some embodiments of this application. The mechanically automated material handling device 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire the three-dimensional point cloud data of the target material, and then generate the grasping space of the target material. Processing module 402, in this application, is mainly used to filter out multiple fingertip adjustment parameters of the robotic arm end effector based on the local curvature distribution of the grasping space, and convert all fingertip adjustment parameters into compliant control parameters of the robotic arm end effector during the closing process; The processing module 402 described in this application is also used to collect the contact signal when the robot end effector initially grasps the target material, and determine the tendency vector when the target material slides relative to the fingertip surface of the robot end effector based on the interaction characteristics of the force gradient and friction fluctuation in the contact signal, and determine the stability of the robot end effector in the sliding state of the target material by the tendency vector. The processing module 402 described in this application is also used to dynamically predict the closed path of the grasping process through the compliance control parameters and the stability, thereby obtaining the grasping stability deviation of the robot end effector; The execution module 403 in this application is mainly used to adjust the closing speed and contact pressure of each fingertip of the end effector of the robot when the grasping stability deviation is greater than the preset stability threshold.

[0056] The foregoing has detailed examples of the automated material handling equipment and method provided in the embodiments of this application. It is understood that, in order to achieve the aforementioned functions, the corresponding apparatus includes hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described mechanical automated material handling method.

[0058] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a structural schematic diagram of the computer equipment implementing the automated material handling method of this application. The automated material handling method in the above embodiments can be achieved through… Figure 5The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0059] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0060] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0061] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0062] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0063] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0064] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0065] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to perform the above-described mechanical automated material handling method.

[0067] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0068] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A mechanically automated material handling method, characterized in that, Includes the following steps: Acquire 3D point cloud data of the target material, and then generate the grasping space for the target material; Based on the local curvature distribution of the grasping space, multiple fingertip adjustment parameters of the robotic end effector are selected, and all fingertip adjustment parameters are converted into compliant control parameters of the robotic end effector during the closing process. The contact signal of the robot end effector when it initially grasps the target material is collected. Based on the interaction characteristics of force gradient and friction fluctuation in the contact signal, the tendency vector when the target material slides relative to the fingertip surface of the robot end effector is determined. The stability of the robot end effector in the sliding state of the target material is determined by the tendency vector. The compliant control parameters and the stability are used to dynamically predict the closed path of the grasping process, thereby obtaining the grasping stability deviation of the robot end effector. When the grasping stability deviation is greater than the preset stability threshold, the closing speed and contact pressure of each fingertip of the robot end effector are adjusted.

2. The method as described in claim 1, characterized in that, The selection of multiple fingertip adjustment parameters for the robotic arm's end effector based on the local curvature distribution of the grasping space specifically includes: For each candidate grasping region in the grasping space, determine the local curvature distribution characteristics of its surface; Preliminary adjustment parameters of the robotic arm end effector fingertip are matched for each candidate grasping region by all local curvature distribution features; Feasibility screening and optimization were performed on all the preliminary adjustment parameters to obtain multiple fingertip adjustment parameters for the robot's end effector.

3. The method as described in claim 1, characterized in that, Converting all fingertip adjustment parameters into compliant control parameters for the robotic end effector during the closing process specifically includes: Based on all fingertip adjustment parameters, key variables related to contact mechanics were extracted. Based on the extracted key variables, determine the real-time position / force adjustment rules required for each fingertip of the robot end effector during the closing process; The compliant control parameters of the robot end effector during the closing process are determined by the real-time position / force adjustment rules.

4. The method as described in claim 1, characterized in that, The tendency vector for the target material to slip relative to the surface of the manipulator's end effector finger, determined based on the interaction characteristics of the force gradient and friction fluctuations in the contact signal, specifically includes: The time series of the normal force and tangential force on the fingertip are extracted from the contact signal to determine the force gradient and friction fluctuation. The coupling relationship and phase difference between the force gradient and the friction fluctuation in the time domain are analyzed to identify the interaction mode characterizing slip initiation. The projection vector of the sliding direction onto the fingertip contact plane is determined based on the interaction mode of the sliding initiation. The direction and amplitude of the projection vector are normalized to obtain the tendency vector when the target material slides relative to the surface of the end effector finger of the robot.

5. The method as described in claim 1, characterized in that, Determining the stability of the robot end effector in the target material slippage state by the approach vector specifically includes: The local stability evaluation value of the individual stability of each fingertip of the robot end effector is determined by the tending vector; The local stability evaluation values ​​of all fingertips are weighted and fused, and the stability of the robot end effector under the sliding state of the target material is determined by combining the balance of force distribution of each fingertips.

6. The method as described in claim 1, characterized in that, By dynamically predicting the closed path of the grasping process using the compliant control parameters and the stability, the grasping stability deviation of the robot end effector is obtained, specifically including: A predictive model for simulating the dynamic behavior of the grasping closure process is constructed based on the compliance control parameters and the stability. The prediction model is used to simulate the movement of the fingertip from its current position to a fully closed state along the planned closed path. The grasping stability deviation of the robot's end effector is determined based on the final state predicted by simulation.

7. The method as described in claim 1, characterized in that, The target material is irregularly shaped, slippery, or soft.

8. A mechanically automated material handling device, characterized in that, include: The acquisition module is used to acquire the three-dimensional point cloud data of the target material, and then generate the grasping space of the target material. The processing module is used to filter out multiple fingertip adjustment parameters of the robotic arm end effector based on the local curvature distribution of the grasping space, and convert all fingertip adjustment parameters into compliant control parameters of the robotic arm end effector during the closing process; The processing module is also used to collect the contact signal when the robot end effector initially grasps the target material, and determine the tendency vector when the target material slides relative to the fingertip surface of the robot end effector based on the interaction characteristics of the force gradient and friction fluctuation in the contact signal, and determine the stability of the robot end effector in the sliding state of the target material by the tendency vector. The processing module is also used to dynamically predict the closed path of the grasping process through the compliance control parameters and the stability, thereby obtaining the grasping stability deviation of the robot end effector. The execution module is used to adjust the closing speed and contact pressure of each fingertip of the robot end effector when the grasping stability deviation is greater than a preset stability threshold.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store a computer program, and the processor being used to retrieve and run the computer program from the memory, causing the computer device to perform the mechanical automated material handling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to perform the mechanically automated material handling method as described in any one of claims 1 to 7.

Images