Robot and grabbing control method thereof

Abstract

The invention provides a robot and a grabbing control method thereof, and the method comprises the steps: extracting a contour feature point set of a target object, converting the contact pressure distribution data of the robot during the current grabbing of the target object into a contact state matrix, and recognizing a stable contact region and a potential sliding region based on the contour feature point set and the contact state matrix; further determining a stress mapping matrix of the target object in the grabbing process, and determining the stability margin of the target object grabbed by the robot through the stress mapping matrix; based on the stability margin, a dynamic coupling coefficient of interaction between the dexterous finger of the robot and the target object is determined, and according to the dynamic coupling coefficient, the self-adaptive adjustment requirement of the current grabbing state of the robot is judged; and dynamically adjusting the grabbing force closed-loop control parameters when the robot dexterous finger executes the grabbing action according to the self-adaptive adjustment requirement. By the adoption of the scheme, self-adaptive grabbing force control based on contact state prediction can be achieved, so that the grabbing performance of the robot in the complex object grabbing process is improved.

Description

Technical Field

[0001] This application relates to the field of robot grasping and control technology, and more specifically, to a robot and a grasping and control method thereof. Background Technology

[0002] Robot grasping control refers to the entire process control method for controlling a robot's end effector (usually a robotic arm or gripper) to complete actions such as grasping, transporting, and placing objects. By coordinating and controlling the robot's perception system, motion system, and execution system, the robot can achieve stable, accurate, and efficient automatic grasping behavior based on the shape, position, posture, and physical characteristics of the target object. It is widely used in many fields such as industrial manufacturing, warehousing and logistics, and medical services, and is a key link in realizing intelligent robot operations.

[0003] However, existing robot grasping control methods still primarily rely on single-modal perception data for assessing grasping stability, such as using only vision or only force perception and static mechanical models. This is limited by incomplete perception information, insufficient consideration of dynamic contact characteristics, and weak model adaptability, leading to inaccurate state assessment, untimely slip warnings, and delayed control response. This often results in grasping failures when grasping slippery objects or encountering external disturbances, demonstrating insufficient stability maintenance capabilities and poor robustness and limited adaptability in the grasping process. In particular, when robots grasp unknown or complex objects, the lack of coupled analysis of spatiotemporal changes during grasping makes it difficult for robots to adjust in real-time according to changes in object posture in non-ideal environments, leading to grasping instability or slippage due to dynamic changes in contact state. Therefore, how to achieve adaptive grasping force control based on contact state prediction to improve robot grasping performance in complex object grasping processes has become a challenge for the industry. Summary of the Invention

[0004] This application provides a robot and its grasping control method, which can realize adaptive grasping force control based on contact state prediction to improve the robot's grasping performance in the process of grasping complex objects.

[0005] In a first aspect, this application provides a robot grasping control method, comprising the following steps: The robot collects depth image information of the target object during the grasping process and obtains contact pressure distribution data when the robot is grasping the target object. The depth image information is processed by point cloud segmentation to extract the contour feature point set of the target object. At the same time, the contact pressure distribution data is converted into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, the stable contact area and potential slip area in the grasping process are identified. Based on the pressure gradient distribution in the stable contact area and the stress concentration characteristics in the potential slip area, the force mapping matrix of the target object during the grasping process is determined, and then the stability margin of the robot's dexterous finger grasping the target object in the current position is determined through the force mapping matrix. Based on the stability margin and the real-time change trend of the contact state matrix, the dynamic coupling coefficient of the interaction between the robot's dexterous fingers and the target object during the grasping process is determined, and the adaptive adjustment requirements of the robot's current grasping state are judged according to the rate of change of the dynamic coupling coefficient. The closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions are dynamically adjusted according to the adaptive adjustment requirements.

[0006] In some embodiments, performing point cloud segmentation processing on the depth image information to extract the contour feature point set of the target object specifically includes: Obtain the 3D point cloud data corresponding to the depth image information; The target object point cloud and the background point cloud are segmented in the three-dimensional point cloud data; Extract the contour feature point set of the target object from the segmented point cloud of the target object.

[0007] In some embodiments, identifying stable contact areas and potential slippage areas during the grasping process based on the spatiotemporal correlation between the contour feature point set and the contact state matrix specifically includes: The spatial position registration is performed between the contour feature point set and the contact state matrix; Determine the spatiotemporal stability characteristics of the contact pressure data represented by the contact state matrix within the registered overlapping region; Based on the spatiotemporal stability characteristics, the stable contact area and the potential slippage area during the grasping process are divided.

[0008] In some embodiments, determining the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip region specifically includes: The normal and tangential force components at each contact point during the grasping process are calculated based on the pressure gradient distribution of the stable contact area. The force distribution confidence weight of each contact point during the grasping process is determined based on the stress concentration characteristics of the potential slip region. The force mapping matrix of the target object during the grasping process is constructed by assigning confidence weights to the normal and tangential force components at all contact points.

[0009] In some embodiments, determining the stability margin of the robot's dexterous finger grasping the target object at the current position using the force mapping matrix specifically includes: Perform matrix property analysis on the force mapping matrix; Stability quantification indicators are extracted based on matrix property analysis results; The stability quantification index is mapped to the stability margin of the robot's dexterous finger when grasping the target object in the current position.

[0010] In some embodiments, determining the adaptive adjustment requirement of the robot's current grasping state based on the rate of change of the dynamic coupling coefficient specifically includes: The rate of change of the dynamic coupling coefficient is compared with a preset adjustment threshold; Based on the comparison results, an adaptive adjustment requirement is generated for the robot's current grasping state.

[0011] In some embodiments, dynamically adjusting the closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action according to the adaptive adjustment requirements specifically includes: The adjustment direction and magnitude level are determined based on the aforementioned adaptive adjustment requirements; Key feature parameters are extracted from the evolution patterns of potential slip regions; The closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action are adjusted based on the adjustment direction and amplitude level and the key feature parameters.

[0012] Secondly, this application provides a robot including a grasping control unit, the grasping control unit comprising: The acquisition module is used to acquire depth image information of the target object during the robot's grasping process and to obtain contact pressure distribution data when the robot is currently grasping the target object. The processing module is used to perform point cloud segmentation processing on the depth image information, thereby extracting the contour feature point set of the target object, and at the same time converting the contact pressure distribution data into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, the module identifies the stable contact area and potential slip area during the grasping process. The processing module is further configured to determine the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip area, and then determine the stability margin of the robot's dexterous finger grasping the target object at the current position through the force mapping matrix. The processing module is also used to determine the dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process based on the stability margin and the real-time change trend of the contact state matrix, and to determine the adaptive adjustment requirements of the robot's current grasping state based on the rate of change of the dynamic coupling coefficient. The execution module is used to dynamically adjust the closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions according to the adaptive adjustment requirements.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described robot grasping control method.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned robot grasping control method.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: In this application, depth image information of the target object is collected during the robot's grasping process, and contact pressure distribution data of the robot's current grasping of the target object is obtained. The depth image information is processed by point cloud segmentation to extract the contour feature point set of the target object. At the same time, the contact pressure distribution data is converted into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, stable contact areas and potential slip areas during the grasping process are identified. The force mapping matrix of the target object during the grasping process is determined according to the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip area. Then, the stability margin of the robot's dexterous finger grasping the target object at the current position is determined through the force mapping matrix. The dynamic coupling coefficient of the interaction between the robot's dexterous finger and the target object during the grasping process is determined based on the stability margin and the real-time change trend of the contact state matrix. The adaptive adjustment requirement of the robot's current grasping state is judged according to the rate of change of the dynamic coupling coefficient. The grasping force closed-loop control parameters of the robot's dexterous finger when performing the grasping action are dynamically adjusted according to the adaptive adjustment requirement.

[0016] Therefore, in this application, firstly, the force mapping matrix of the target object during the grasping process is determined based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slippage area. This matrix can elevate local tactile signals to global object balance information, reduce single-sensor misjudgments, distinguish between local anomalies and global imbalances, and directly provide physical quantitative basis for stability assessment and control decisions, thereby reducing slippage false alarms and improving the physical consistency and accuracy of grasping criteria. Secondly, the stability margin of the robot's dexterous finger grasping the target object at the current position is determined through the force mapping matrix. The stability margin can compress the complex force distribution into a thresholdable risk indicator, which can be prioritized and triggered by conditions to achieve targeted force application or reduction, local compensation or posture adjustment, thereby reducing the grasping failure rate under external disturbances or unknown surfaces. Then, based on the stability margin and the real-time change trend of the contact state matrix, the relationship between the robot's dexterous finger and the target object during the grasping process is determined. The dynamic coupling coefficients between the interacting components can characterize the non-static nature of the interaction at both local and global scales. This allows for direct mapping of perceived contact evolution to control gain or damping correction, enhancing adaptability and robustness to unknown or changing conditions. Finally, by determining the adaptive adjustment requirements of the robot's current grasping state based on the rate of change of the dynamic coupling coefficients, the robot can keenly detect early signals of sudden changes or accelerated deterioration in the grasping state. This enables the robot to identify microscopic instability trends before macroscopic slippage occurs and to generate adaptive adjustment requirements in real time, achieving a fundamental shift from passive response to active intervention. Furthermore, based on these adaptive adjustment requirements, the robot dynamically adjusts the closed-loop control parameters of the grasping force when its dexterous fingers perform grasping actions, significantly improving the grasping success rate and operational efficiency in complex, disturbed, or unknown surface environments. In summary, this scheme enables adaptive grasping force control based on contact state prediction to improve the robot's grasping performance in complex object grasping processes. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an exemplary flowchart of a robot grasping control method according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating point cloud segmentation processing according to some embodiments of this application; Figure 3This is an exemplary flowchart illustrating the determination of stability margin according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of the grasping control unit according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing a robot grasping control method according to some embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] refer to Figure 1 The figure is an exemplary flowchart of a robot grasping control method according to some embodiments of this application. The robot grasping control method mainly includes the following steps: In step 101, depth image information of the target object is collected during the robot's grasping process, and contact pressure distribution data is obtained when the robot is currently grasping the target object.

[0021] In specific implementation, the depth image information of the target object during the robot's grasping process can be acquired in the following way: the target object can be imaged by a depth camera fixed on the robot's wrist or above the workspace to obtain raw depth data; then the raw depth data is preprocessed, which may include smoothing and denoising using a Gaussian filter, and converting the pixel depth values ​​into three-dimensional point cloud data in the robot's base coordinate system using camera intrinsic parameters, which is used as the depth image information of the target object during the robot's grasping process, wherein each point in the three-dimensional point cloud data contains three-dimensional spatial coordinate information; wherein, as a preferred embodiment, the depth camera can be an embedded structured light camera, and in other embodiments, the integrity of the point cloud can be improved by fusing the depth information of multiple cameras, which is not specifically limited here.

[0022] In specific implementation, the contact pressure distribution data when the robot is currently grasping the target object can be obtained in the following way: real-time sensing can be achieved through a flexible pressure sensor array that is attached to the curved surface of the fingertips and fingertips of the dexterous hand. This array consists of miniature piezoresistive or capacitive sensing units arranged in a matrix. During the grasping process, the signal acquisition circuit integrated inside the dexterous hand synchronously reads the pressure readings of all sensing units at a sampling frequency of not less than 100Hz. Based on the pre-calibrated spatial position of each unit on the finger surface, the pressure data collected at the same time is organized into a multi-dimensional pressure distribution map, which serves as the contact pressure distribution data when the robot is currently grasping the target object. In a preferred embodiment, the flexible pressure sensor array can be a thin-film sensor based on a conductive elastomer. In other embodiments, a fiber optic grating sensor can also be used to achieve distributed pressure measurement, which is not specifically limited here.

[0023] It should be noted that the depth image information in this application refers to a set of data used to characterize the surface geometry of the target object in three-dimensional space and its spatial pose relative to the robot; the contact pressure distribution data in this application refers to a set of data used to characterize the spatial distribution of pressure at the contact interface between the robot's dexterous hand and the target object.

[0024] In step 102, the depth image information is processed by point cloud segmentation to extract the contour feature point set of the target object. At the same time, the contact pressure distribution data is converted into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, the stable contact area and potential slip area in the grasping process are identified.

[0025] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart of point cloud segmentation processing in some embodiments of this application. In this embodiment, the point cloud segmentation processing of the depth image information to extract the contour feature point set of the target object can be achieved by the following steps: First, in step 1021, the three-dimensional point cloud data corresponding to the depth image information is obtained; Secondly, in step 1022, the target object point cloud and the background point cloud in the three-dimensional point cloud data are segmented; Finally, in step 1023, the contour feature point set of the target object is extracted from the segmented target object point cloud.

[0026] In specific implementation, the segmentation of the target object point cloud from the background point cloud in the 3D point cloud data can be achieved as follows: A random sampling consensus algorithm can be used to extract the largest planar point cloud from the 3D point cloud data as the background point cloud and remove it. The remaining point cloud is the initially segmented target object point cloud. Subsequently, Euclidean clustering segmentation is performed on the remaining point cloud. Based on the spatial distance threshold between point clouds, the interconnected target object point clouds are separated from the discrete noise point clouds, ultimately obtaining the complete target object point cloud. In a preferred embodiment, the spatial distance threshold can be set to 2-3 times the average point cloud density. In other embodiments, a region growing segmentation method can be used instead of Euclidean. Clustering segmentation is not limited here; the extraction of the target object's contour feature point set from the segmented target object point cloud can be achieved in the following way: the curvature features of each point in the segmented target object point cloud can be calculated based on principal component analysis, and points with curvature greater than a preset threshold can be selected as candidate feature points; then, non-maximum suppression is performed on the candidate feature points through an internal shape search algorithm, and points with curvature extrema within the neighborhood are retained to form a feature point set representing the geometric contour of the target object, which serves as the contour feature point set of the target object; wherein, as a preferred embodiment, the curvature threshold can be set to 1.5-2 times the average curvature of the point cloud, and the neighborhood radius can be set to 5% of the diagonal length of the point cloud bounding box; in other embodiments, scale-invariant feature transformation algorithms can also be used for feature point extraction, which is not limited here.

[0027] It should be noted that the three-dimensional point cloud data in this application is a data set used to accurately characterize the three-dimensional spatial structure and geometric shape of the target object surface in the robot coordinate system; the target object point cloud obtained by segmentation in this application is a subset of point cloud data used to calibrate and isolate the complete surface shape of the target object to be grasped in three-dimensional space; the contour feature point set in this application is a sparse point cloud subset used to characterize the key geometric contours and significant shape features of the target object surface.

[0028] In specific implementation, the contact pressure distribution data can be converted into a contact state matrix in the following way: a two-dimensional index matrix can be established for each finger based on the pre-calibrated spatial arrangement of each flexible pressure sensor array on the multi-finger dexterous hand, where the position of each matrix element corresponds to the physical coordinates of a sensing unit on the finger surface; the pressure readings of all sensing units in the contact pressure distribution data are filled according to their corresponding finger numbers and two-dimensional index positions to construct a three-dimensional contact state matrix with the finger number as the first dimension, the sensor row index as the second dimension, and the sensor column index as the third dimension; in a preferred embodiment, for missing data caused by sensor failure or communication interruption, bilinear interpolation of adjacent units can be used to complete it; in other embodiments, the pressure readings can also be normalized to the [0, 1] interval before constructing the matrix to eliminate the influence of differences in the range of different sensors, which is not limited here.

[0029] It should be noted that the contact state matrix in this application is a structured data set used to quantify the spatial distribution and intensity characteristics of the contact pressure between the robot's multi-fingered dexterous hand and the target object.

[0030] In some embodiments, the identification of stable contact areas and potential slip areas during the grasping process based on the spatiotemporal correlation between the contour feature point set and the contact state matrix can be achieved through the following steps: The spatial position registration is performed between the contour feature point set and the contact state matrix; Determine the spatiotemporal stability characteristics of the contact pressure data represented by the contact state matrix within the registered overlapping region; Based on the spatiotemporal stability characteristics, the stable contact area and the potential slippage area during the grasping process are divided.

[0031] In specific implementation, the spatial registration of the contour feature point set with the contact state matrix can be achieved in the following way: the contour feature point set can be transformed from the camera coordinate system to the robot hand coordinate system through a pre-calibrated hand-eye transformation matrix, so that it is in the same coordinate system as the contact state matrix; then, the iterative nearest point algorithm is used to perform fine registration of the transformed contour feature point set with the pressure distribution spatial region represented by the contact state matrix, minimizing the positional error between the two in three-dimensional space; wherein, as a preferred embodiment, the hand-eye transformation matrix can be obtained in advance through the Tsai-Lenz calibration method; in other embodiments... In this approach, when the target object has obvious geometric features, a feature-point-based matching method can also be used for fast coarse registration, which is not limited here. Determining the spatiotemporal stability characteristics of the contact pressure data represented by the contact state matrix within the registered overlapping region can be achieved as follows: Local pressure data of the contact state matrix over a continuous time series can be extracted within the registered spatial overlapping region; then, a sliding time window can be used to calculate the mean and standard deviation of pressure in each local region within the window, serving as a static feature representing spatial stability; simultaneously, the Frobeniu value of the difference matrix of pressure distribution between adjacent time frames can be calculated. The s-norm, as a dynamic feature characterizing temporal stability, is used to represent the spatiotemporal stability features of the contact pressure data characterized by the contact state matrix, thereby combining the obtained static features characterizing spatial stability with the dynamic features characterizing temporal stability. In a preferred embodiment, the sliding time window length can be set to 5-10 control cycles. In other embodiments, the spatiotemporal stability feature can also be determined by calculating the correlation coefficient of the pressure distribution or by performing frequency domain analysis; this is not limited here. Dividing the stable contact region and potential slip region during the grasping process based on the spatiotemporal stability feature can be achieved in the following way: [The text abruptly ends here, likely due to an incomplete translation or source material.] A pressure mean threshold, a standard deviation threshold, and a pressure change norm threshold are used to define regions in the spatiotemporal stability features that simultaneously satisfy a pressure mean higher than a first threshold and a standard deviation lower than a second threshold as stable contact regions. Regions in the spatiotemporal stability features that satisfy a pressure change norm higher than a third threshold or a standard deviation higher than a fourth threshold are defined as potential slip regions. In a preferred embodiment, the thresholds can be pre-calibrated experimentally based on the weight of the target object and the surface friction coefficient. In other embodiments, fuzzy logic classifiers or support vector machines can also be used to perform pattern recognition on the spatiotemporal stability features to achieve region division, which is not limited here.

[0032] It should be noted that, in this application, spatial registration refers to the process of establishing a spatial correspondence between visual contour information and tactile pressure information in a unified coordinate system; the overlapping area after registration in this application is a three-dimensional spatial range used for visual-tactile data fusion analysis, within which the visual contour of the target object and the tactile pressure distribution of the robotic hand coexist; the spatiotemporal stability characteristics in this application refer to a set of indicators used to quantitatively evaluate the spatial uniformity and temporal stability of the pressure distribution at the hand-object contact interface during robot grasping; the stable contact area in this application is the part of the hand-object contact interface used to provide reliable gripping force for the robot and maintain the balance of the grasping state; the potential slippage area in this application is the part of the hand-object contact interface used to indicate the relative motion risk or force transmission failure tendency of the hand-object contact interface during robot grasping.

[0033] In step 103, the force mapping matrix of the target object during the grasping process is determined based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip area. Then, the stability margin of the robot's dexterous finger grasping the target object at the current position is determined through the force mapping matrix.

[0034] In some embodiments, determining the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip region can be achieved by the following steps: The normal and tangential force components at each contact point during the grasping process are calculated based on the pressure gradient distribution of the stable contact area. The force distribution confidence weight of each contact point during the grasping process is determined based on the stress concentration characteristics of the potential slip region. The force mapping matrix of the target object during the grasping process is constructed by assigning confidence weights to the normal and tangential force components at all contact points.

[0035] In specific implementation, the calculation of the normal and tangential force components at each contact point during the grasping process based on the pressure gradient distribution of the stable contact area can be achieved in the following way: First, the original pressure value of each contact point during the grasping process can be obtained through the contact state matrix, and multiplied by the pre-calibrated effective contact area of ​​the sensor unit to obtain the normal force component of each contact point during the grasping process; simultaneously, the pressure gradient distribution of the stable contact area is calculated through the Sobel operator, the pressure gradient direction of each contact point is identified as the potential direction of the tangential force component at that point, and the pressure gradient amplitude at that point is normalized and multiplied by the maximum static friction force determined by Coulomb's law of friction. The obtained results are used as the tangential force components of each contact point during the grasping process. In a preferred embodiment, the effective contact area of ​​the sensor unit can be obtained from the calibration file of the pressure sensor array. In other embodiments, a neural network-based method can be used to directly regress the force components from the pressure distribution image; this is not limited here. The determination of the force distribution confidence weights of each contact point during the grasping process based on the stress concentration characteristics of the potential slip region can be achieved in the following way: the pressure distribution entropy of each contact point within the potential slip region can be calculated, and the entropy value can be normalized to the [0, 1] interval, with the complement taken as the corresponding value for the grasping process. The initial confidence level of each contact point is determined; simultaneously, the pressure change rate of the potential slip region in continuous time frames is detected, and an additional penalty coefficient is applied to contact points whose pressure change rate exceeds a threshold. The initial confidence level is multiplied by the penalty coefficient to obtain the force distribution confidence weight of each contact point during the grasping process. In a preferred embodiment, the pressure distribution entropy can be obtained by calculating the information entropy of the pressure within a 3×3 neighborhood. In other embodiments, the number of local pressure extremes and the distribution uniformity can also be used as alternative indicators of stress concentration characteristics; this application does not limit this. The confidence level of each contact point is determined by the normal and tangential force components and the corresponding force distribution confidence levels. The force mapping matrix of the target object during the grasping process can be constructed by the following method: a force and torque balance equation with the grasping center as the reference point can be established. The normal and tangential force components of each contact point are generated into corresponding force spirals according to their spatial positions. The force spirals corresponding to each contact point are multiplied by the force components corresponding to each contact point and then weighted according to their confidence level. The results are then concatenated column by column to form the force mapping matrix of the target object during the grasping process. In a preferred embodiment, the grasping center is set as the geometric center of the palm of the robot's multi-fingered dexterous hand. In other embodiments, the centroid of the target object can also be used as the grasping center for calculation. This is not a limitation.

[0036] It should be noted that the normal and tangential force components at the contact points in this application are two mutually perpendicular mechanical parameters used to characterize the degree of clamping of the hand-object contact interface in the vertical direction and the anti-slip capability in the tangential direction during the grasping process, respectively; the force component configuration reliability weight in this application is a coefficient used to quantify and evaluate the reliability of the force information provided by each contact point during the robot's grasping process and its importance in the overall force system balance; the force mapping matrix in this application is a mathematical structure used to establish a linear mapping relationship between the local contact forces at all contact points and the overall net external force and net torque acting on the target object.

[0037] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart of determining the stability margin in some embodiments of this application. The determination of the stability margin of the robot's dexterous finger grasping the target object in the current position through the force mapping matrix can be achieved by the following steps: Perform matrix property analysis on the force mapping matrix; Stability quantification indicators are extracted based on matrix property analysis results; The stability quantification index is mapped to the stability margin of the robot's dexterous finger when grasping the target object in the current position.

[0038] In specific implementation, the matrix property analysis of the force mapping matrix can be performed in the following way: the singular value decomposition algorithm can be used to decompose the force mapping matrix into the product of three specific matrices, thereby extracting the singular value spectrum and the corresponding singular vector of the force mapping matrix, which are then used as the matrix property analysis result. In a preferred embodiment, the singular value decomposition can be implemented using a numerical calculation method combining Householder transformation and Givens rotation. In other embodiments, the Jacobi eigenvalue algorithm can be used to approximate the singular values ​​of the force mapping matrix, which is not limited here. The stability quantification index can be extracted based on the matrix property analysis result in the following way: the smallest singular value can be selected from the singular value spectrum corresponding to the matrix property analysis result as the stability quantification index. This smallest singular value characterizes the stability of the robot's dexterous fingers grasping the target object in the weakest direction. The stability quantification index is determined by the following method: ...

[0039] It should be noted that the matrix property analysis results in this application are a dataset used to reveal the intrinsic characteristics and structural features of the robot's dexterous finger in the force transmission process when grasping a target object; the stability quantification index in this application is a parameter used to comprehensively measure the robot's dexterous finger's ability to resist external disturbances and maintain force balance when grasping a target object with a single scalar value; the stability margin in this application is a normalized measure used to evaluate the relative degree to which the robot's current grasping state deviates from the instability boundary, and its value directly characterizes the robustness of the robot's grasping action.

[0040] In step 104, the dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process is determined based on the stability margin and the real-time change trend of the contact state matrix. The adaptive adjustment requirements of the robot's current grasping state are then determined based on the rate of change of the dynamic coupling coefficient.

[0041] In some embodiments, determining the dynamic coupling coefficient between the robot's dexterous finger and the target object during the grasping process, based on the stability margin and the real-time changing trend of the contact state matrix, can be achieved through the following steps: Extract trend feature vectors based on the real-time changing trend of the contact state matrix; The stability margin and the trend feature vector are fused to generate a fused feature; The dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process is determined by the fusion features.

[0042] In specific implementation, the trend feature vector extracted based on the real-time change trend of the contact state matrix can be achieved in the following way: within a set sliding time window, the difference sequence of the Frobenius norm of the contact state matrix in consecutive time frames can be calculated, and the mean and variance of the difference sequence can be extracted as features representing the overall pressure change intensity; at the same time, the change angle sequence of the main direction of the contact state matrix in consecutive time frames can be calculated, and its standard deviation can be extracted as a feature representing the stability of the pressure distribution direction; finally, the mean, variance, and standard deviation are combined to form a trend feature vector; wherein, as a preferred embodiment, the length of the sliding time window can be set to 10 control cycles; in other embodiments, the trend feature vector can also be constructed by calculating the change of the eigenvalues ​​of the contact state matrix, and this application does not limit this; the stability margin and the trend feature vector can be fused to generate fused features in the following way: the stability margin and the trend feature vector can be concatenated to form an initial concatenated feature; then, through a single-layer fully connected neural network The network performs nonlinear transformation and dimensionality reduction on the initial spliced ​​features, outputting fused features with unified dimensions. In a preferred embodiment, the weight parameters of the fully connected neural network can be obtained through training on historical robot grasping data. In other embodiments, feature fusion can also be achieved by directly using weighted splicing, which is not limited here. The dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process can be determined using the fused features by inputting the fused features into a Sigmoid function, and using the function's output value as the dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process. The dynamic coupling coefficient has a range of (0, 1), and a larger value indicates a higher degree of dynamic coupling between the hand and the object when the robot grasps the target object. In a preferred embodiment, the slope parameter of the Sigmoid function can be adjusted according to the needs of different grasping tasks. In other embodiments, a piecewise linear function can also be used to map from the fused features to the dynamic coupling coefficient, which is not limited here.

[0043] It should be noted that the trend feature vector in this application is a statistical feature used to quantify the dynamic pattern of the intensity and direction of the contact pressure between the robot and the object during the grasping process, which evolves over time; the fusion feature in this application is a feature representation used to characterize the intrinsic relationship between the instantaneous stable state and the dynamic change trend of the contact pressure during the robot's dexterous finger grasps the target object; the dynamic coupling coefficient in this application is a dimensionless parameter used to quantify the tightness of the interaction and the strength of the stability correlation between the robot's dexterous finger and the target object during the grasping process. It reflects the synergy of the robot's dexterous finger grasping the target object. The higher the value, the tighter the connection between the hand and the object and the more synchronized the movement is during the robot's dexterous finger grasps the target object.

[0044] In some embodiments, the adaptive adjustment requirement of the robot's current grasping state based on the rate of change of the dynamic coupling coefficient can be achieved by the following steps: The rate of change of the dynamic coupling coefficient is compared with a preset adjustment threshold; Based on the comparison results, an adaptive adjustment requirement is generated for the robot's current grasping state.

[0045] In specific implementation, comparing the rate of change of the dynamic coupling coefficient with a preset adjustment threshold can be achieved in the following way: A numerical comparator within the embedded system can be used to numerically compare the rate of change of the dynamically coupling coefficient, calculated in real time, with a preset adjustment threshold stored in non-volatile memory. The preset adjustment threshold can be obtained through prior experimental calibration based on the target object's mass, surface friction coefficient, and desired grasping robustness level. As a preferred embodiment, a digital comparator with hysteresis can be used to prevent judgment jitter caused by signal noise at the threshold boundary. In other embodiments, an adaptive threshold adjustment strategy can also be used to dynamically fine-tune the threshold based on the historical success rate of the grasping task; this application does not limit this approach. The adaptive adjustment requirement for the robot's current grasping state, based on the comparison results, can be achieved in the following way: when the absolute value of the rate of change output by the numerical comparator exceeds a preset adjustment threshold, the adaptive adjustment requirement flag of the robot's current grasping state is immediately set to a valid state, and this state is sent to the grasping force control module via the robot control center's internal bus. In a preferred embodiment, the adaptive adjustment requirement flag is encoded using a single-bit Boolean variable, where "1" indicates an adjustment requirement and "0" indicates no adjustment is needed. In other embodiments, positive or negative sign information of the rate of change can be added to the flag to indicate the required adjustment direction, such as increasing or decreasing the grasping force; this is not limited here.

[0046] It should be noted that the rate of change of the dynamic coupling coefficient in this application is a dynamic index used to characterize the drastic change in the interaction state between the robot's dexterous finger and the target object; the comparison result in this application is a logical state used as a direct judgment basis for whether the drastic change in the dynamic characteristics of the robot when it is currently grasping the target object has reached the level requiring intervention; the adaptive adjustment requirement in this application is a decision signal used to send a command to the robot's grasping force control module to decide whether the grasping force parameters and the direction adjustment are to be adjusted immediately.

[0047] In step 105, the closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions are dynamically adjusted according to the adaptive adjustment requirements.

[0048] In some embodiments, dynamically adjusting the closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action according to the adaptive adjustment requirements can be achieved through the following steps: The adjustment direction and magnitude level are determined based on the aforementioned adaptive adjustment requirements; Key feature parameters are extracted from the evolution patterns of potential slip regions; The closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action are adjusted based on the adjustment direction and amplitude level and the key feature parameters.

[0049] In specific implementation, determining the adjustment direction and magnitude level based on the adaptive adjustment requirements can be achieved in the following way: the adjustment direction can be determined by parsing the change rate sign information contained in the adaptive adjustment requirements, where a positive sign indicates that the gripping force needs to be increased, and a negative sign indicates that the gripping force needs to be decreased; simultaneously, the magnitude level is determined based on the numerical range of the absolute value of the change rate, dividing the absolute value of the change rate into three pre-defined non-overlapping intervals, corresponding to three levels: fine adjustment, medium adjustment, and large adjustment; wherein, as a preferred embodiment, the setting of the level interval boundary values ​​is determined through statistical analysis of a large amount of gripping experimental data; in other embodiments, it is also possible to... This application does not limit the direct mapping of the absolute value of the rate of change to a continuous amplitude value using a continuous function. The key feature parameters extracted from the evolution law of the potential slip region can be achieved in the following way: In a continuous time frame sequence, the pixel area change rate of the potential slip region in the image coordinate system can be calculated as the area expansion speed feature; simultaneously, the magnitude and direction of the displacement vector of the centroid of this region between consecutive frames can be calculated as the movement speed and movement direction features, respectively. The obtained area expansion speed feature, movement speed, and movement direction features are all used as key feature parameters. In a preferred embodiment, the pixel area change rate can be calculated by... The difference in the number of pixels in adjacent frames is divided by the time interval to obtain the result; in other embodiments, the rate of change of region shape or the curvature change of region boundary can also be introduced as supplementary features, which is not limited here; the adjustment of the closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action based on the adjustment direction and amplitude level and the key feature parameters can be achieved in the following way: a rule base can be established with the adjustment direction and amplitude level and the key feature parameters as input and the proportional-integral-derivative controller parameter correction amount as output, and the corresponding proportional, integral, and derivative parameter adjustment amounts can be obtained by looking up a table; then the obtained parameters are adjusted The integer value is added to the current operating parameters of the robot's dexterous finger to obtain the new closed-loop control parameters for the grasping force when the robot's dexterous finger performs a grasping action. These adjusted closed-loop control parameters can then be downloaded in real-time to the joint controller of the robot's dexterous finger, replacing the original control parameters to drive the motors of each joint of the dexterous finger to generate corresponding torque outputs. This achieves precise adjustment of the grasping force and completes the adaptive grasping action. In a preferred embodiment, the rule base is constructed based on expert experience and offline simulation data. In other embodiments, a fuzzy inference system can also be used to calculate the parameter adjustment amount in real time; this is not limited here.

[0050] It should be noted that the adjustment direction and magnitude level in this application are decision parameters used to clearly guide the specific directionality and magnitude of change in the grasping force adjustment operation; the evolution law of the potential slip region in this application is used to describe the dynamic characteristics of the development trend and behavior pattern of the unstable contact region in the spatiotemporal dimension during the robot's grasping process; the key feature parameters in this application are a set of specific indicators used to quantify the representative dynamic characteristics in the evolution law of the potential slip region; and the grasping force closed-loop control parameters in this application are a set of adjustable coefficients used to directly determine the dynamic response characteristics and steady-state performance of the robot's dexterous finger grasping force control loop.

[0051] In another aspect, in some embodiments, this application provides a robot including a grasping control unit, referenced to... Figure 4 The figure is a schematic diagram of the structure of a grasping control unit 400 according to some embodiments of this application. The grasping control unit 400 includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire depth image information of the target object during the robot's grasping process, and to obtain contact pressure distribution data when the robot is currently grasping the target object. Processing module 402 in this application is mainly used to perform point cloud segmentation processing on the depth image information, thereby extracting the contour feature point set of the target object, and converting the contact pressure distribution data into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, it identifies the stable contact area and potential slip area in the grasping process. The processing module 402 described in this application is further configured to determine the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip area, and then determine the stability margin of the robot's dexterous finger grasping the target object at the current position through the force mapping matrix. The processing module 402 described in this application is further used to determine the dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process based on the stability margin and the real-time change trend of the contact state matrix, and to determine the adaptive adjustment requirements of the robot's current grasping state based on the rate of change of the dynamic coupling coefficient. The execution module 403 in this application is mainly used to dynamically adjust the closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions according to the adaptive adjustment requirements.

[0052] The foregoing has detailed examples of the robot and its grasping control method provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0053] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the robot grasping control method described above.

[0054] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the structure of a computer device implementing the robot grasping control method of this application. The robot grasping control method in the above embodiments can be... Figure 5 The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0055] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0056] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0057] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0058] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0059] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0060] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0061] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the robot grasping control method described above.

[0063] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0064] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A robot grasping control method, wherein, The robot has a multi-fingered dexterous hand, characterized by the following steps: The robot collects depth image information of the target object during the grasping process and obtains contact pressure distribution data when the robot is grasping the target object. The depth image information is processed by point cloud segmentation to extract the contour feature point set of the target object. At the same time, the contact pressure distribution data is converted into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, the stable contact area and potential slip area in the grasping process are identified. Based on the pressure gradient distribution in the stable contact area and the stress concentration characteristics in the potential slip area, the force mapping matrix of the target object during the grasping process is determined, and then the stability margin of the robot's dexterous finger grasping the target object in the current position is determined through the force mapping matrix. Based on the stability margin and the real-time change trend of the contact state matrix, the dynamic coupling coefficient of the interaction between the robot's dexterous fingers and the target object during the grasping process is determined, and the adaptive adjustment requirements of the robot's current grasping state are judged according to the rate of change of the dynamic coupling coefficient. The closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions are dynamically adjusted according to the adaptive adjustment requirements.

2. The method as described in claim 1, characterized in that, The point cloud segmentation process performed on the depth image information to extract the contour feature point set of the target object specifically includes: Obtain the 3D point cloud data corresponding to the depth image information; The target object point cloud and the background point cloud are segmented in the three-dimensional point cloud data; Extract the contour feature point set of the target object from the segmented point cloud of the target object.

3. The method as described in claim 1, characterized in that, Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, identifying stable contact areas and potential slippage areas during the grasping process specifically includes: The spatial position registration is performed between the contour feature point set and the contact state matrix; Determine the spatiotemporal stability characteristics of the contact pressure data represented by the contact state matrix within the registered overlapping region; Based on the spatiotemporal stability characteristics, the stable contact area and the potential slippage area during the grasping process are divided.

4. The method as described in claim 1, characterized in that, Determining the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution in the stable contact area and the stress concentration characteristics in the potential slip area specifically includes: The normal and tangential force components at each contact point during the grasping process are calculated based on the pressure gradient distribution of the stable contact area. The force distribution confidence weight of each contact point during the grasping process is determined based on the stress concentration characteristics of the potential slip region. The force mapping matrix of the target object during the grasping process is constructed by assigning confidence weights to the normal and tangential force components at all contact points.

5. The method as described in claim 1, characterized in that, Determining the stability margin of the robot's dexterous finger in grasping the target object at the current position using the force mapping matrix specifically includes: Perform matrix property analysis on the force mapping matrix; Stability quantification indicators are extracted based on matrix property analysis results; The stability quantification index is mapped to the stability margin of the robot's dexterous finger when grasping the target object in the current position.

6. The method as described in claim 1, characterized in that, The adaptive adjustment requirements for determining the robot's current grasping state based on the rate of change of the dynamic coupling coefficient specifically include: The rate of change of the dynamic coupling coefficient is compared with a preset adjustment threshold; Based on the comparison results, an adaptive adjustment requirement is generated for the robot's current grasping state.

7. The method as described in claim 1, characterized in that, The specific adjustments to the closed-loop control parameters of the gripping force when the robot's dexterous fingers perform a grasping action based on the aforementioned adaptive adjustment requirements include: The adjustment direction and magnitude level are determined based on the aforementioned adaptive adjustment requirements; Key feature parameters are extracted from the evolution patterns of potential slip regions; The closed-loop control parameters of the grasping force when the robot's dexterous finger performs a grasping action are adjusted based on the adjustment direction and amplitude level and the key feature parameters.

8. A robot comprising a grasping control unit, characterized in that, The grasping control unit includes: The acquisition module is used to acquire depth image information of the target object during the robot's grasping process and to obtain contact pressure distribution data when the robot is currently grasping the target object. The processing module is used to perform point cloud segmentation processing on the depth image information, thereby extracting the contour feature point set of the target object, and at the same time converting the contact pressure distribution data into a contact state matrix. Based on the spatiotemporal correlation between the contour feature point set and the contact state matrix, the module identifies the stable contact area and potential slip area during the grasping process. The processing module is further configured to determine the force mapping matrix of the target object during the grasping process based on the pressure gradient distribution of the stable contact area and the stress concentration characteristics of the potential slip area, and then determine the stability margin of the robot's dexterous finger grasping the target object at the current position through the force mapping matrix. The processing module is also used to determine the dynamic coupling coefficient between the robot's dexterous fingers and the target object during the grasping process based on the stability margin and the real-time change trend of the contact state matrix, and to determine the adaptive adjustment requirements of the robot's current grasping state based on the rate of change of the dynamic coupling coefficient. The execution module is used to dynamically adjust the closed-loop control parameters of the grasping force when the robot's dexterous fingers perform grasping actions according to the adaptive adjustment requirements.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store a computer program, and the processor being used to retrieve and run the computer program from the memory, causing the computer device to perform the robot grasping control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the robot grasping control method as described in any one of claims 1 to 7.

Images