Robot grasping pose planning method based on three-dimensional point cloud recognition

Abstract

The application discloses a robot grasping pose planning method based on three-dimensional point cloud recognition, comprising the following steps: collecting and preprocessing scene three-dimensional point cloud data to generate a standardized point cloud data set; constructing a deformation-driven multi-scale topological potential field to generate a continuous potential well area that can be deformed to avoid; extracting a high potential fragment as a candidate contact fragment to construct a four-layer homologous contact flow graph structure; performing topological shape-preserving optimal transmission and pairing to determine a feasible grasping area and a contact pair combination; based on an adaptive pose optimization strategy, outputting a grasping pose that meets multi-domain constraints; performing end control and feedback update judgment to realize grasping stability monitoring and self-correction. Through the deformation-driven potential field and multi-layer cross-domain optimization, the application realizes accurate planning and stable control of the robot grasping pose in a complex scene.

Description

Technical Field

[0001] This invention relates to the fields of robot intelligent control and three-dimensional visual perception technology, and in particular to a robot grasping pose planning method based on three-dimensional point cloud recognition. Background Technology

[0002] With the development of industrial robots and 3D vision sensing technology, target recognition and grasping methods based on 3D point clouds have been widely used in scenarios such as logistics sorting, automated assembly, and flexible manufacturing. Existing technologies typically employ depth cameras or LiDAR to acquire 3D point cloud data of the target scene. Preprocessing techniques such as filtering, downsampling, and normal estimation are used to obtain a relatively regular point cloud. This is then combined with template matching, pose estimation algorithms, or deep learning networks to identify the target object and provide grasping pose planning results. These methods can accomplish basic grasping tasks when the target shape is regular and the environment is relatively simple.

[0003] However, in complex and unknown scenarios, existing grasping pose planning methods based on 3D point clouds generally focus on static geometric features, analyzing only information such as the object's surface shape, normal, and curvature. This makes it difficult to accurately reflect the deformation characteristics of the object under force. When grasping thin-walled, flexible, or locally fragile objects, problems such as improper selection of grasping points and excessive deformation during grasping leading to detachment or damage can easily occur. Existing methods mostly score grasping candidate points or grasping postures at a single level, lacking collaborative modeling of geometric relationships, mechanical constraints, topological connectivity, and dynamic feedback.

[0004] Existing grasping pose optimization methods mostly perform local optimization only within the geometric or mechanical domains, failing to effectively integrate multi-level structural information with visual and force feedback during execution. This results in insufficient robustness of grasping planning under conditions of sensor noise, registration errors, and environmental uncertainties, making it difficult to adjust grasping strategies in a timely manner and to perform backtracking or replanning. Existing technologies also suffer from problems in complex and unknown scenarios, such as difficulty in accurately identifying feasible grasping regions, inability to handle grasping instability caused by object deformation, and unreliable grasping pose planning due to the lack of multi-level structural constraints and cross-domain adaptive optimization.

[0005] Therefore, how to provide a robot grasping pose planning method based on 3D point cloud recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a robot grasping pose planning method based on 3D point cloud recognition. This invention comprehensively utilizes 3D point cloud processing, topological potential energy modeling, graph structure mapping, and cross-domain adaptive optimization techniques. By constructing a deformation-driven multi-scale topological potential energy field, establishing a multi-layer homogeneous contact flow graph structure, and introducing a cross-domain adaptive posture optimization strategy, it achieves high-precision grasping planning and dynamic stability control of the robot in complex and unknown environments. This invention can recognize the 3D morphology and structural features of target objects, automatically identify graspable areas, generate optimal grasping poses, and continuously correct the grasping posture and contact force distribution in real time based on visual and force feedback during the grasping process, ensuring the stability and safety of the grasping process. This invention possesses advantages such as strong robustness, high recognition accuracy, strong adaptability to flexible and irregular objects, and high grasping success rate.

[0007] The robot grasping pose planning method based on 3D point cloud recognition according to embodiments of the present invention includes:

[0008] Collect 3D point cloud data of the target scene, perform preprocessing on the 3D point cloud data, and obtain a standardized point cloud dataset;

[0009] Based on a standardized point cloud dataset, a deformation-driven multi-scale topological potential energy field is constructed. Neighborhood analysis is performed on each local region of the point cloud to calculate the geometric stiffness tensor, forming a deformation response field. The potential energy distribution is then smoothed by convolution at multiple scales to generate a continuous potential well region that can be deformed and avoided.

[0010] High-potential connected segments are extracted from continuous potential well regions as candidate contact segments, and a multi-layer homogeneous contact flow graph structure is constructed. The structural correspondence between each layer is established through the homogeneous mapping matrix.

[0011] On a multi-layer homogeneous contact flow graph structure, a joint pairing solution of topological conformal optimal transport and physical constraints is performed. The cross-layer coupling mapping matrix is ​​obtained by consistent alignment of the Laplace spectrum. The cost tensor and constraint set are constructed, the sparse pairing scheme is solved, the contact pair combination is output, and the feasible grasping region is determined.

[0012] Based on the cross-domain adaptive pose optimization strategy, pose optimization calculations are performed on the feasible grasping area. Quaternions and spiral parameters are used as joint pose variables to perform a multi-domain adaptive optimization process and output the optimal grasping pose result.

[0013] The robot's end effector performs motion planning and control based on the optimal grasping pose, collects visual and force feedback data in real time to update the deformation response field and potential energy distribution, and triggers a backtracking and replanning process when the grasping stability margin is detected to be lower than a preset threshold, thus completing the stable grasping of the target object.

[0014] Optionally, the three-dimensional point cloud data includes spatial coordinate information, surface reflection intensity information, object texture grayscale information, and corresponding normal vector information collected by a depth camera, structured light sensor, or lidar.

[0015] Optionally, the preprocessing of the 3D point cloud data refers to performing statistical filtering, voxel downsampling, normal estimation, coordinate normalization, and extrinsic parameter calibration on the 3D point cloud data.

[0016] Optionally, generating the deformable, evasive, continuous potential well region includes:

[0017] In the standardized point cloud dataset, the spatial neighborhood range is determined for each point cloud sample. Statistical analysis is performed on the spatial distribution, surface normal direction, curvature change and point density of the point cloud in each neighborhood to establish a set of local geometric features.

[0018] Based on the set of local geometric features, the distribution of the main direction, the extent of expansion along the normal and tangential directions, and the local thickness changes of each point cloud neighborhood are comprehensively judged to obtain the geometric stiffness feature parameters that characterize the deformation resistance of the point cloud neighborhood. According to the value range of the geometric stiffness feature parameters, the point cloud is divided into high stiffness region, medium stiffness region and low stiffness region.

[0019] Based on the normal direction, curvature change and geometric stiffness characteristic parameters of each point cloud, a deformation response intensity value is assigned to each point, and a three-dimensional deformation response field is constructed on the entire standardized point cloud dataset.

[0020] Based on the three-dimensional deformation response field, an initial potential energy value is assigned to each point cloud location. The initial potential energy value takes into account the distance from the point to the object boundary, the local geometric continuity, and the deformation response intensity. Neighborhoods with different radii are selected at multiple preset spatial scales, and weighted aggregation is performed on the initial potential energy values ​​at each scale.

[0021] The weighted aggregation results at multiple scales are superimposed to obtain the multi-scale topological potential energy distribution on the standardized point cloud dataset. Threshold segmentation and connectivity analysis are performed on the multi-scale topological potential energy distribution to retain point cloud regions with potential energy values ​​higher than the preset threshold and spatial connectivity. These point cloud regions are then used as continuous potential well regions in the deformation-driven multi-scale topological potential energy field.

[0022] Optionally, the construction of the multi-layer homogeneous contact flow graph structure includes:

[0023] Connectivity analysis is performed on the continuous potential well region. Based on the spatial distance and adjacency relationship between point cloud samples, the continuous potential well region is divided into several independent connected segments to obtain a set of candidate contact segments. Each candidate contact segment consists of high potential energy point cloud samples that are spatially connected.

[0024] For each candidate contact segment in the candidate contact segment set, the geometric attribute parameter set, mechanical attribute parameter set, topological attribute parameter set, and dynamic attribute parameter set of the point cloud samples within the segment are statistically analyzed.

[0025] Based on candidate contact segments and their corresponding sets of geometric, mechanical, topological, and dynamic attribute parameters, a multi-layer homogeneous contact flow graph structure is constructed. In the geometric layer, candidate contact segments are used as nodes and connections are established according to spatial adjacency. In the mechanical layer, candidate contact segments are used as nodes and connections are established according to force coupling. In the topological layer, candidate contact segments are used as nodes and connections are established according to structural connectivity. In the dynamic perception layer, candidate contact segments are used as nodes and connections are established according to the correlation between visual and force perception changes.

[0026] In the multi-layer homogeneous contact flow graph structure, each candidate contact segment is assigned a unique identifier in the geometric layer, mechanical layer, topological layer and dynamic perception layer respectively. A one-to-one correspondence between the four layers is established through the unique identifiers of the same candidate contact segment in different layers.

[0027] Optionally, the combination of output contact pairs and determination of feasible grasping areas includes:

[0028] Spectral features for characterizing the intralayer structure are extracted from the geometric layer, mechanical layer, topological layer and dynamic sensing layer respectively. The four layers are aligned based on the spectral features to generate cross-layer coupling mapping results for describing the interlayer correspondence. The corresponding index of the same candidate contact segment in different layers is recorded.

[0029] For any two candidate contact segments, read the potential energy value and trend of the multi-scale topological potential energy field at the segment, the opposition relationship between segment normals, the complementary relationship between segment curvatures, the matching relationship between the distance between segments and the gripper opening, and the occlusion margin of the visible reachable segments to obtain the comprehensive pairing value of each pair of candidate contact segments.

[0030] For each fragment pair, define a set of physical feasibility constraints, including:

[0031] The tangential contact force must not exceed the product of the coefficient of friction and the normal contact force;

[0032] The robot's joint angles and end-effector poses must meet the kinematic reachability requirements.

[0033] A safe distance must be maintained between the fragment and the environment or other fragments to avoid collisions;

[0034] The segments can form a wrench closed loop under the action of force and torque to support gripping stability;

[0035] Fragment pairs that do not meet the physical feasibility constraints are removed;

[0036] On a set of fragment pairs that meet physical feasibility, a sparse pairing solution process is established with the goal of minimizing the total pairing cost. The supply and demand limits for each fragment are set, the number of effective pairings is limited to not exceeding a preset sparsity threshold, and the cross-layer coupling mapping result is used as a consistency constraint so that the selected pairings simultaneously meet the intra-layer adjacency relationship and the inter-layer correspondence relationship, thereby obtaining a globally consistent pairing scheme.

[0037] Based on the pairing scheme, fragment pairs with a pairing strength not lower than a preset threshold are selected to form contact pair combinations. Then, the contact pair combinations are connected and closure is performed on the geometric and topological layers to obtain feasible grasping regions.

[0038] Optionally, the output of the optimal grasping pose result includes:

[0039] A set of candidate poses is generated within the feasible grasping area. The pose variables are initialized by using quaternions to represent rotations, helical parameters to represent fine-tuning, and translation vectors. A set of four domain weights for the geometric domain, mechanical domain, topological domain, and dynamic potential field domain is established and the initial settings are completed.

[0040] The candidate poses are evaluated, including:

[0041] Candidate poses are evaluated in the geometric domain, and a sequence of geometric consistency indices is generated based on the energy gradient alignment of the multi-scale topological potential energy distribution, the consistency of the contact segment normal, and the matching relationship between the distance and the incident direction range.

[0042] Candidate poses are evaluated in the mechanical domain, and a sequence of mechanical stability indices is generated based on force balance, friction constraints, and wrench space coverage.

[0043] The candidate poses are evaluated in the topological domain, and a sequence of topological consistency indicators is generated based on the connectivity between contact segments, loop structure, matching with the topological layer adjacency relationship in the multi-layer homogeneous contact flow graph structure, and the integrity of the boundary of the feasible grabbing region.

[0044] Candidate poses are evaluated in a dynamic potential field, and a dynamic response index sequence is generated based on the rate of change of potential energy, visual change, and force change on the approach trajectory.

[0045] The adaptive weighting and dual-loop optimization process of multi-domain confidence gating is implemented. The outer loop automatically updates the weights of the four domains based on the temporal stability and sensing consistency of the four domain index sequences. The inner loop iterates the joint pose variables and contact force allocation alternately without changing the cross-layer coupling mapping relationship and physical feasibility constraints. When the judgment results between the geometric domain, mechanical domain and topological domain conflict, the stability condition of the dynamic potential field is maintained first, and the amplitude limiting adjustment is performed on the other three domains.

[0046] Under constraints such as accessibility, friction, collision safety distance, clamp opening and cross-layer consistency, the candidate pose set is screened and refined to determine the candidate pose with the minimum comprehensive cost under the current four-domain weight setting, and the corresponding contact force distribution and end-point incident direction are recorded simultaneously.

[0047] Based on the uncertainty assessment results, a robust stability margin is generated. When the robust stability margin is not lower than the preset threshold, the optimal grasping pose is output for execution. When the robust stability margin is lower than the preset threshold, the three domain weights are adjusted sequentially, the contact segment combination is replaced, or the incident angle range is reduced, and the optimization is repeated until the preset threshold is met.

[0048] Optionally, the stable grasping of the target object includes:

[0049] Read the optimal grasping pose, generate motion trajectories including approach pose, pre-grab pose, grasp pose and withdrawal pose, complete trajectory discretization in joint space and Cartesian space, set upper limits for velocity and acceleration, set clamp opening and grasping force, and perform collision detection with environmental constraints.

[0050] The motion control of the end effector is performed along the motion trajectory. Visual data, force data and end position odometry data are collected in real time. The timestamp synchronization and coordinate system alignment are completed. The end incident direction deviation, contact force deviation and position deviation are calculated based on the collected data and the visual and force state quantities are output.

[0051] The deformation response field and multi-scale topological potential energy distribution are updated online based on visual and force state quantities. The robust stability margin is evaluated based on the updated deformation response field and multi-scale topological potential energy distribution. When the robust stability margin is not lower than the preset threshold, the current trajectory continues to be executed. When the robust stability margin is lower than the preset threshold, the rollback process is triggered, the incident angle range is reduced, or the contact segment is replaced and the motion trajectory is regenerated.

[0052] During the grasping and evacuation phases, closed-loop corrections are performed on the gripper opening, grasping force, and end-effector pose to achieve target lifting and attitude stabilization. Execution logs, visual and force keyframes, and updated deformation response fields and multi-scale topological potential energy distributions are recorded, thus concluding the grasping process.

[0053] The beneficial effects of this invention are:

[0054] This invention effectively overcomes the shortcomings of traditional geometric feature-based grasping methods in deformation recognition and mechanical constraint modeling by introducing a deformation-driven multi-scale topological potential energy field. Through convolutional smoothing and neighborhood stiffness analysis of the point cloud potential energy in a multi-scale space, the system can automatically distinguish between rigid, flexible, and critical transition regions. This proactively avoids easily deformable or fragile structures during the grasping planning stage, ensuring the stability and reliability of grasping point selection. This method can accurately identify graspable regions even in complex and unknown environments, providing more physically constrained initial conditions for subsequent pose planning.

[0055] The multi-layered homogeneous contact flow graph structure constructed in this invention establishes homogeneous mapping relationships between the geometric layer, mechanical layer, topological layer, and dynamic perception layer, integrating the geometric distribution, force characteristics, structural connectivity, and dynamic feedback of the grasping points. This structure not only enables information transmission and constraint coordination between multiple layers but also achieves global optimization pairing between contact points through a topology-preserving optimal transmission strategy, thereby significantly improving the overall coordination of grasping pose planning and its adaptability to complex object structures. Compared to traditional single-layer models, this invention exhibits higher accuracy and robustness when dealing with occlusion, irregular shapes, and multi-contact-point grasping.

[0056] The cross-domain adaptive attitude optimization strategy proposed in this invention jointly optimizes the features of the geometric domain, mechanical domain, topological domain, and dynamic potential field domain, and combines real-time visual and force feedback to achieve dynamic self-correction of attitude and force distribution. When deformation exceeds limits, slippage risk occurs, or stability margin decreases during the grasping process, the system can automatically trigger backtracking and replanning, realizing adaptive closed-loop control throughout the process. It can maintain high stability and high success rate under various complex working conditions, and shows significant advantages, especially in the grasping of flexible parts, thin-walled parts, and locally fragile objects, effectively improving the intelligence level and operational safety of the robot grasping system. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 This is a flowchart of the robot grasping pose planning method based on 3D point cloud recognition proposed in this invention;

[0059] Figure 2 This is a schematic diagram of the hierarchical relationship and homogeneous mapping of the multi-layer homogeneous contact flow graph structure of the robot grasping pose planning method based on three-dimensional point cloud recognition proposed in this invention. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0061] refer to Figure 1 and Figure 2 A robot grasping pose planning method based on 3D point cloud recognition includes:

[0062] Collect 3D point cloud data of the target scene, perform preprocessing on the 3D point cloud data, and obtain a standardized point cloud dataset;

[0063] Based on a standardized point cloud dataset, a deformation-driven multi-scale topological potential energy field is constructed. Neighborhood analysis is performed on each local region of the point cloud to calculate the geometric stiffness tensor, forming a deformation response field. The potential energy distribution is then smoothed by convolution at multiple scales to generate a continuous potential well region that can be deformed and avoided.

[0064] High-potential connected segments are extracted from continuous potential well regions as candidate contact segments, and a multi-layer homogeneous contact flow graph structure is constructed. The structural correspondence between each layer is established through the homogeneous mapping matrix.

[0065] On a multi-layer homogeneous contact flow graph structure, a joint pairing solution of topological conformal optimal transport and physical constraints is performed. The cross-layer coupling mapping matrix is ​​obtained by consistent alignment of the Laplace spectrum. The cost tensor and constraint set are constructed, the sparse pairing scheme is solved, the contact pair combination is output, and the feasible grasping region is determined.

[0066] Based on the cross-domain adaptive pose optimization strategy, pose optimization calculations are performed on the feasible grasping area. Quaternions and spiral parameters are used as joint pose variables to perform a multi-domain adaptive optimization process and output the optimal grasping pose result.

[0067] The robot's end effector performs motion planning and control based on the optimal grasping pose, collects visual and force feedback data in real time to update the deformation response field and potential energy distribution, and triggers a backtracking and replanning process when the grasping stability margin is detected to be lower than a preset threshold, thus completing the stable grasping of the target object.

[0068] In this embodiment, the three-dimensional point cloud data includes spatial coordinate information, surface reflection intensity information, object texture grayscale information, and corresponding normal vector information collected by a depth camera, structured light sensor, or lidar.

[0069] In this embodiment, the preprocessing of the 3D point cloud data refers to performing statistical filtering, voxel downsampling, normal estimation, coordinate normalization, and extrinsic parameter calibration on the 3D point cloud data.

[0070] In this embodiment, generating a deformable, evasive, continuous potential well region includes:

[0071] In the standardized point cloud dataset, the spatial neighborhood range is determined for each point cloud sample. Statistical analysis is performed on the spatial distribution, surface normal direction, curvature change and point density of the point cloud in each neighborhood to establish a set of local geometric features.

[0072] Based on a set of local geometric features, the distribution along the main direction, the extent of expansion along the normal and tangential directions, and the local thickness variation of each point cloud neighborhood are comprehensively judged to obtain geometric stiffness characteristic parameters that characterize the deformation resistance of the point cloud neighborhood. The point cloud is then divided into high-stiffness, medium-stiffness, and low-stiffness regions according to the value range of the geometric stiffness characteristic parameters. Specifically, this division of the point cloud into high-stiffness, medium-stiffness, and low-stiffness regions according to the value range of the geometric stiffness characteristic parameters is as follows:

[0073] When the value of the geometric stiffness characteristic parameter is higher than the first threshold, it is marked as a high stiffness region, which is used to represent the point cloud part where the structure is stable and the deformation is weak after being subjected to force.

[0074] When the value of the geometric stiffness characteristic parameter is between the first threshold and the second threshold, it is marked as a medium stiffness region, which is used to represent the part of the point cloud that has finite deformation after being subjected to force.

[0075] When the value of the geometric stiffness characteristic parameter is lower than the second threshold, it is marked as a low stiffness region, which is used to represent the part of the point cloud that is prone to large deformation or stress concentration.

[0076] Based on the normal direction, curvature change and geometric stiffness characteristic parameters of each point cloud, a deformation response intensity value is assigned to each point, and a three-dimensional deformation response field is constructed on the entire standardized point cloud dataset.

[0077] Based on the three-dimensional deformation response field, an initial potential energy value is assigned to each point cloud location. The initial potential energy value takes into account the distance from the point to the object boundary, the local geometric continuity, and the deformation response intensity. Neighborhoods with different radii are selected at multiple preset spatial scales, and weighted aggregation is performed on the initial potential energy values ​​at each scale.

[0078] The weighted aggregation results at multiple scales are superimposed to obtain the multi-scale topological potential energy distribution on the standardized point cloud dataset. Threshold segmentation and connectivity analysis are performed on the multi-scale topological potential energy distribution to retain point cloud regions with potential energy values ​​higher than the preset threshold and spatial connectivity. These point cloud regions are then used as continuous potential well regions in the deformation-driven multi-scale topological potential energy field.

[0079] In this embodiment, the construction of the multi-layer homogeneous contact flow graph structure includes:

[0080] Connectivity analysis is performed on the continuous potential well region. Based on the spatial distance and adjacency relationship between point cloud samples, the continuous potential well region is divided into several independent connected segments to obtain a set of candidate contact segments. Each candidate contact segment consists of high potential energy point cloud samples that are spatially connected.

[0081] For each candidate contact segment in the candidate contact segment set, the geometric attribute parameter set, mechanical attribute parameter set, topological attribute parameter set, and dynamic attribute parameter set of the point cloud samples within the segment are statistically analyzed. The geometric attribute parameter set includes the segment center position, average normal direction, and curvature distribution. The mechanical attribute parameter set includes the contact direction, achievable clamping distance range, and friction-related parameters. The topological attribute parameter set includes connectivity, adjacency relationship, and boundary shape features. The dynamic attribute parameter set includes the visual feedback change, the force feedback change, and the feature quantity of multi-scale topological potential energy changing over time.

[0082] Based on candidate contact segments and their corresponding sets of geometric, mechanical, topological, and dynamic attribute parameters, a multi-layer homogeneous contact flow graph structure is constructed. In the geometric layer, candidate contact segments are used as nodes and connections are established according to spatial adjacency. In the mechanical layer, candidate contact segments are used as nodes and connections are established according to force coupling. In the topological layer, candidate contact segments are used as nodes and connections are established according to structural connectivity. In the dynamic perception layer, candidate contact segments are used as nodes and connections are established according to the correlation between visual and force perception changes.

[0083] In the multi-layer homogeneous contact flow graph structure, each candidate contact segment is assigned a unique identifier in the geometric layer, mechanical layer, topological layer and dynamic perception layer respectively. A one-to-one correspondence between the four layers is established through the unique identifiers of the same candidate contact segment in different layers.

[0084] In this embodiment, the process of combining output contact pairs and determining a feasible grasping area includes:

[0085] Spectral features for characterizing the intralayer structure are extracted from the geometric layer, mechanical layer, topological layer and dynamic sensing layer respectively. The four layers are aligned based on the spectral features to generate cross-layer coupling mapping results for describing the interlayer correspondence. The corresponding index of the same candidate contact segment in different layers is recorded.

[0086] For any two candidate contact segments, read the potential energy value and trend of the multi-scale topological potential energy field at the segment, the opposition relationship between segment normals, the complementary relationship between segment curvatures, the matching relationship between the distance between segments and the gripper opening, and the occlusion margin of the visible reachable segments to obtain the comprehensive pairing value of each pair of candidate contact segments.

[0087] For each fragment pair, define a set of physical feasibility constraints, including:

[0088] The tangential contact force must not exceed the product of the coefficient of friction and the normal contact force;

[0089] The robot's joint angles and end-effector poses must meet the kinematic reachability requirements.

[0090] A safe distance must be maintained between the fragment and the environment or other fragments to avoid collisions;

[0091] The segments can form a wrench closed loop under the action of force and torque to support gripping stability;

[0092] Fragment pairs that do not meet the physical feasibility constraints are removed;

[0093] On a set of fragment pairs that meet physical feasibility, a sparse pairing solution process is established with the goal of minimizing the total pairing cost. The supply and demand limits for each fragment are set, the number of effective pairings is limited to not exceeding a preset sparsity threshold, and the cross-layer coupling mapping result is used as a consistency constraint so that the selected pairings simultaneously meet the intra-layer adjacency relationship and the inter-layer correspondence relationship, thereby obtaining a globally consistent pairing scheme.

[0094] Based on the pairing scheme, fragment pairs with a pairing strength not lower than a preset threshold are selected to form contact pair combinations. Then, the contact pair combinations are connected and closure is performed on the geometric and topological layers to obtain feasible grasping regions.

[0095] In this embodiment, the step of outputting the optimal grasping pose result includes:

[0096] A set of candidate poses is generated within the feasible grasping area. The pose variables are initialized by using quaternions to represent rotations, helical parameters to represent fine-tuning, and translation vectors. A set of four domain weights for the geometric domain, mechanical domain, topological domain, and dynamic potential field domain is established and the initial settings are completed.

[0097] The candidate poses are evaluated, including:

[0098] Candidate poses are evaluated in the geometric domain. A geometric consistency index sequence is generated based on the energy gradient alignment of the multi-scale topological potential distribution, the consistency of the contact segment normal, and the matching relationship between the distance and the incident direction range. Specifically, the generation of the geometric consistency index sequence is as follows:

[0099] Calculate the multi-scale potential energy gradient direction of each contact segment in the candidate pose, determine the degree of difference in the angle between the gradient and the principal normal of the local surface, and use the degree of energy gradient alignment as the first type of geometric consistency sub-index.

[0100] The normal difference between each contact segment in the candidate pose is statistically analyzed, and the coordination of the geometric surface opposition relationship between segments is characterized by the variance of the normal angle, thus generating a second type of geometric consistency sub-index.

[0101] Compare whether the range of the candidate pose's offset and the incident direction meets the preset geometric matching constraints. Normalize the poses that deviate from the threshold range to form a third type of geometric consistency sub-index. Combine the three types of sub-indexes according to their weights to form a geometric consistency index sequence.

[0102] The candidate pose is evaluated in the mechanical domain, and a sequence of mechanical stability indices is generated based on force balance, friction constraints, and wrench space coverage. Specifically, the generation of the mechanical stability indices sequence is as follows:

[0103] Calculate the force distribution of each contact segment under the candidate pose, determine the balance of the contact force in the normal and tangential components, and take the force imbalance deviation as the first type of mechanical stability sub-index.

[0104] Based on the contact surface normal and friction coefficient range, it is determined whether the contact force is within the static friction constraint. Contact points that exceed the slip threshold are penalized and scored to generate a second type of mechanical stability sub-index.

[0105] In the three-dimensional wrench space, the coverability of each contact force and torque component is analyzed, the volume ratio of the stable domain of the candidate pose is calculated, which is used as the third type of mechanical stability sub-index, and the three types of sub-indexes are combined according to weight to form a mechanical stability index sequence.

[0106] Candidate poses are evaluated in the topological domain. A sequence of topological consistency indicators is generated based on the connectivity between contact segments, loop structure, matching with the topological layer adjacency relationships in the multi-layer homogeneous contact flow graph structure, and the integrity of the feasible grasping region boundary. Specifically, the generation of the topological consistency indicator sequence involves:

[0107] The adjacency matrix structure of each contact segment in the topology layer under the candidate pose is analyzed, and the connectivity difference with the original topology graph is calculated. The connectivity preservation rate is used as the first type of topology consistency sub-index.

[0108] The number and distribution of loops corresponding to candidate poses are detected to determine whether they match the loop structure of the original topology layer. Penalties are applied to loop breaks or redundancy, and a second type of topology consistency sub-index is generated.

[0109] The boundary closure degree of feasible capture areas is compared with the topology layer boundary integrity requirements. The boundary gap rate is normalized and quantified to generate a third type of topology consistency sub-indicator. The three types of sub-indicators are combined according to preset weights to form a topology consistency index sequence.

[0110] Candidate poses are evaluated in a dynamic potential field. A dynamic response index sequence is generated based on the rate of change of potential energy, visual change, and force change on the approach trajectory. Specifically, the generation of the dynamic response index sequence involves:

[0111] The potential energy change sequence is sampled along the approach trajectory of the robot's end effector, the rate of potential energy change per unit displacement is calculated, and a higher score is assigned to the trajectory segment with stable change, forming the first type of dynamic response sub-index.

[0112] During the capture process, continuous frame visual data is extracted, the texture and depth changes of the target area are calculated, the consistency of pose tracking is quantified by visual change stability, and a second type of dynamic response sub-index is generated.

[0113] Force signals are collected synchronously, and the torque fluctuation and impact response amplitude during the contact process are analyzed. Fluctuations exceeding the stability threshold are normalized and penalized to form a third type of dynamic response sub-indicator. The three types of sub-indicators are then fused together by time weighting to form a dynamic response index sequence.

[0114] The combined effect of the geometric consistency index sequence, mechanical stability index sequence, topological consistency index sequence, and dynamic response index sequence is to evaluate the reliability and stability of candidate grasping poses from multiple dimensions, realize the synergistic constraints between geometric shape, force state, structural topology and dynamic feedback. Through the multi-sequence fusion evaluation mechanism, this invention can achieve precise matching of grasping point and posture in complex, deformable or flexible object environments, ensure the geometric rationality, mechanical balance and structural connectivity of the contact pair, and at the same time have the ability to adaptively adjust during dynamic grasping, thereby improving the stability, anti-disturbance and success rate of robot grasping process.

[0115] The adaptive weighting and dual-loop optimization process of multi-domain confidence gating is implemented. The outer loop automatically updates the weights of the four domains based on the temporal stability and sensing consistency of the four domain index sequences. The inner loop iterates the joint pose variables and contact force allocation alternately without changing the cross-layer coupling mapping relationship and physical feasibility constraints. When the judgment results between the geometric domain, mechanical domain and topological domain conflict, the stability condition of the dynamic potential field is maintained first, and the amplitude limiting adjustment is performed on the other three domains.

[0116] Under constraints such as accessibility, friction, collision safety distance, clamp opening and cross-layer consistency, the candidate pose set is screened and refined to determine the candidate pose with the minimum comprehensive cost under the current four-domain weight setting, and the corresponding contact force distribution and end-point incident direction are recorded simultaneously.

[0117] Based on the uncertainty assessment results, a robust stability margin is generated. When the robust stability margin is not lower than the preset threshold, the optimal grasping pose is output for execution. When the robust stability margin is lower than the preset threshold, the three domain weights are adjusted sequentially, the contact segment combination is replaced, or the incident angle range is reduced, and the optimization is repeated until the preset threshold is met.

[0118] In this embodiment, the stable grasping of the target object includes:

[0119] Read the optimal grasping pose, generate motion trajectories including approach pose, pre-grab pose, grasp pose and withdrawal pose, complete trajectory discretization in joint space and Cartesian space, set upper limits for velocity and acceleration, set clamp opening and grasping force, and perform collision detection with environmental constraints.

[0120] The motion control of the end effector is performed along the motion trajectory. Visual data, force data and end position odometry data are collected in real time. The timestamp synchronization and coordinate system alignment are completed. The end incident direction deviation, contact force deviation and position deviation are calculated based on the collected data and the visual and force state quantities are output.

[0121] The deformation response field and multi-scale topological potential energy distribution are updated online based on visual and force state variables. The robust stability margin is evaluated based on the updated deformation response field and multi-scale topological potential energy distribution. If the robust stability margin is not lower than a preset threshold, execution continues according to the current trajectory. If the robust stability margin is lower than the preset threshold, a backtracking process is triggered, the incident angle range is reduced, or the contact segment is replaced and the motion trajectory is regenerated. Specifically, the process of updating the deformation response field and multi-scale topological potential energy distribution online based on visual and force state variables and evaluating the robust stability margin based on the updated deformation response field and multi-scale topological potential energy distribution involves:

[0122] Based on continuously acquired visual frame sequences, the depth change and texture offset of the target surface are extracted and mapped to the corresponding point cloud positions, and the geometric change terms of the local deformation response field are updated in real time.

[0123] Based on the changes in contact force and torque collected by the force sensor, the force state of the point cloud neighborhood is incrementally corrected, and the force-related terms and energy gradient change terms in the multi-scale topological potential energy distribution are updated.

[0124] The updated deformation response field and multi-scale topological potential energy distribution are input into the robust stability margin calculation process to quantify the deformation offset, force fluctuation and energy change rate during the grasping process, generate robust stability margin and use it to determine whether to maintain the current trajectory or trigger backtracking and replanning.

[0125] During the grasping and evacuation phases, closed-loop corrections are performed on the gripper opening, grasping force, and end-effector pose to achieve target lifting and attitude stabilization. Execution logs, visual and force keyframes, and updated deformation response fields and multi-scale topological potential energy distributions are recorded, thus concluding the grasping process.

[0126] Example 1:

[0127] To verify the feasibility of this invention in practice, it was applied to a home appliance factory. The production line contained a mix of materials, including power cord cable ties, small thin-shell plastic parts, foam pads, and flexible packaging instructions. These materials generally exhibit characteristics such as thinness, easily deformable local structures, and uneven surface reflection. They also suffer from stacking, partial obstruction, and random orientation. A six-axis industrial robot, a 3D structured light camera, and a gripper end effector with a torque sensor were deployed on-site. Traditional gripping strategies based on simple geometric features often resulted in gripping points falling on thin edges or hollow areas, and materials being squeezed, deformed, or even slipping during the gripping process, leading to low gripping success rates and frequent adjustments.

[0128] In the aforementioned scenario, the method of this invention first acquires 3D point cloud data of the workbench area using a 3D structured light camera. Through preprocessing such as filtering, downsampling, and normal estimation, a standardized point cloud dataset aligned with the robot's base coordinate system is obtained. Subsequently, the spatial distribution, normal direction, curvature, and point density within the point cloud neighborhood are analyzed to construct a deformation-driven multi-scale topological potential energy field. Each point cloud location is assigned a potential energy value that balances geometric continuity and deformation response, automatically forming a potential well distribution that "avoids" thin-walled and flexible components. Based on this, the system extracts contact segments from high-potential-energy connected regions and constructs a multi-layered homogeneous contact flow graph structure including a geometric layer, a mechanical layer, a topological layer, and a dynamic perception layer. Through homogeneous mapping relationships, geometric information, force relationships, connected structures, and visual / force dynamic feedback at different levels are correlated. Next, a topology-conformal optimal transport and physical constraint joint pairing solution is performed on the multi-layer graph structure. Under the premise of considering physical constraints such as friction, accessibility, collision and wrench space, a set of contact pair combinations that have geometric rationality, mechanical stability and topological connectivity are selected to determine the feasible grasping area.

[0129] In the pose planning and execution phase, the system inputs the feasible grasping region into the cross-domain adaptive posture optimization strategy. Quaternions and helical parameters are used to jointly describe the robot's end-effector pose. Cross-domain weights from the geometric, mechanical, topological, and dynamic potential fields are introduced. Through multi-domain index evaluation and adaptive weighting, the grasping pose and contact force distribution are iteratively optimized to form the optimal grasping pose. Guided by the generated optimal grasping pose, the robot performs approach, gripping, and lifting actions. Visual and force feedback are collected in real time during this process and used to update the deformation response field and potential energy distribution. When the grasping stability margin calculated by the system is lower than a preset threshold, automatic backtracking and replanning are triggered to avoid forcibly completing the grasp under conditions of excessive deformation or unstable contact. This solves the problems of instability and slippage during grasping of flexible parts, thin-walled parts, and locally fragile structures mentioned in the background art.

[0130] To verify the practical effectiveness of the method of this invention, this embodiment conducted a comparative experiment with the traditional point cloud grasping method based on geometric features and the conventional deep learning grasping network method in the same production line environment. The experiment ran continuously for 5 shifts, with approximately 300 materials randomly placed in each shift, and a total of 1500 valid grasping data were collected.

[0131] Table 1. Comparison of different methods in grasping flexible and thin-walled materials.

[0132]

[0133] As can be seen from the data in Table 1, the robot grasping pose planning method based on 3D point cloud recognition proposed in this invention has significant performance advantages in grasping complex flexible materials and thin-walled objects. Taking a power cable tie bag as an example, the grasping success rate of traditional geometric feature methods is only 84.0%, with an average deformation of 1.35 mm during the grasping process. Problems such as slippage or deformation often occur due to improper selection of grasping points. Although deep learning grasping methods have made some improvements, they still suffer from insensitivity to deformation response. However, after adopting the method of this invention, the grasping success rate increases to 94.2%, the average deformation decreases to 0.96 mm, and the grasping posture stabilization time is shortened to 2.5 seconds, indicating that the system can effectively identify stiffness-stable regions and avoid high-deformation-risk areas.

[0134] For targets with complex and easily deformable structures, such as thin-shell plastic parts and foam pads, the method of this invention also demonstrates better adaptability. The success rate of gripping thin-shell plastic parts increased from 82.3% to 92.8% compared to the traditional method, and for foam pads, it increased from 80.5% to 91.6%, with the deformation of both types of materials reduced by approximately 0.4 to 0.5 mm. The accuracy of identifying feasible gripping areas was above 90%, indicating that through the synergistic effect of the deformation-driven multi-scale topological potential energy field and the multi-layer homogeneous contact flow graph structure, the system can accurately locate contact areas with sufficient structural support and balanced stress, achieving mechanical rationality and topological stability constraints at the gripping point.

[0135] The experimental results from the flexible packaging instructions show that the method of this invention maintains a 90.5% grasping success rate when handling extremely thin, highly reflective, and complex-edged flexible objects, far exceeding the 79.7% of traditional methods. The post-grasp posture stabilization time is shortened by approximately 0.5 seconds, indicating that the cross-domain adaptive posture optimization strategy, under the multi-source feedback weighting mechanism, effectively controls dynamic deformation during the grasping process, maintaining stable contact between the end effector and the target object. The average grasping success rate of this invention on four typical flexible and thin-walled materials reaches 92.3%, an improvement of approximately 10 percentage points compared to traditional methods, while the planning time remains within the range of 2.8–3.0 seconds, demonstrating excellent engineering feasibility and real-time performance. This invention not only solves the problems of easy instability in grasping flexible and thin-walled objects, insufficient recognition accuracy, and lack of mechanical-topological constraint coordination in existing technologies, but also achieves comprehensive improvements in robustness, grasping efficiency, and stability.

[0136] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A robot grasping pose planning method based on three-dimensional point cloud recognition, characterized in that, The method comprises the following steps: Collecting three-dimensional point cloud data of a target scene, preprocessing the three-dimensional point cloud data, and obtaining a standardized point cloud data set; Based on the standardized point cloud data set, a deformation-driven multi-scale topological potential energy field is constructed, the neighborhood of each point cloud local region is analyzed to calculate the geometric stiffness tensor, a deformation response field is formed, the potential energy distribution is convolved and smoothed under multi-scale, and a continuous potential well region that can be deformed to avoid is generated; Based on the continuous potential well region, a high potential energy connected segment is extracted as a candidate contact segment, a multi-layer homologous contact flow graph structure is constructed, and a structure correspondence relationship between layers is established through a homologous mapping matrix; Topological shape-preserving optimal transmission and physical constraint joint pairing solving are performed on the multi-layer homologous contact flow graph structure, a cross-layer coupling mapping matrix is obtained through Laplacian spectral consistent alignment, a cost tensor and a constraint set are constructed, a sparse pairing scheme is solved, and a contact pair combination is output and a feasible grasping region is determined; According to the cross-domain adaptive pose optimization strategy, pose optimization calculation is performed on the feasible grasping region, quaternions and screw parameters are used as joint pose variables, a multi-domain adaptive optimization process is performed, and an optimal grasping pose result is output; According to the optimal grasping pose, motion planning and control of the robot end effector are performed, visual and force feedback data are collected in real time to update the deformation response field and the potential energy distribution, and when it is detected that the grasping stability margin is lower than a preset threshold, a rollback and re-planning process is triggered to complete stable grasping of the target object; The method for generating a continuous potential well region that can be deformed to avoid comprises the following steps: In the standardized point cloud data set, the spatial neighborhood range of each point cloud sample is determined, the spatial distribution, surface normal direction, curvature change and point density of the point cloud in each neighborhood are statistically analyzed, and a local geometric feature set is established; Based on the local geometric feature set, the main direction distribution of each point cloud neighborhood, the extension degree along the normal and tangent directions, and the local thickness change are comprehensively judged to obtain geometric stiffness characteristic parameters representing the deformation resistance of the point cloud neighborhood, and the point cloud is divided into a high stiffness region, a medium stiffness region and a low stiffness region according to the value interval of the geometric stiffness characteristic parameters, wherein the point cloud is divided into a high stiffness region, a medium stiffness region and a low stiffness region according to the value interval of the geometric stiffness characteristic parameters, specifically: When the value of the geometric stiffness characteristic parameter is higher than a first threshold value, it is marked as a high stiffness region, which represents a point cloud part with stable structure and weak deformation under stress; When the value of the geometric stiffness characteristic parameter is between the first threshold value and a second threshold value, it is marked as a medium stiffness region, which represents a point cloud part that has limited deformation under stress; When the value of the geometric stiffness characteristic parameter is lower than the second threshold value, it is marked as a low stiffness region, which represents a stress concentrated point cloud part; According to the normal direction, curvature change and geometric stiffness characteristic parameter of each point cloud, a deformation response intensity value is assigned to each point, and a three-dimensional deformation response field on the entire standardized point cloud data set is constructed. On the basis of the three-dimensional deformation response field, an initial potential value is assigned to each point cloud position, the initial potential value simultaneously referring to a distance of a point to a boundary of an object, local geometric continuity and deformation response intensity, a neighborhood of different radii is selected under a plurality of preset spatial scales, and weighted aggregation is performed on the initial potential value in each scale; The weighted aggregation results under the plurality of scales are superimposed to obtain a multi-scale topological potential distribution on the standardized point cloud dataset, threshold segmentation and connectivity analysis are performed on the multi-scale topological potential distribution, a point cloud region with a potential value higher than a preset threshold and keeping spatial connectivity is retained, and the point cloud region is taken as a continuous potential well region in a multi-scale topological potential field driven by deformation.

2. The robot grasp pose planning method based on three-dimensional point cloud recognition according to claim 1, characterized in that, The three-dimensional point cloud data includes spatial coordinate information, surface reflection intensity information, object texture gray information and corresponding normal vector information collected by a depth camera, a structured light sensor or a laser radar.

3. The method of claim 1, wherein, The preprocessing of the three-dimensional point cloud data refers to statistical filtering, voxel downsampling, normal estimation, coordinate normalization and external parameter calibration of the three-dimensional point cloud data.

4. The method of claim 1, wherein, The multi-layer homologous contact flow graph structure is constructed, including: The connectivity analysis is performed on the continuous potential well region, the continuous potential well region is divided into a plurality of independent connected segments according to spatial distances and adjacency relationships between point cloud samples, a candidate contact segment set is obtained, and each candidate contact segment is composed of high-potential point cloud samples keeping spatial connectivity; The geometric attribute parameter set, the mechanical attribute parameter set, the topological attribute parameter set and the dynamic attribute parameter set of the point cloud samples in each candidate contact segment in the candidate contact segment set are respectively counted; Based on the candidate contact segment and the corresponding geometric attribute parameter set, the mechanical attribute parameter set, the topological attribute parameter set and the dynamic attribute parameter set, a multi-layer homologous contact flow graph structure is constructed, the candidate contact segment is taken as a node in a geometric layer and connection is established according to spatial adjacency relationship, the candidate contact segment is taken as a node in a mechanical layer and connection is established according to force coupling relationship, the candidate contact segment is taken as a node in a topological layer and connection is established according to structure connectivity relationship, and the candidate contact segment is taken as a node in a dynamic perception layer and connection is established according to visual and force change correlation relationship; In the multi-layer homologous contact flow graph structure, a unique identifier is respectively assigned to each candidate contact segment in the geometric layer, the mechanical layer, the topological layer and the dynamic perception layer, and a one-to-one correspondence relationship between the four layers is established through the unique identifier of the same candidate contact segment in different layers.

5. The method of claim 1, wherein, The output contact pair combination and determination of a feasible grasping region include: Spectrum features for representing structures in the layers are respectively extracted from the geometric layer, the mechanical layer, the topological layer and the dynamic perception layer, the four layers are aligned based on the spectrum features, a cross-layer coupling mapping result for describing the interlayer correspondence relationship is generated, and the corresponding indexes of the same candidate contact segment in different layers are recorded. For any two candidate contact segments, read the potential energy value and trend of the multi-scale topological potential field at the segment, the opposite relationship between the segment normal, the complementary relationship between the segment curvature, the matching relationship between the segment distance and the jaw opening, and the occlusion margin between the visual and reachable segments, to obtain the comprehensive pairing generation value of each pair of candidate contact segments; Set a set of physical feasibility constraints for each segment pair, including: The tangential contact force must not exceed the product of the friction coefficient and the normal contact force; The robot joint angle and end pose must satisfy the kinematic reachable range; The segment and the environment or other segments must satisfy the safety distance to avoid collision; The segment pair can form a wrench closed loop under the action of force and torque to support the grasping stability; Eliminate the segment pairs that do not meet the physical feasibility constraints; On the set of segment pairs that meet the physical feasibility, establish a sparse pairing solving process with the goal of minimizing the total pairing cost, set the upper limit of the supply and demand of each segment, limit the number of effective pairs to not exceed the pre-set sparsity threshold, and use the cross-layer coupling mapping result as a consistency constraint, so that the selected pairing meets both the intra-layer adjacency relationship and the inter-layer correspondence relationship, thereby obtaining a globally consistent pairing scheme; According to the pairing scheme, select the segment pairs with pairing strength not lower than the pre-set threshold to form a contact pair combination, and perform connected closure on the contact pair combination in the geometric layer and the topological layer to obtain a feasible grasping region.

6. The method of claim 1, wherein, The output optimal grasping pose result includes: Generate a candidate pose set in the feasible grasping region, initialize the joint pose variable using quaternion to represent rotation, screw parameter to represent fine tuning, and combine with the translation vector, establish a four-domain weight set of geometric domain, mechanical domain, topological domain, and dynamic potential field domain and complete the initial setting; Evaluate the candidate pose, including: Evaluate the candidate pose in the geometric domain, generate a geometric consistency index sequence according to the energy gradient alignment of the multi-scale topological potential distribution, the consistency of the contact segment normal, and the matching relationship of the jaw distance and the incident direction range; Evaluate the candidate pose in the mechanical domain, generate a mechanical stability index sequence according to the force balance, friction constraint, and wrench space coverage; Evaluate the candidate pose in the topological domain, generate a topological consistency index sequence according to the matching of the connected relationship between the contact segments, the loop structure, and the topological layer adjacency relationship in the multi-layer homogenous contact flow graph structure, as well as the integrity of the feasible grasping region boundary; Evaluate the candidate pose in the dynamic potential field domain, generate a dynamic response index sequence according to the potential energy change rate on the approaching trajectory, the visual change amount, and the force sense change amount; Perform adaptive weighting of multi-domain confidence threshold gating and double-loop optimization process, the outer loop automatically updates the four-domain weights according to the time series stability and sensing consistency of the four-domain index sequence, the inner loop alternately iterates the joint pose variable and contact force distribution without changing the cross-layer coupling mapping relationship and physical feasibility constraints, when the determination results between the geometric domain, the mechanical domain, and the topological domain conflict, the stable condition of the dynamic potential field domain is prioritized, and the remaining three domains are subjected to amplitude limiting adjustment; Under the constraints of reachability, friction, collision safety distance, gripper opening and cross-layer consistency, the candidate pose set is screened and refined to determine the candidate pose with the minimum comprehensive cost under the current four-domain weight setting, and the corresponding contact force distribution and end incident direction are recorded synchronously; A robust stability margin is generated according to the uncertainty evaluation result, and when the robust stability margin is not lower than the preset threshold, the optimal grasping pose is output for execution, and when the robust stability margin is lower than the preset threshold, the three-domain weight is adjusted, the contact segment combination is replaced or the incident angle range is reduced, and the optimization is repeated until the preset threshold is met.

7. The method of claim 1, wherein, The stable grasping of the target object is completed, including: The optimal grasping pose is read, and a motion trajectory including an approaching pose, a pre-grasping pose, a grasping pose and a retreat pose is generated, trajectory discretization in joint space and Cartesian space, speed upper limit and acceleration upper limit setting, gripper opening and grasping force setting, and collision detection with environmental constraints are completed; The end effector motion control is executed along the motion trajectory, real-time acquisition of visual data, force sensation data and end pose odometry data is completed, time stamp synchronization and coordinate system alignment are completed, end incident direction deviation, contact force deviation and pose deviation are calculated according to the acquired data, and visual and force sensation state quantities are output; The deformation response field and the multi-scale topological potential distribution are updated online according to the visual and force sensation state quantities, the robust stability margin is evaluated according to the updated deformation response field and the multi-scale topological potential distribution, and when the robust stability margin is not lower than the preset threshold, the current trajectory is continued to be executed, and when the robust stability margin is lower than the preset threshold, the rollback process is triggered, the incident angle range is reduced or the contact segment is replaced, and the motion trajectory is regenerated; In the grasping stage and the retreat stage, the gripper opening, the grasping force and the end pose are closed-loop corrected, the target is lifted and the attitude is stably maintained, the execution log, the visual and force sensation key frame, and the updated deformation response field and the multi-scale topological potential distribution are recorded, and the current grasping process is ended.

Images