Robot soft tissue multi-point contact shape control method and system based on vision

Abstract

The invention belongs to the technical field of robot control, and particularly discloses a robot soft tissue multi-point contact shape control method and system based on vision. Comprising the steps that soft tissue is discretized into a grid formed by n nodes, e feature points on the surface of the soft tissue are selected, and control variables are extracted from feature point position vectors according to task features; a plurality of candidate contact points are selected from nodes on the surface of the soft tissue, a contact point combination with the maximum affordability value is obtained based on a differentiable soft tissue deformation model according to the variable quantity of feature points when all possible unit actions are applied to the multiple contact points, and the combined contact point serves as the optimal contact point; and according to the optimal contact point, adopting an MPC framework to carry out closed-loop shape control so as to solve an optimal action sequence of the robot. According to the method, the optimal multi-point contact scheme can be automatically and quickly evaluated and selected according to specific task requirements, and the real-time, efficient and stable control process is realized.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, and more specifically, relates to a vision-based method and system for controlling the shape of multi-point contact in robot soft tissue. Background Technology

[0002] Robot-assisted surgery, especially minimally invasive surgery, has greatly improved the precision and flexibility of surgical procedures. Many surgical procedures require precise manipulation of soft tissues, such as traction, dissection, and repositioning, to expose the surgical field or prepare for subsequent operations (such as cutting and suturing). However, the irregular geometry, high deformability, and nonlinear mechanical properties of soft tissues present significant challenges to the automated operation of robots.

[0003] Currently, researchers have proposed various technical solutions to achieve shape control of soft tissue. One type is adaptive control, which addresses system uncertainties by updating model parameters online, such as using visual feedback to adjust controller parameters, but typically does not establish an explicit tissue model. Another type is model-based control, which uses a predictive model (such as a finite element model or a mass-spring model) to plan robot movements and uses visual feedback for state estimation and correction. Patent document CN117709116A uses a mass-spring-damped model and ADMM algorithm optimization to improve the solution speed of deformation models to a clinically acceptable level. In recent years, with the development of artificial intelligence, data-driven methods have also emerged, such as using deep learning networks to learn the deformation patterns and affordances of soft tissue from simulation or real data (i.e., where and how to manipulate to achieve the desired effect).

[0004] While the aforementioned existing technologies have promoted the development of soft tissue manipulation techniques to some extent, they still have significant limitations. Adaptive control methods, although not requiring precise modeling, typically have slow convergence speeds and struggle to guarantee stability in complex systems like soft tissue, making it difficult to assess the analytical impact of online parameter updates on control performance. The model control method employed in Chinese patent CN117709116A, limited by the solution speed of biological tissue models, cannot adjust maneuvers in real-time online, thus failing to reduce errors during the operation. Data-driven methods heavily rely on large-scale, diverse, and precisely labeled training datasets. In the field of surgery, acquiring such data is not only costly but also ethically and practically challenging, limiting the generalization ability and practicality of these methods in real surgical environments. Existing methods assume that the contact points during the operation are pre-given, and how to autonomously and intelligently select the optimal contact points based on task requirements to efficiently complete the deformation task (i.e., the availability estimation problem) remains poorly addressed.

[0005] Therefore, existing technologies for soft tissue manipulation, especially in the intelligent selection of contact points based on task requirements and the realization of real-time, precise, and stable shape control, still face a series of technical problems that urgently need to be solved, such as computational efficiency, data dependence, stability, and interpretability. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a vision-based method and system for multi-point contact shape control of robotic soft tissue. Based on a differentiable soft tissue deformation model, it proposes an analytical availability estimation algorithm to select the optimal combination of contact points from a series of candidate contact points. Then, using the selected contact points as input, within a model predictive control (MPC) framework, the robot's motion sequence is optimized to drive the soft tissue shape to a target state. This method requires no offline data training, achieving real-time, efficient, and stable control.

[0007] To achieve the above objectives, according to one aspect of the present invention, a vision-based method for controlling the shape of multi-point contact in robotic soft tissue is proposed, characterized by comprising the following steps:

[0008] Step 1: Discretize the soft tissue into a grid consisting of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vectors according to the task characteristics.

[0009] Step 2: Select multiple candidate contact points among the nodes on the soft tissue surface. Based on the differentiable soft tissue deformation model, obtain the contact point combination with the largest availability value according to the change of feature points when all possible unit actions are applied to multiple contact points, and take the combined contact point as the optimal contact point.

[0010] Step 3: Based on the optimal contact point, use the MPC framework for closed-loop shape control to solve for the optimal motion sequence of the robot.

[0011] As a further preferred option, step one includes the following steps:

[0012] The soft tissue is discretized into a mesh consisting of n nodes. The shape of the soft tissue at time k is described by the position vector q(k) of all nodes. e feature points are selected on the surface of the soft tissue. According to the task requirements, a low-dimensional control variable κ(k) is extracted from the feature point position vector s(k).

[0013] As a further preferred option, step one also includes the following steps:

[0014] A soft tissue dynamics model is constructed based on the aforementioned grid nodes, feature points, and control variables:

[0015]

[0016] κ(k)=Φ(q(k)),

[0017] In the formula, q(k+1) represents the node position in the next time step. This represents the soft tissue deformation dynamics model, where a(k) is the action applied by the robot, q(k) is the node position in this time step, and q(0) is the node position in time step 0. init Let κ(k) be the initial node position of the soft tissue, Φ(q(k)) be the control variable, and Φ(q(k)) be the function that maps the node position to the control variable.

[0018] q(k+1) is the node position in the next time step. This represents a soft tissue deformation dynamics model.

[0019] As a further preferred option, in step two, a method based on projection dynamics is used to simulate the deformation of soft tissue.

[0020] As a further preferred embodiment, the method of simulating soft tissue deformation using a projection dynamics model includes:

[0021] The global nonlinear problem of soft tissue deformation dynamics is decomposed into parallel local projections and a single global solution problem using a projection dynamics model:

[0022]

[0023] Then, the deformation Jacobian matrix of the soft tissue is solved by implicit differentiation using the optimal conditions of the shadow dynamics model:

[0024]

[0025] In the formula, q(k+1) is the node position in the next time step, y(k) is the predicted unconstrained node position, M is the quality matrix, and E is the mass matrix. i (q) is the local deformation energy term, Δt is the discrete time step, and M c It is the nodal mass matrix associated with the contact point, y c It is the contact point position after the robot performs an action, where 'a' is the action performed by the robot. It is the globally deformable Jacobian matrix.

[0026] As a further preferred embodiment, the step of obtaining the contact point combination with the maximum availability value based on the change in feature points when all possible unit actions are applied to the multiple contact points includes:

[0027] (211) At a candidate contact point Apply all possible unit actions to obtain the feature point s.i The set of displacements, this set is called the operability set.

[0028] (212) When multiple robot tools operate multiple contact points, a single feature point s i Aggregate the comprehensive set of operable features;

[0029] (213) Calculate the set of all possible changes in control variables caused by the multi-point contact combination ψ. Then calculate the set in vector (κ) * Maximum projection length on -κ)

[0030] (214) Traverse all possible combinations of contact points and calculate the maximum projected length for each combination. The combination that maximizes this value is selected as the optimal contact point combination.

[0031] As a further preferred option, step (212) includes:

[0032] When multiple robotic tools are used to manipulate multiple contact points, the Minkowski sum operation is employed to obtain the combination of contact points for a single feature point s. i The overall operability, namely:

[0033] First, each operability set Use a convex polygon Approximation is performed, and then a rapid merging is achieved using the Minkowski sum theorem for convex polygons:

[0034]

[0035] In the formula, It is the Minkowski summation symbol.

[0036] As a further preferred embodiment, in step (213), the maximum projection length The computational models include:

[0037]

[0038] In the formula, κ * Let κ be the target value of the control variable, ψ be the value of the control variable at this time step, and Δκ(u,ψ) be the change of κ under the contact point combination ψ. j Let be the possible actions of the j-th robot.

[0039] As a further preferred embodiment, in step three, the MPC framework includes:

[0040]

[0041] st q(0)=q init

[0042]

[0043] ε min ≤ε i (k)≤ε max ,

[0044] In the formula, For the optimal action sequence, The loss function under control variables, Represents a soft tissue deformation dynamics model, a k:k+h For the action sequence in the time step interval [k:k+h], To represent the node position in time step k, To indicate time steps The amount of motion in κ * The target value of the control variable is q(0), where q(0) is the node position in time step 0. init a represents the initial node position of the soft tissue. max To ensure safety, the maximum motion value of the robot is set, ε min ε max ε represents the minimum and maximum strains, respectively, to ensure no damage to soft tissue. i (k) represents the soft tissue strain at time step k.

[0045] According to another aspect of the present invention, a vision-based robotic soft tissue multi-point contact shape control system is also provided, comprising:

[0046] The first main control module is used to discretize the soft tissue into a grid composed of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vector according to the task characteristics.

[0047] Step 2: Select multiple candidate contact points among the nodes on the soft tissue surface. Based on the differentiable soft tissue deformation model, obtain the contact point combination with the largest availability value according to the change of feature points when all possible unit actions are applied to multiple contact points, and take the combined contact point as the optimal contact point.

[0048] Step 3: Based on the optimal contact point, use the MPC framework for closed-loop shape control to solve for the optimal motion sequence of the robot.

[0049] Furthermore, this system is used to execute a vision-based robotic soft tissue multi-point contact shape control method based on any of the above embodiments or combinations of embodiments.

[0050] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0051] 1. This invention proposes an analytical, operability-based availability estimation algorithm that can automatically and quickly evaluate and select the optimal multi-point contact scheme according to specific task requirements. This avoids dependence on predetermined contact points and the blindness of manual selection, significantly improving operational efficiency and task success rate.

[0052] 2. This invention, by integrating a differentiable projective dynamics model, can predict the deformation of soft tissue under robot movements in real time and accurately, and calculate the precise deformation Jacobian matrix. Combined with a model predictive control framework, the controller can predict and compensate for deformation, thereby achieving faster convergence speed and lower tracking error than traditional adaptive control methods.

[0053] 3. This invention, based on a physical model and analytical derivation, requires no offline learning or training, overcoming the bottlenecks of existing data-driven methods in the medical field, such as difficulty in data acquisition and weak generalization ability. This method possesses strong interpretability and robustness, making it more suitable for surgical scenarios with extremely high requirements for safety and reliability.

[0054] 4. The control target of this invention can be flexibly constructed by defining different geometric features (such as points, lines, angles, distances, etc.), so that the framework can adapt to a variety of surgical manipulation tasks, such as organ traction, tissue flattening, pre-tensioning for cutting, etc., and has strong versatility.

[0055] 5. This invention explicitly incorporates the physiological strain limit of soft tissue as a constraint condition into the MPC framework, ensuring that the tissue stress is within a safe range throughout the entire manipulation process, effectively avoiding secondary damage caused by excessive traction or compression. Attached Figure Description

[0056] Figure 1 This is a control framework diagram of a vision-based robot soft tissue multi-point contact shape control method according to an embodiment of the present invention;

[0057] Figure 2 This is a flowchart illustrating the contact point selection optimization process according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0059] like Figure 1 As shown in the figure, this invention provides a vision-based method for multi-point contact shape control of robot soft tissue. First, based on a differentiable soft tissue deformation model, an analytical availability estimation algorithm is proposed to select the optimal combination of contact points from a series of candidate contact points. Then, using the selected contact points as input, within a model predictive control (MPC) framework, the robot's motion sequence is optimized to drive the soft tissue shape to the target state. This method requires no offline data training and achieves real-time, efficient, and stable control. The specific algorithm steps are as follows:

[0060] (1) System state definition and objective description:

[0061] The soft tissue is discretized into a mesh consisting of n nodes, whose shape at time k is determined by the position vectors q(k)∈R of all nodes. 3n Description. Select e feature points on the soft tissue surface, with spatial coordinates s(k)∈R. 3e And tracked in real time through a vision system. Based on specific task requirements (such as maintaining distance, adjusting angle, etc.), a low-dimensional control variable κ(k)∈R is extracted from the feature point position vector s(k). o For example, the control variable could be the distance between two feature points, or the angle formed by three feature points.

[0062] The grid nodes, feature points, and control variables are linked by the following dynamic equation.

[0063]

[0064] κ(k)=Φ(q(k)),

[0065] in, Let Φ(·) represent the soft tissue deformation dynamics model, where Φ(·) is a function that maps the positions of feature points to control variables.

[0066] The control objective in this invention is to cause the control variable κ(k) to eventually converge to a preset target value κ by having the robot apply an action a(k). * .

[0067] (2) Differentiable soft tissue deformation model:

[0068] This invention employs a method based on Projective Dynamics (PD) to efficiently simulate the deformation of soft tissues.

[0069] Forward simulation: The system's dynamic updates follow Newtonian mechanics and are integrated using the implicit Euler method. Its core is solving an energy minimization problem, as shown in the following equation. The PD method achieves high computational efficiency by decomposing the complex global nonlinear problem into two parallel steps: "local projection" and a single "global solution."

[0070]

[0071] Where y(k) is the predicted unconstrained node position, M is the quality matrix, and E i (q) is the local deformation energy term, and Δt is the discrete time step.

[0072] Backward differentiation: To perform gradient-based optimization control, it is necessary to calculate the derivative of the node position q with respect to the robot action a, i.e., the deformed Jacobian matrix. This invention analytically derives the Jacobian matrix by implicitly differentiating the optimal conditions of the PD model, as shown below.

[0073]

[0074] Where y c It is the displacement of the contact point caused by the robot's movements. It is the Hessian matrix of the system energy, M c It is a submatrix of the quality matrix related to the contact points. By employing an iterative update method, the columns of the Jacobian matrix can be calculated efficiently without directly inverting a large matrix, thus ensuring real-time performance.

[0075] (3) Operability-based availability estimation algorithm:

[0076] This is one of the core innovations of this invention, used to select the optimal set of contact points from multiple candidate contact points for operation before the task begins, such as... Figure 2 As shown.

[0077] Single-point operability set definition: for a candidate contact point and a feature point s i When When all possible unit actions are applied to the feature point s, i The set that causes a displacement. This set is called the operability set. Its shape and size reflect the operation point To manipulate feature points s i The ability. This set is composed of the local Jacobian matrix J. i,j A decision is expressed as:

[0078]

[0079] in It is the set of actions that are allowed to be performed at the contact point.

[0080] Multi-point operability aggregation: When multiple robotic tools are used to operate multiple contact points (the combination of contact points is ψ), its operability for a single feature point s i The overall operability is obtained by performing Minkowski sum operations on the operability sets generated at each contact point. To improve computational efficiency, each operability set is first... Use a convex polygon We approximate the sum, and then use the Minkowski sum theorem for convex polygons for rapid merging, as follows:

[0081]

[0082] in It is the Minkowski summation symbol.

[0083] Task-adaptive availability assessment: The quality of availability is related to the specific control task. This invention defines "optimal" availability as: the fastest descent direction from the current state to the target state (i.e., the task loss function). negative gradient direction (κ) * -κ) The ability to generate maximum displacement. Specifically, firstly, the set of all possible changes in control variables caused by the multi-point contact combination ψ is calculated. Then calculate the set in vector (κ) * Maximum projection length on -κ)

[0084]

[0085] The larger the value of ψ, the more "effective" the contact point combination ψ is in completing the current task, that is, the better the availability.

[0086] Optimal contact point selection: Iterate through all possible combinations of contact points and calculate the optimal contact point selection for each combination. The combination that maximizes this value is selected as the final robot contact point. The formula is as follows:

[0087]

[0088] (4) Model Predictive Controller (MPC) Design:

[0089] After selecting the optimal contact point ψ * A closed-loop shape control is implemented using an MPC framework. In each control cycle, the controller solves a finite-time optimization problem, as shown below, to find an optimal sequence of actions.

[0090]

[0091] st q(0)=q init

[0092]

[0093] ε min ≤ε i (k)≤ε max ,

[0094] The goal of this optimization problem is to minimize the error between the predicted control variables and the target values ​​over a future period. The optimization process involves two types of constraints: one is the limitation on the range of robot movements. Secondly, to ensure organizational safety, the deformation strain of each discrete unit ∈ i (k) must be maintained at the physiological safety threshold of soft tissue [∈ min ,∈ max ]Inside.

[0095] After solving, only the first action 'a' of the optimal action sequence is selected. * (k) is sent to the robot for execution, and then the process is repeated in the next control cycle.

[0096] In another embodiment of the present invention, a vision-based method for controlling the shape of multi-point contact in robotic soft tissue includes the following steps:

[0097] Step 1: Discretize the soft tissue into a mesh composed of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vectors according to the task characteristics. Specifically, the soft tissue is discretized into a mesh composed of n nodes, and the shape of the soft tissue at time k is described by the position vector q(k) of all nodes. e feature points are selected on the surface of the soft tissue, and a low-dimensional control variable κ(k) is extracted from the feature point position vector s(k) according to the task requirements.

[0098] In one embodiment, the control variable can be the distance between two feature points, or the angle formed by three feature points. The mesh nodes, feature points, and control variable are linked by the following dynamic equation.

[0099]

[0100] κ(k)=Φ(q(k)),

[0101] in, Let Φ(·) represent the soft tissue deformation dynamics model, where Φ(·) is a function that maps the positions of feature points to control variables.

[0102] In this embodiment, the control objective is to cause the control variable κ(k) to eventually converge to the preset target value κ by having the robot perform an action a(k). * .

[0103] Step two involves selecting multiple candidate contact points among the nodes on the soft tissue surface. Based on a differentiable soft tissue deformation model, the combination of contact points with the highest availability value is obtained by considering the change in feature points when all possible unit actions are applied to the multiple contact points. This combination of contact points is then selected as the optimal contact point. Specifically, a method based on projection dynamics is used to simulate the deformation of the soft tissue.

[0104] More specifically, in this step, the global nonlinear problem of soft tissue deformation dynamics is first decomposed into parallel local projections and a single global solution problem using a projection dynamics model:

[0105]

[0106] Then, the deformation Jacobian matrix of the soft tissue is solved by implicit differentiation using the optimal conditions of the shadow dynamics model:

[0107]

[0108] In the formula, q(k+1) is the node position in the next time step, y(k) is the predicted unconstrained node position, M is the quality matrix, and E is the mass matrix. i (q) is the local deformation energy term, Δt is the discrete time step, and M c It is the nodal mass matrix associated with the contact point, y c It is the contact point position after the robot performs an action, where 'a' is the action performed by the robot. It is the globally deformable Jacobian matrix.

[0109] More specifically, the combination of contact points that maximizes availability, based on the change in feature points when all possible unit actions are applied to multiple contact points, includes:

[0110] (211) At a candidate contact point Apply all possible unit actions to obtain the feature point s. i The set of displacements, this set is called the operability set.

[0111] (212) When multiple robot tools operate multiple contact points, a single feature point s i Aggregate the comprehensive set of operable features. That is:

[0112] When multiple robotic tools are used to manipulate multiple contact points, the Minkowski sum operation is employed to obtain the combination of contact points for a single feature point s.i The overall operability, namely:

[0113] First, each operability set Use a convex polygon Approximation is performed, and then a rapid merging is achieved using the Minkowski sum theorem for convex polygons:

[0114]

[0115] In the formula, It is the Minkowski summation symbol.

[0116] (213) Calculate the set of all possible changes in control variables caused by the multi-point contact combination ψ. Then calculate the set in vector (κ) * Maximum projection length on -κ)

[0117]

[0118] (214) Traverse all possible combinations of contact points and calculate the maximum projected length for each combination. The combination that maximizes this value is selected as the optimal contact point combination.

[0119] Step 3: Based on the optimal contact point, use the MPC framework for closed-loop shape control to solve for the optimal motion sequence of the robot.

[0120] More specifically, a vision system is used to capture the position vector s(κ) of feature points on the soft tissue surface, the current control variable κ(k) is calculated, and then an optimization problem is solved to plan the robot's optimal action a. * (k). The objectives of solving the optimization problem include minimizing the negative gradient direction (κ). * -κ) error, constraints include robot motion limitations and soft tissue safety strain limitations. Based on the above, the MPC framework includes:

[0121]

[0122] st q(0)=q init

[0123]

[0124] ε min ≤ε i (k)≤ε max ,

[0125] In the formula, For the optimal action sequence, The loss function under control variables, Represents a soft tissue deformation dynamics model, a k:k+h For the action sequence in the time step interval [k:k+h], To indicate time steps The node position in To indicate time steps The amount of motion in κ * The target value of the control variable is q(0), where q(0) is the node position in time step 0. init a represents the initial node position of the soft tissue. max To ensure safety, the maximum motion value of the robot is set, ε min ε max ε represents the minimum and maximum strains, respectively, to ensure no damage to soft tissue. i (k) represents the soft tissue strain at time step k.

[0126] Obtain the optimal action instruction a * (k) after which the optimal action instruction a * (k) is sent to the robot controller, and the robot combines ψ at the optimal contact point. * The operation is performed on the soft tissue.

[0127] According to another aspect of the present invention, a vision-based robotic soft tissue multi-point contact shape control system is also provided, comprising:

[0128] The first main control module is used to discretize the soft tissue into a grid composed of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vector according to the task characteristics.

[0129] The second main control module is used to select multiple candidate contact points among the nodes on the soft tissue surface. Based on the differentiable soft tissue deformation model, it obtains the contact point combination with the largest availability value according to the change of feature points when all possible unit actions are applied to multiple contact points, and takes the combined contact point as the optimal contact point.

[0130] The third main control module is used to perform closed-loop shape control using the MPC framework based on the optimal contact point, in order to solve for the optimal action sequence of the robot.

[0131] Of course, the above is only the overall module function of the control system of the present invention. The system is also used to implement the methods of any of the above embodiments or combinations of multiple embodiments.

[0132] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vision-based method for multi-point contact shape control of robotic soft tissue, characterized in that, Includes the following steps: Step 1: Discretize the soft tissue into a grid consisting of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vectors according to the task characteristics. Step 2: Select multiple candidate contact points among the nodes on the soft tissue surface. Based on the differentiable soft tissue deformation model, obtain the contact point combination with the largest availability value according to the change of feature points when all possible unit actions are applied to multiple contact points, and take the combined contact point as the optimal contact point. Step 3: Based on the optimal contact point, use the MPC framework for closed-loop shape control to solve for the optimal motion sequence of the robot.

2. The vision-based robotic soft tissue multi-point contact shape control method according to claim 1, characterized in that, Step one includes the following steps: The soft tissue is discretized into a mesh consisting of n nodes. The shape of the soft tissue at time k is described by the position vector q(k) of all nodes. e feature points are selected on the surface of the soft tissue. According to the task requirements, a low-dimensional control variable κ(k) is extracted from the feature point position vector s(k).

3. The vision-based robotic soft tissue multi-point contact shape control method according to claim 1, characterized in that, Step one also includes the following steps: A soft tissue dynamics model is constructed based on the aforementioned grid nodes, feature points, and control variables: q(0)=q init κ(k)=Φ(q(k)), In the formula, q(k+1) is the node position in the next time step. This represents the soft tissue deformation dynamics model, where a(k) is the action applied by the robot, q(k) is the node position in this time step, and q(0) is the node position in time step 0. init Let κ(k) be the initial node position of the soft tissue, Φ(q(k)) be the control variable, and Φ(q(k)) be the function that encodes the node position into the control variable.

4. The vision-based robotic soft tissue multi-point contact shape control method according to claim 1, characterized in that, In step two, a method based on projection dynamics is used to simulate the deformation of soft tissue.

5. The vision-based robotic soft tissue multi-point contact shape control method according to claim 4, characterized in that, The method of simulating soft tissue deformation using a projection dynamics model includes: The global nonlinear problem of soft tissue deformation dynamics is decomposed into parallel local projections and a single global solution problem using a projection dynamics model: Then, the deformation Jacobian matrix of the soft tissue is solved by implicit differentiation using the optimal conditions of the projection dynamics model: In the formula, q(k+1) is the node position in the next time step, y(k) is the predicted unconstrained node position, M is the quality matrix, and E is the mass matrix. i (q) is the local deformation energy term, Δt is the discrete time step, and M c It is the nodal mass matrix associated with the contact point, y c It is the contact point position after the robot performs an action, where 'a' is the action performed by the robot. It is the globally deformable Jacobian matrix.

6. The vision-based robotic soft tissue multi-point contact shape control method according to claim 1, characterized in that, The method of obtaining the contact point combination with the maximum availability value based on the change in feature points when applying all possible unit actions to multiple contact points includes: (211) At a candidate contact point Apply all possible unit actions to obtain the feature point s. i The set of displacements, this set is called the operability set. (212) When multiple robot tools operate multiple contact points, a single feature point s i Aggregate the comprehensive set of operable features; (213) Calculate the set of all possible changes in control variables caused by the multi-point contact combination ψ. Then calculate the set in vector (κ) * Maximum projection length on -κ) (214) Traverse all possible combinations of contact points and calculate the maximum projected length for each combination. The combination that maximizes this value is selected as the optimal contact point combination.

7. The vision-based robotic soft tissue multi-point contact shape control method according to claim 6, characterized in that, Step (212) includes: When multiple robotic tools are used to manipulate multiple contact points, the Minkowski sum operation is employed to obtain the combination of contact points for a single feature point s. i The overall operability, namely: First, each operability set Use a convex polygon Approximation is performed, and then a rapid merging is achieved using the Minkowski sum theorem for convex polygons: In the formula, It is the Minkowski summation symbol.

8. The vision-based robotic soft tissue multi-point contact shape control method according to claim 6, characterized in that, In step (213), the maximum projection length The computational models include: In the formula, κ * Let κ be the target value of the control variable, ψ be the value of the control variable at this time step, and Δκ(u,ψ) be the change of κ under the contact point combination ψ. j Let be the possible actions of the j-th robot.

9. A vision-based robotic soft tissue multi-point contact shape control method according to any one of claims 1-8, characterized in that, In step four, the MPC framework includes: s.t q(0)=q init e min ≤e i (k)≤ε max , In the formula, For the optimal action sequence, The loss function under control variables, Represents a soft tissue deformation dynamics model, a k:k+h For the action sequence in the time step interval [k:k+h], To represent the node position in time step k, To indicate time steps The amount of motion in κ * The target value of the control variable is q(0), where q(0) is the node position in time step 0. init a represents the initial node position of the soft tissue. max To ensure safety, the maximum motion value of the robot is set, ε min ε max ε represents the minimum and maximum strains, respectively, to ensure no damage to soft tissue. i (k) represents the soft tissue strain at time step k.

10. A vision-based robotic soft tissue multi-point contact shape control system, characterized in that, include: The first main control module is used to discretize the soft tissue into a grid composed of n nodes, select e feature points on the surface of the soft tissue, and extract control variables from the feature point position vector according to the task characteristics. The second main control module is used to select multiple candidate contact points among the nodes on the soft tissue surface. Based on the differentiable soft tissue deformation model, it obtains the contact point combination with the largest availability value according to the change of feature points when all possible unit actions are applied to multiple contact points, and takes the combined contact point as the optimal contact point. The third main control module is used to perform closed-loop shape control using the MPC framework based on the optimal contact point, in order to solve for the optimal action sequence of the robot.

Images