A method for learning robotic surgery skills under remote motion center constraints
Through task parameter technology, the remote motor center constraint space and surgical task constraint space are unified into metaspace, solving the problem of inconsistency in minimally invasive surgery, improving the accuracy and flexibility of robot surgical skills, and realizing the Autonomous execution.
Patent Information
- Application Number
- CN202211335145.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-10-28
AI Technical Summary
In minimally invasive surgery, the remote motor center constraint space is inconsistent with the surgical task constraint space, and the existing demonstration learning methods cannot learn these two constraints at the same time, resulting in insufficient generalization of skills.
Introduce task parameter technology to unify the operational task requirements of different constraint spaces into the same metaspace, extract implicit constraints in doctor operations through fusion of probability models, and generate a probability model covering all constraints of surgical skills for new surgical task scenarios. autonomous execution.
Improves the accuracy and robustness of skills, provides higher flexibility and adaptability, reduces complexity, improves skill generalization, and expands to other tasks in multi-constraint spaces to have good migration .
Smart Images

Figure CN115502982B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of intelligent robot control technology, specifically a robot surgical skill learning method under remote motion center constraints, and relates to a robot surgical skill learning method based on demonstration learning. Specifically, the robot can learn the surgical skills of human doctors and actively follow the remote motion center constraints in minimally invasive surgery. Background Art
[0002] In recent years, with the continuous development of robotics technology, robots assisting doctors in clinical surgery can reduce the physiological tremors of doctors and improve the accuracy and stability of surgery. Compared with traditional open surgery, the master-slave remote-operated surgical robot system can reduce damage to the patient's body and shorten the postoperative recovery time. Specifically, a minimally invasive surgical incision is opened on the patient's body surface, and the robot is controlled to make the surgical instruments installed at the end pass through the incision into the lesion in the surgical cavity. During the operation, the robot is controlled to move around the incision, and the instruments and the incision cavity wall are not squeezed or torn. This incision is called the remote motion center, and the corresponding motion constraints are called remote motion center constraints.
[0003] Improving the automation level of surgical robots can replace doctors in repetitive operations, reduce doctors' fatigue and mental stress, and better assist doctors in performing operations. However, there is currently no standardized definition of surgical operations, and they rely more on doctors' continuous training and accumulated clinical experience. It is difficult to directly program and control surgical tasks. Therefore, migrating doctors' operating experience to surgical robot systems can avoid programming surgical skills and provide more flexibility and adaptability. This type of robot learning of human skills is called demonstration learning method. However, in minimally invasive surgery, the remote motion center constraint space and the surgical task constraint space are not in the same space. Therefore, how to learn the skills demonstrated by doctors and meet the remote motion center constraints, while being able to autonomously perform the skills at new incision locations and lesion locations is a thorny problem that needs to be solved. Summary of the invention
[0004] In order to solve the problem that the remote motion center constraint space and the surgical task constraint space are inconsistent in minimally invasive surgery, and the existing demonstration learning method cannot learn these two constraints at the same time, and the generalization of the learned skills is insufficient, the present invention proposes a robotic surgical skill learning method under the remote motion center constraint. The task parameter technology is introduced in this learning method to solve the problem of inconsistency between the remote motion center constraint space and the surgical task constraint space. Compared with other similar studies that only focus on the operation tasks of the robot end, this method can unify the operation task requirements of different constraint spaces into the same metaspace, and all constraints of the learned skills are completely retained, which improves the accuracy and robustness of the skills.
[0005] The technical solution of the present invention to solve the technical problem is to design a robot surgery skill learning method under the constraint of remote motion center, characterized in that the method comprises the following steps:
[0006] Step 1: Obtain demonstration data of multiple groups of doctors performing a surgical task, where the surgical task is to control the surgical instrument to track a preset trajectory in a cavity using a robotic manipulator under the constraint of a remote motion center;
[0007] Step 2: Based on the demonstration data, determine the data in the same constraint space in different demonstrations, and build a probability model for the data in the same type of constraint space; select the metaspace and obtain the task parameters of each constraint space in different demonstrations relative to the metaspace;
[0008] Step 3: transform the probability models describing the corresponding constraints in the different types of constraint spaces obtained in step 2 into the meta-space based on the corresponding task parameters, and fuse the probability models representing the constraint characteristics of each constraint space in the meta-space to obtain a probability model covering all constraints of surgical skills;
[0009] Step 4: Solve the task parameters corresponding to each constraint space according to the new surgical task scenario data, and give the environmental input variables according to the task requirements, calculate the conditional distribution probability model of the probability model covering all constraints of surgical skills obtained in step 3 in the meta-space, and obtain the corresponding output variables in the meta-space; transform the output variables described in the meta-space to each constraint space through the above-mentioned task parameters, and obtain the corresponding output variables of the generalized learned skills under the new surgical task scenario, and finally the surgical robot system executes and completes the new surgical task.
[0010] Compared with the prior art, the beneficial effect of the present invention is that the learning method of the present invention introduces task parameters to solve the problem of inconsistency between the remote motion center constraint space and the surgical task constraint space, unifies the operation task requirements of different constraint spaces into the same metaspace, extracts the implicit constraints in the doctor's operation from the doctor's skill demonstration process, and fuses them to obtain a skill model containing all the constraint features of surgical skills. In the new task scenario, the corresponding task parameters can be directly input to autonomously calculate the corresponding skill action instructions, and then the surgical robot executes the task. Compared with other similar studies that only focus on the operation tasks at the end of the robot, this method can unify the operation task requirements of different constraint spaces into the same metaspace, and all constraints of the learned skills are completely retained, which significantly improves the accuracy and robustness of the skills. Compared with the traditional a priori constraint definition method, implicit constraints can provide a more accurate constraint form that fits the doctor's operation skills, has higher flexibility, and has better adaptability in different surgical task scenarios. Compared with traditional task-oriented programming, it reduces complexity, while still having good performance in non-demonstration scenarios, and improves skill generalization. The learning method proposed in the present invention can also be extended to other tasks in multiple constraint spaces and has good transferability. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 Schematic diagram of remote motion center constraint when remotely operating a surgical robot system for minimally invasive surgery (Figure: 1, serial robot arm in the surgical robot system; 2, surgical instrument; 3, remote motion center constraint; 4, patient surgical cavity; 5, remote motion center constraint error; 6, surgical task; the black part represents the state diagram of the slave end of the surgical robot system before movement; the gray part represents the state diagram of the slave end of the surgical robot system after movement; P flange and P′ flange Respectively represent the position of the flange at the end of the robot arm before and after the movement; P RCM Indicates the position of the incision, i.e. the remote motion center; y yyy and P′tip represent the position of the end of the surgical instrument before and after the movement, respectively).
[0012] Figure 2 It is a flowchart of an embodiment of a method for learning robot surgery skills under remote motion center constraints of the present invention. DETAILED DESCRIPTION
[0013] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0014] The present invention provides a method for learning robot surgery skills under remote motion center constraints, the method comprising the following steps:
[0015] Step 1: Obtain demonstration data of multiple groups of doctors performing a surgical task, which is to use a robotic arm to control surgical instruments to track a preset trajectory within a cavity under the constraints of a remote motion center.
[0016] In this embodiment, the skill task to be learned is to control the surgical instrument to track a sinusoidal trajectory in the cavity under the constraint of the remote motion center. First, the doctor controls the remotely operated surgical robot system to demonstrate how to complete the task and obtain the corresponding demonstration data. The specific steps include the following:
[0017] Step 1.1: A doctor with operation experience controls the remote surgical robot system to perform surgical tasks under the constraints of the remote motion center and record the position of the remote motion center, the position of the lesion area and the motion data.
[0018] In this specific embodiment, the remotely operated surgical robot system used is composed of a master end and a slave end. The master end is a force / tactile feedback device, which is directly operated by the doctor. Then the force / tactile feedback device maps the doctor's hand movements to the slave end to indirectly control the slave end to move; the slave end is a seven-degree-of-freedom serial robot arm, and an instrument with a length of 0.3 meters and a diameter of 0.005 meters is installed on the end flange. It enters the cavity through the incision (cannula point) on the cavity simulator. The end of the instrument performs corresponding movements according to the action instructions mapped by the master end and satisfies the remote motion center constraint. The information inside the cavity is fed back to the doctor through the endoscope vision so as to control the robot system in real time to track the preset sinusoidal trajectory. The trajectory is drawn on the surface of an artificial skin, and the corresponding curve equation is y-0.01sin(100x), x[0, 0.02][m].
[0019] In this embodiment, the position P of the remote motion center is recorded. RCM The position of the lesion area specifically includes the origin P of the task area coordinate system described in the robot base coordinate system task With the attitude rotation matrix The motion data includes the joint angle data of the robot arm, the remote motion center error, and the motion trajectory of the end of the instrument.
[0020] Step 1.2: Change the position of the remote motion center and the position of the lesion area, and continue to perform the same surgical task. In this specific embodiment, other cannula points are selected, the position of the artificial skin block is changed, and then the same tracking task is performed again.
[0021] Step 1.3: Repeat steps 1.1 to 1.2 to collect the doctor's operation demonstration data containing the internal consistency and external adaptability of the surgical skill constraint; when the number of demonstrations reaches the set number, stop the demonstration of the surgical task and store the data of each demonstration respectively. In this specific embodiment, a total of eight groups of demonstration data of the same tracking task in different scenarios are collected and the data are saved.
[0022] Step 2: Based on the demonstration data, determine the data in the same constraint space in different demonstrations, and construct a probability model for the data in the same type of constraint space, and express the implicit constraints in the doctor's demonstration operation probabilistically; select the metaspace and obtain the task parameters of each constraint space in different demonstrations relative to the metaspace. Specifically, it includes the following steps:
[0023] Step 2.1: Determine the data in the same constraint space in different demonstrations. Specifically, collect demonstration data in different constraint spaces. in, is the data point described in the constraint space {j}, and t is the time corresponding to the data point.
[0024] Step 2.2: Construct a probability model for the data in the same constraint space, that is, A probability model is constructed, which is a description of the corresponding constraint in the constraint space. In this specific embodiment, for data in the same constraint space, a Gaussian mixture model is used to express the motion constraint features in a probabilistic form. The expression of the Gaussian mixture model is:
[0025]
[0026] Among them, π i is the weight coefficient, D is the data dimension, is the multidimensional Gaussian distribution density, and the center of the i-th multidimensional Gaussian distribution density sub-model is μ i , the covariance matrix is Σ i , there are K sub-models constituting the Gaussian mixture model, and the number of sub-models can be selected by Akaike information criterion or Bayesian information criterion. The parameters of Gaussian mixture modeling can be estimated by the expectation maximization algorithm.
[0027] Step 2.3: Based on the demonstration data, a metaspace is selected and the task parameters corresponding to the constraint space in different demonstrations are determined. The task parameters represent the transformation parameters of the constraint space relative to the metaspace (the selected space or meta coordinate system).j , b j}, where A j is the posture transformation matrix from constraint space {j} to metaspace, b j is the position of the origin of the constraint space {j} in the metaspace.
[0028] In this embodiment, the constraint spaces where the data involved are located include three: the joint constraint space of the robot arm {1}, the remote motion center constraint space {2}, and the surgical task constraint space of the end motion of the instrument {3}. The joint constraint space is selected as the metaspace. According to the robot kinematics derivation, we can get:
[0029] Task parameters of joint constraint space {1} for:
[0030]
[0031]
[0032] In the above formula, N RCM,t-1 is the null space matrix corresponding to the remote motion constraint, I is the identity matrix, J RCM,t-1 is the Jacobian matrix corresponding to the remote motion center point on the robot arm at time t-1, is the corresponding MP pseudo-inverse matrix; is the joint angle data of the robot arm at time t-1.
[0033] Task parameters of the remote motion center constraint space {2} for:
[0034]
[0035]
[0036] In the above formula, q t-1 is the joint angle data of the robot arm at t-1, is the remote motion center error of the robot at time t-1.
[0037] Task parameters of surgical task constraint space {3} for:
[0038]
[0039]
[0040] In the above formula, is the MP pseudo-inverse matrix of the Jacobian matrix corresponding to the end of the instrument on the robotic arm at time t-1, is the motion trajectory of the instrument end described by the robotic arm in the surgical task space at time t-1.
[0041] Step 3: Transform the probability models describing the corresponding constraints in the different types of constraint spaces obtained in step 2 into the meta-space based on the corresponding task parameters, and fuse the probability models representing the constraint characteristics of each constraint space in the meta-space to obtain a probability model covering all constraints of surgical skills. This embodiment is to unify and coordinate the remote motion center constraint space and the surgical task constraint space, and specifically includes the following contents:
[0042] Step 3.1: Transform the probability model describing the corresponding constraints in the constraint space obtained in step 2 into the meta-space based on the corresponding task parameters. In this embodiment, the joint constraint space is selected as the meta-space, and the linear transformation property of the Gaussian distribution is used to obtain:
[0043]
[0044] The Gaussian mixture model in each constraint space is transformed into the meta-space, thereby unifying different constraints into the same space to solve the problem of inconsistent constraint spaces.
[0045] Step 3.2: In the metaspace, the probability models representing the constraint features of each constraint space are fused to obtain a probability model covering all constraints of surgical skills. In this embodiment, the metaspace is the joint constraint space, and the Gaussian mixture models obtained in different constraint spaces can be fused into one Gaussian mixture model by using the Gaussian distribution product property:
[0046]
[0047] In the above formula, the mean center corresponding to the fused Gaussian mixture model is With the covariance matrix for:
[0048]
[0049] Step 4: Solve the task parameters corresponding to each constraint space according to the new surgical task scenario data, and give the environmental input variables according to the task requirements, calculate the conditional distribution probability model of the probability model covering all surgical skill constraints obtained in step 3 in the meta-space, and obtain the corresponding output variables in the meta-space; transform the output variables described in the meta-space to each constraint space through the above task parameters, and obtain the corresponding output variables of the generalized learned skills in the new surgical task scenario, and finally the surgical robot system executes and completes the new surgical task. Specifically including the following contents:
[0050] Step 4.1: Determine the task parameters corresponding to each constraint space in the new surgical task scenario. In this embodiment, according to the position P of the remote motion center corresponding to the new surgical task scenario, RCMThe position of the lesion area specifically includes the origin P of the task area coordinate system described in the robot base coordinate system task With the attitude rotation matrix Determine the corresponding task parameters.
[0051] Step 4.2: Given environment input variables according to task requirements Calculate the conditional distribution probability model of the probability model covering all constraints of surgical skills obtained in step 3.2 in the metaspace Get the corresponding output variable In this embodiment, firstly according to the input variable With output variables The probability model parameters are decomposed into blocks, as follows:
[0052]
[0053] Calculate the conditional Gaussian distribution probability based on block decomposition,
[0054]
[0055] in,
[0056]
[0057]
[0058] In this embodiment, time is selected as the input variable Joint angles as output variables in metaspace
[0059] Step 4.3: The output variables described in the metaspace calculated in step 4.2 are transformed into each constraint space respectively through the task parameters obtained in step 4.1, and the corresponding output variables of the generalized learned skills in the new task scenario are obtained, and finally the new surgical task is performed by the surgical robot system. In this embodiment, since the joint constraint space of the robotic arm is selected as the metaspace, the output variable calculated in step 4.2 is the joint angle, which can be directly executed by the robot system as an action instruction and obtain the corresponding execution result. In the case where the metaspace and the motion space (in this specific embodiment, the metaspace and the motion space are both joint constraint spaces) are inconsistent, it is necessary to transform the data of the output variable into the motion space, and then perform the corresponding action.
[0060] The experimental results of this embodiment are as follows:
[0061] Table 1 Comparison of remote motion center constraint errors
[0062]
[0063]
[0064] Table 2 Tracking error comparison
[0065]
[0066] The experimental results are shown in Tables 1 and 2, which indicate that the robotic surgical skill learning method under remote motion center constraints based on demonstration learning proposed in the present invention can effectively learn the skills demonstrated by the operator under the remote motion center constraints, and within the allowable remote motion center constraint error range, the accuracy of the skills performed by the surgical robot system exceeds the average level of human operators.
[0067] The present invention introduces task parameters in the robot surgical skill learning method under the constraint of remote motion center based on demonstration learning. First, according to the overall process of executing the task, the constraint characteristics in different constraint spaces are represented by probability models respectively, and then the constraints learned in each constraint space are transformed into the same space-the metaspace, and fused to obtain a skill model containing all the constraint characteristics of the task skills. Finally, in the new surgical scenario, the corresponding output variables are calculated according to the input variables of the environment through the conditional distribution model, and the skill constraints are still maintained. The corresponding task parameters transform the output variables into the new corresponding surgical constraint space, and finally the surgical robot executes the skill to complete the task.
[0068] The learning method of the present invention uses the covariance matrix and mean center vector of the probability model to represent the implicit constraints in the doctor's demonstration skill operation, and ensures the flexibility and smoothness of the skill action. At the same time, different constraint spaces and different task scenarios are summarized and represented in the form of task parameters, which improves the internal consistency and external adaptability of the learned skills, and can be expanded to the robot learning application of related multi-constraint skills, including other similar zero space control problems, whole body control problems, etc., and is not limited to the examples and constraints mentioned in this description. This learning method provides a feasible strategy for learning skills under the constraints of the remote motion center in a minimally invasive surgical environment.
[0069] The present invention may also have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.
[0070] Any matters not described in the present invention are applicable to the prior art.
Claims
1. A method for learning robotic surgery skills under remote motion center constraints, characterized in that: The method comprises the following steps: Step 1: Obtain demonstration data of multiple groups of doctors performing a surgical task, where the surgical task is to control the surgical instrument to track a preset trajectory in a cavity using a robotic manipulator under the constraint of a remote motion center; Step 2: Based on the demonstration data, determine the data in the same constraint space in different demonstrations, and build a probability model for the data in the same type of constraint space; select the metaspace and obtain the task parameters of each constraint space in different demonstrations relative to the metaspace; Step 3: transform the probability models describing the corresponding constraints in the different types of constraint spaces obtained in step 2 into the meta-space based on the corresponding task parameters, and fuse the probability models representing the constraint characteristics of each constraint space in the meta-space to obtain a probability model covering all constraints of surgical skills; Step 4: Solve the task parameters corresponding to each constraint space according to the new surgical task scenario data, and give the environmental input variables according to the task requirements, calculate the conditional distribution probability model of the probability model covering all surgical skill constraints obtained in step 3 in the meta-space, and obtain the corresponding output variables in the meta-space; transform the output variables described in the meta-space to each constraint space through the above-mentioned task parameters, and obtain the corresponding output variables of the generalized learned skills in the new surgical task scenario, and finally the surgical robot system executes and completes the new surgical task; The step 1 specifically includes the following steps: Step 1.1: A doctor with operation experience controls the remote surgical robot system to perform the surgical task under the constraints of the remote motion center and record the position of the remote motion center, the position of the lesion area and the motion data; Step 1.2: Change the position of the remote motion center and the position of the lesion area, and continue to perform the same surgical task; Step 1.3: Repeat steps 1.1 to 1.
2. When the demonstration times reach the set times, stop the demonstration of the surgical task and store the data of each demonstration respectively. The step 2 specifically includes the following steps: Step 2.1: Determine the data in the same constraint space in different demonstrations. Specifically, collect demonstration data in different constraint spaces. in, is the data point described in the constraint space {j}, is the time corresponding to the data point; Step 2.2: Construct a probability model for the data in the same constraint space, that is, Construct a probability model, which is a description of the corresponding constraints in the constraint space; Step 2.3: Based on the demonstration data, select the metaspace and determine the task parameters corresponding to the constraint space in different demonstrations. The task parameters represent the transformation parameters of the constraint space relative to the metaspace. j , b j }, where A j is the posture transformation matrix from constraint space {j} to metaspace, b j is the position of the origin of the constraint space {j} in the metaspace; The step three specifically includes the following steps: Step 3.1: Use linear transformation to transform the probability model describing the corresponding constraints in each constraint space obtained in step 2 into the meta-space based on the corresponding task parameters; Step 3.2: Using the linear transformation property of Gaussian distribution, the probability models representing the constraint features of each constraint space are multiplied and fused in the meta-space to obtain a probability model covering all constraints of surgical skills; The step 4 specifically includes the following steps: Step 4.1: Determine the task parameters corresponding to each constraint space in the new surgical task scenario, and determine the corresponding task parameters according to the position of the remote motion center and the position of the lesion area in the new surgical task scenario; Step 4.2: Given environment input variables according to task requirements Calculate the conditional distribution probability model of the probability model covering all constraints of surgical skills obtained in step 3.2 in the metaspace Get the corresponding output variable Step 4.3: The output variables described in the metaspace calculated in step 4.2 are transformed into each constraint space through the task parameters obtained in step 4.1, and the corresponding output variables of the generalized learned skills in the new task scenario are obtained. Finally, the surgical robot system executes and completes the new surgical task.
2. The method for learning robotic surgery skills under remote motion center constraints according to claim 1, characterized in that: The preset trajectory in step 1 is a sine curve.
3. The method for learning robotic surgery skills under remote motion center constraints according to claim 1, characterized in that: The probability model in step 2 is a Gaussian mixture model.
