A programmable RCM motion accuracy measurement method

Abstract

A programmable RCM motion precision measurement method, the programmable RCM motion implementation device includes a surgical mechanical arm and an end effector mounted on the surgical mechanical arm; the end effector is used for carrying a surgical tool and providing operating power; a measuring device includes a binocular camera and a visual auxiliary marker point, and is used for collecting a homogeneous transformation matrix between the end effector and the binocular camera. The programmable RCM motion precision measurement method of the present application obtains the precision of the RCM point and the RCM constrained motion through optimal calculation by collecting the homogeneous transformation matrix of the marker coordinate system fixedly connected with the end effector relative to the camera coordinate system. The method of the present application provides an important evaluation basis for evaluating the programmable RCM motion algorithm.

Description

Technical Field

[0001] This invention belongs to the field of control and relates to a method for measuring the accuracy of robot constrained motion, and more particularly to a method for measuring the motion accuracy of fixed points (RCM points) under programmable remote center of motion (“RCM”) constraints. Background Technology

[0002] In minimally invasive surgery, surgical tools (such as surgical instruments and endoscopes) are inserted into the body through a small incision in the patient's skin. Guided by an endoscopic camera, the surgical tools are operated from outside the body to complete the surgery. In robot-assisted minimally invasive surgery, after a robotic arm carrying surgical tools enters the body through a small incision in the patient's skin, the movement of the surgical tools must revolve around this incision opening. Translation outside the incision direction is prohibited to avoid injury to the patient. More specifically, the robotic link with the surgical tools can only translate along its axis and rotate using the incision opening as a fulcrum. The fulcrum on the surgical tool's axis that coincides with the incision is called the remote center of motion ("RCM"). Currently, there are two main ways to achieve RCM-constrained motion in this field: one is to ensure RCM constraint through mechanical design, called mechanical RCM constraint; the other is to achieve programmable RCM constraint by controlling the motion of a series of robotic arms. Programmable RCM constraint has been widely used in recent years due to its high flexibility and space utilization. Compared to mechanical RCM constraints, programmable RCM constraints rely on the precision of algorithm control to ensure the safety of surgery. The motion precision of RCM points (RCM Precision) is an important evaluation indicator for evaluating the quality of programmable RCM constraint algorithms.

[0003] Currently, most measurements of the motion accuracy of programmable RCMs are performed manually and offline. These methods involve inserting the surgical tool axis at the end of the robotic arm into a trocar in teaching mode, performing telecentric constraint motion around the trocar point, and reading multiple sets of data from the robotic arm's Cartesian coordinate system to estimate the position of the RCM point. However, due to the influence of sample data volume, the flexible offset of the trocar point, and algorithm calculation errors, the measurement error is relatively large and cannot truly reflect the control accuracy of the algorithm. Furthermore, this manual and offline method is inefficient, increasing preparation procedures and time. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for measuring the accuracy of programmable RCM motion. The programmable RCM motion implementation device includes a surgical robotic arm and an end effector mounted on the surgical robotic arm. The end effector is used to carry surgical tools and provide operational power. The measuring device includes a binocular camera and visual auxiliary markers for acquiring the homogeneous transformation matrix between the end effector and the binocular camera. The method for measuring the accuracy of programmable RCM motion includes the following steps:

[0005] Step 1: Fix the visual auxiliary marker (hereinafter referred to as marker) on the end effector, keep its relative position with the stereo camera unchanged, and ensure that there is no obstruction between the marker and the stereo camera to ensure the integrity of the acquisition process.

[0006] Step Two: The operator uses the master hand or master program to control the end effector of the surgical robot to move around a predetermined initial RCM point. The specific constraint motion control can employ any feasible control method (for example, refer to Chinese Patent Publication No. CN113180828A, "Constraint Motion Control Method for Surgical Robots Based on Spinor Theory"). The operator's translational and rotational movements are converted into the RCM motion of the surgical robot in real time. Simultaneously, the homogeneous transformation matrix of the marker relative to the binocular camera is acquired and recorded using a binocular camera.

[0007] Step 3: Assuming the rotational motion of the surgical instrument held by the end effector satisfies the RCM constraint, there must exist a fixed point RCM in the base coordinate system, a rotation axis in the marker coordinate system, and a point on the axis. Theoretically, there exists a point such that the distance to all axes is minimized. Based on this principle, by optimizing a series of homogeneous transformation matrices between the marker coordinate system and the camera coordinate system, the RCM point, rotation axis, and point on the axis in the camera coordinate system can be obtained. The calculation formula is as follows:

[0008]

[0009] in, C p RCM The RCM point in the camera coordinate system C p i These represent the unit vector of the rotation axis in the camera coordinate system and a point on the axis, respectively. <,> denote the dot product operation. The loss function F... loss Let be the root mean square distance from the RCM point to the rotation axis. To describe the quality of the programmable RCM-constrained motion control algorithm, the square root of the loss function residual is used to evaluate the accuracy of the RCM-constrained motion, which geometrically represents the root mean square distance from the RCM point to the rotation axis.

[0010] Based on the above calculation principle, the homogeneous transformation matrix obtained in step two is substituted into formula (1) for iterative optimization, which yields the RCM point in the camera coordinate system. At the same time, the loss function value of the RCM point is obtained, and the square root is taken to obtain the motion precision of the RCM point.

[0011] Preferably, in step two, when using a binocular camera to acquire maker coordinates, in order to improve measurement accuracy and avoid errors caused by jitter, the end effector should stop moving only after the binocular camera stops acquiring data.

[0012] Preferably, since the relative positions of the base coordinate system and the camera coordinate system remain unchanged, in order to facilitate measurement, the RCM points in the base coordinate system, the rotation axes in the marker coordinate system, and the points on the axes described in step three are all transformed into the camera coordinate system for description.

[0013] Compared with existing technologies, this invention provides a method for measuring the motion accuracy of programmable RCM (Restricted Motion Flow). By acquiring the homogeneous transformation matrix of the marker coordinate system fixed to the end effector relative to the camera coordinate system, the accuracy of the RCM points and RCM-constrained motion is obtained through optimization calculation. The measurement method provided by this invention offers an important evaluation criterion for programmable RCM motion algorithms; that is, the smaller the motion accuracy value of the RCM points obtained from the calculation results, the better the performance of the programmable RCM motion algorithm, and the smaller the deviation of the tool axis from the RCM points during the motion.

[0014] The beneficial technical effects of this invention include: replacing the traditional manual, offline method with an automatic, real-time programmable RCM motion accuracy measurement method, simplifying the measurement steps, increasing the amount of sample data, and thus improving measurement efficiency and accuracy, providing a basis for correctly evaluating programmable RCM motion algorithms. Simultaneously, the coordinates of the RCM points obtained by the calculation method of this invention can provide positive feedback to the control algorithm; that is, the initial RCM points set by the control algorithm may not be accurate and can be replaced with the RCM points calculated by this invention, further improving control accuracy. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a measuring device according to an embodiment of the present invention;

[0016] Figure 2 This is a flowchart of the measurement method of the present invention;

[0017] Figure 3 This is a schematic diagram illustrating the calculation method of RCM points and RCM motion accuracy in one embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. However, those skilled in the art will understand that this invention is not limited to the accompanying drawings and the following embodiments.

[0019] In the description of the invention, it should be noted that directional terms such as "length," "width," "upper," "lower," "far," and "near" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. They should not be construed as limiting the specific scope of protection of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only to distinguish technical features and do not have substantive meaning. They should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features.

[0020] like Figure 1 As shown, the apparatus used in the embodiments of the present invention includes: a surgical robotic arm 1 for realizing programmable RCM motion; an end effector 2 mounted on the surgical robotic arm 1 for carrying surgical tools and providing operational power; and a measuring device including a binocular camera 3 and visual auxiliary markers 4 for acquiring the homogeneous transformation matrix between the end effector 2 and the binocular camera 3. The coordinate system of the visual auxiliary markers is denoted as {M}, the base coordinate system of the surgical robotic arm 1 is denoted as {B}, and the coordinate system of the binocular camera 3 is denoted as {C}. (Refer to...) Figure 2 The measurement method of this invention includes the following steps:

[0021] Step S101: Fix the visual auxiliary marker (hereinafter referred to as marker) on the end effector 2, keep its relative position with the binocular camera 3 unchanged, and ensure that there is no obstruction between the marker and the binocular camera 3 to ensure the integrity of the acquisition process.

[0022] Step S102: The operator manually controls the end effector 2 of the surgical robotic arm 1 to perform constrained motion around a predetermined initial RCM point. The translational and rotational motions (d, α, β, γ) of the operator's hand are converted into the RCM motion of the surgical robotic arm in real time using a surgical robot constrained motion control method (the specific constrained motion control can adopt any feasible control method; for example, refer to the Chinese patent publication "Surgical Robot Constrained Motion Control Method Based on Spin Theory," application publication number: CN113180828A). Simultaneously, the homogeneous transformation matrix of the marker coordinate system {M} relative to the binocular camera coordinate system {C} is acquired and recorded by the binocular camera 3.

[0023] Furthermore, in step S102, when using a binocular camera to collect maker coordinates, in order to improve measurement accuracy and avoid errors caused by jitter, the end effector 2 needs to stop moving only after the binocular camera 3 stops collecting data.

[0024] Step S103: Calculate the RCM point coordinates and RCM motion accuracy. First, assuming that the rotational motion of the end effector holding the surgical tool satisfies the RCM constraint, then there must exist a fixed point in the base coordinate system {B}. B p RCM An axis of rotation in the {M} coordinate system M s and a point on the axis M p, represented as ( M p, M s). Since the relative positions of the base coordinate system {B} and the camera coordinate system {C} remain unchanged, and the description of the base coordinate system can be transformed to the camera coordinate system through hand-eye calibration, based on this principle, such as Figure 3 As shown, there must exist an RCM point that is equidistant from the N rotation axes s1, s2, ..., s obtained in step S102. n The distances d1, d2, ..., d n The sum of these distances is minimized. n , which can be determined by C p RCM With a point on the axis C Distance to p || C p RCM - C p i ||and|| C p RCM - C p i The projected distance on the rotation axis is obtained using the Pythagorean theorem. Therefore, the loss function F of the optimization algorithm is set. loss Defined as the mean square distance from the RCM point to the rotation axis. The accuracy of the RCM-constrained motion is evaluated using the square root of the loss function residuals. Geometrically, it represents the root mean square distance from the RCM point to the axis of rotation. The calculation formula is as follows:

[0025]

[0026] in, <,> represents the dot product operation. and These represent the rotation and translation transformations of the marker coordinate system {M} relative to the stereo camera coordinate system {C}. Since this optimization algorithm is a nonlinear least squares problem, the Levenberg-Marquardt trust region algorithm (LM algorithm) is used for iterative optimization. A quadratic model is employed to optimize the loss function F.loss A simulation is performed, and the loss function F is given by the first-order partial derivative. loss The gradient h is then used to derive the loss function F using the finite difference method. loss The Hessian matrix G. The basic iterative steps are as follows:

[0027] 1. Set the initial value x0, including the initial position of RCM. B p RCM0 Direction of the axis of rotation M s0 and a point on the axis M p0, initial trust region radius h0 = ||g0||2, iteration first-order optimality termination tolerance is 10- 6 , which represents the degree of proximity between the measurement point x and the optimal point.

[0028] 2. At step k, calculate the corresponding g. k and G k

[0029] 3. The displacement s of the LM algorithm iteration is obtained by solving the trust region quadratic model. k And based on the ratio r of the actual decrease to the predicted decrease in the k-th iteration. k Make a judgment;

[0030] 4. If r k <0.25 indicates that the iterative displacement s k The value is too large; the trust region radius should be reduced to make it smaller.

[0031] 5. If r k >0.75 and ||s k ||2=h k This indicates the iterative displacement s k Too small; the trust region radius should be increased, let h k+1 =2h k ;

[0032] 6. If 0.25 < r k <0.75 indicates that the iterative displacement s k Suitable, h k+1 =h k Return to step 2 and continue iterating until the set first-order optimality termination tolerance value is reached;

[0033] 7. If r k <0 indicates that the function value is increasing rather than decreasing; keep x constant. k+1 =x k And reduce the trust region radius; through the above steps, the coordinates of the RCM point that satisfies the optimal solution can be obtained, and the motion accuracy P corresponding to the coordinates can be obtained.

[0034] In practical operation, the homogeneous transformation matrix mentioned in the embodiments of this invention... The measurement process is not limited to the binocular camera 3 and visual auxiliary markers 4; it can be replaced by other measurement devices, such as optical tracking devices. The purpose is to obtain the homogeneous transformation matrix. The calculation methods for RCM point coordinates and RCM motion accuracy are still obtained from formula (1).

[0035] Those skilled in the art will understand that the structures and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments, and the descriptions, disclosures, and figures should be considered merely as exemplary embodiments. Therefore, it should be understood that this disclosure is not limited to the precise embodiments described, and those skilled in the art can make various other changes and modifications without departing from the scope or spirit of this disclosure. Furthermore, elements and features shown or described in certain embodiments can be combined with elements and features of other embodiments without departing from the scope of this disclosure, and such modifications and variations are also included within the scope of this disclosure. Therefore, the subject matter of this disclosure is not limited to the content specifically shown and described.

Claims

1. A method of measuring programmable RCM motion accuracy, characterized by, The method for measuring the motion accuracy of the programmable RCM includes: Step S101: Fix the visual auxiliary markers on the end effector (2), keep their relative position with the binocular camera (3) unchanged, and ensure that there is no obstruction between them and the binocular camera (3) to ensure the integrity of the acquisition process; Step S102: The operator uses their main hand to control the end effector (2) of the surgical robotic arm (1) to perform constrained motion around the set initial RCM point, thereby controlling the translation and rotation of the operator's main hand. The motion is converted into RCM motion of the surgical arm in real time using the surgical robot constrained motion control method, where d corresponds to translation and α, β and γ correspond to rotation; at the same time, the homogeneous transformation matrix of the visual auxiliary marker coordinate system {M} relative to the binocular camera coordinate system {C} is collected and recorded by the binocular camera (3). ; Step S103: Calculate the RCM point coordinates and RCM motion accuracy; First, assuming the rotational motion of the end effector holding the surgical tool satisfies the RCM constraint, then there must exist a base coordinate system { B Fixed point under} 、{ M An axis of rotation in the coordinate system and a point on the axis , represented as ( , ); due to the base coordinate system { B } and camera coordinate system { C The relative positions remain unchanged, and the description of the base coordinate system can be transformed to the camera coordinate system through hand-eye calibration; based on this principle, there must exist an RCM point, which is connected to the N rotation axes acquired in step S102. S 1 , S 2 ,...., S N distance d 1 , d 2 , …, d N The sum is minimized; the distance d 1 , d 2 , …, d N From fixed point With a point on the axis distance as well as The projected distance on the rotation axis is obtained using the Pythagorean theorem; the loss function of the optimization algorithm is then defined based on this. Defined as the mean square distance from the RCM point to the rotation axis; and the square root of the residual of the loss function is used to evaluate the accuracy of the RCM-constrained motion. Geometrically, it represents the root mean square distance from the RCM point to the axis of rotation.

2. The programmable RCM motion accuracy measurement method of claim 1, wherein, In step S102, when the binocular camera is used to collect the coordinates of the visual auxiliary marker points, the end effector (2) stops moving after the binocular camera (3) stops collecting data.

3. The programmable RCM motion accuracy measurement method of claim 2, wherein, F loss The calculation formula is as follows: Equation (1) in, , , , This represents the dot product operation. and These are the coordinate systems of the visual auxiliary marker points { M } relative to the stereo camera coordinate system { C Rotation and translation transformations of} 4. The programmable RCM motion accuracy measurement method of claim 3, wherein, Formula (1) is iteratively optimized using the Levenberg-Marquardt Trust Region Algorithm (LM algorithm).

5. The programmable RCM motion accuracy measurement method of claim 4, wherein, In iterative optimization, a quadratic model is used to optimize the loss function. Simulations are performed, and the loss function is given through first-order partial derivatives. gradient g Then, the loss function is given by the finite difference method. Hessian matrix G .

6. The programmable RCM motion accuracy measurement method of claim 5, wherein, The iterative steps in iterative optimization include: (1) Set initial values Including the initial position of RCM Direction of the axis of rotation and a point on the axis Initial trust region radius ,in g 0 The initial gradient is given, and the first-order optimality termination tolerance of the iteration is 10. -6 , representing the measurement point How close to the optimal point; (2) At the kth step, calculate the corresponding and ; wherein, g k is the gradient of the kth step, G k is the Hesse matrix of the kth step; (3) The displacement of the LM algorithm iteration is obtained by solving the trust region quadratic model. And based on the ratio of the actual decrease to the predicted decrease in the k-th iteration. Make a judgment; (4) If , it means that the iteration displacement is too large, and the radius of the trust region should be reduced to ; (5) If and , then the radius of the trust domain is enlarged, and ; (6) If , , return to step 2 to continue iteration until a set first-order optimality termination tolerance value is reached; (7) If , keep and reduce the radius of the confidence domain; through the above steps, the RCM point coordinates that meet the optimal solution are obtained, and the motion accuracy P corresponding to the coordinates is obtained.

Images