Surgical assisting robot control method and surgical assisting robot

Abstract

The application belongs to the technical field of manipulators or robots specially used for surgery, and particularly relates to a surgical auxiliary robot control method and a surgical auxiliary robot, the method comprising: constructing a robot inverse kinematics solving formula; constructing a robot forward kinematics solving formula based on the inverse kinematics solving formula; constructing a hand-eye calibration model; and performing point cloud registration based on the hand-eye calibration model, and determining an optimal registration solution through a root mean square error; the surgical auxiliary robot control method provided by the application can accurately position surgical instruments such as a puncture needle to a target position through forward and inverse kinematics solving, combined with hand-eye calibration technology and point cloud registration technology, so as to reduce surgical errors and improve the success rate and safety of surgery; through cooperation of a 3D camera and the first and second calibration plates, real-time visual feedback is provided, high-precision positioning of a robot end effector is realized, and the doctor can be better helped to perform surgical planning and operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of manipulators or robotics specifically adapted for surgery, and in particular to a surgical auxiliary robot control method and surgical auxiliary robot. BACKGROUND

[0002] In recent years, neurosurgical robot has begun to develop greatly, which can assist doctors to carry out aspiration drainage, intracranial biopsy and other neurosurgical operations. Most of the surgical robots on the market are mainly mechanical arms, which have automatic positioning capability. Doctors control the mechanical arm through the robot software, and complete the actual operation by the end operation platform. The mechanical arm can perform fine and complex surgical operations in long-term complex operations, eliminating the errors caused by hand natural tremor and doctor fatigue, and providing a stable and reliable platform for intraoperative operation. At the same time, the surgical robot seamlessly integrates the surgical operation system and the navigation system, dynamically adjusts the position and path of the surgical instrument by using real-time navigation information, ensures the accurate operation, and reduces the operation burden of the surgeon.

[0003] However, the current surgical robot has some problems. First, the human-computer interaction complexity is high, and the operation interface is not suitable for clinical practice. Second, the cost-benefit ratio is unbalanced, and the equipment purchase and maintenance cost is far more than the conventional medical budget, and the reliability doubt caused by the lack of safety redundancy mechanism in the actual medical scene leads to a very low activation rate of the automation function.

[0004] Therefore, the present application provides a surgical auxiliary robot control method and surgical auxiliary robot to solve the problem of high human-computer interaction complexity and high cost of the surgical robot in the prior art. SUMMARY

[0005] The present application provides a surgical auxiliary robot control method and surgical auxiliary robot, which aims to solve the above-mentioned problems existing in the prior art.

[0006] The technical scheme of the present application is as follows:

[0007] A surgical auxiliary robot control method, comprising:

[0008] S1. Constructing a robot inverse kinematics solving formula;

[0009] S2. Constructing a robot forward kinematics solving formula based on the inverse kinematics solving formula;

[0010] S3. Constructing a hand-eye calibration model;

[0011] S4. Based on the hand-eye calibration model, performing point cloud registration, and determining the optimal registration solution by root mean square error;

[0012] S5. Based on the optimal registration solution and the robot's forward and inverse kinematics calculation formulas, the puncture and guidance points are determined, and the corresponding motion paths are calculated and adjusted in real time through visual feedback.

[0013] Further, S1 includes the following steps:

[0014] S1.1 Set the coordinate system and initial parameters, and construct the reference spherical coordinate system;

[0015] S1.2. Determine the relevant vectors and planes after the robot's motion;

[0016] S1.3. Solve for the relevant vectors;

[0017] S1.4. Construct the inverse kinematics solution formula through geometric and vector operations.

[0018] Further, S2 includes the following steps:

[0019] S2.1. Determine the relationship between the known joint motor angles and the fixed position of the end effector;

[0020] S2.2. Perform vector decomposition;

[0021] S2.3. Combine the inverse kinematics solution formula to complete the forward kinematics solution formula.

[0022] Further, S3 includes the following steps:

[0023] S3.1. Based on the correspondence between feature points of real objects in space under different coordinate systems, construct a mathematical mapping model;

[0024] S3.2. Perform data acquisition and preprocessing;

[0025] S3.3. Perform feature point extraction and coordinate determination;

[0026] S3.4. Construct a robot coordinate system and obtain the robot's coordinates in the coordinate system;

[0027] S3.5. Solve for the rigid body transformation parameters.

[0028] Further, S4 includes the following steps:

[0029] S4.1. Collect data and generate point clouds;

[0030] S4.2. Perform preliminary rotation and alignment of the point cloud;

[0031] S4.3. Perform point cloud segmentation;

[0032] S4.4. Perform point cloud registration to determine the optimal registration solution.

[0033] A surgical assistive robot, using the aforementioned surgical assistive robot control method, includes a robot body and a processor and a 3D camera electrically connected to the robot body. The robot body includes a mounting base, an arc motion frame, a 3D camera, a neck brace, a first motor, and an end effector. The arc motion frame is rotatably connected to the mounting base, the neck brace is connected to the mounting base, the first motor is located on the side wall of the mounting base, and the end effector is slidably connected to the arc motion frame.

[0034] Furthermore, the end effector includes a positioning seat, a second motor, and a spherical rotary joint. The positioning seat is slidably connected to the arc motion frame. The second motor is located on the top of the positioning seat and is electrically connected to the processor. The spherical rotary joint is located on the side wall of the positioning seat.

[0035] Furthermore, the end effector also includes a locking knob located at the bottom of the positioning base.

[0036] Furthermore, the end effector is also connected to a first calibration plate, which is located on top of the positioning base.

[0037] Furthermore, the mounting base is also connected to a second calibration plate, which is located on the side wall of the mounting base.

[0038] The beneficial effects of this invention are as follows:

[0039] This invention provides a surgical assistive robot control method that, through inverse kinematics and forward kinematics calculations, combined with hand-eye calibration technology and point cloud registration technology, can accurately position surgical instruments such as puncture needles to target positions, reducing surgical errors and improving the success rate and safety of surgery; by using a 3D camera in conjunction with the first and second calibration plates, real-time visual feedback is provided, achieving high-precision positioning of the robot's end effector, which can help doctors better plan and operate the surgery;

[0040] The surgical robot provided by this invention has a simple design, is easy to manufacture and maintain, and reduces production costs. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the method provided by the present invention;

[0043] Figure 2 This is a schematic diagram of the robot's operation process provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the spherical coordinate system constructed by the method provided in this invention;

[0045] Figure 4 This is a three-dimensional structural diagram of the robot provided by the present invention;

[0046] Figure 5 This is a schematic diagram of the three-dimensional structure of the robot after the first calibration plate is removed, provided by the present invention.

[0047] Figure 6 This is a front view of the robot provided by the present invention;

[0048] Figure 7 This is a three-dimensional structural diagram of the end effector provided by the present invention.

[0049] Legend:

[0050] 1-Mounting base; 2-Neck brace; 3-Circular arc motion frame; 4-First calibration plate; 5-End effector; 51-Positioning seat; 52-Spherical rotary pair; 53-Second motor; 54-Locking knob; 6-Second calibration plate; 7-First motor. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0052] In the description of this invention, it should be understood that the terms indicating orientation or positional relationship are based on the orientation or positional relationship shown in the drawings and are only for the convenience of describing the invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0053] Example

[0054] This embodiment provides a surgical assistive robot, including a robot body and a processor and a 3D camera electrically connected to the robot body. The robot body includes a mounting base 1, an arc motion frame 3, a 3D camera, a neck brace 2, a first motor 7, and an end effector 5. The arc motion frame 3 is rotatably connected to the mounting base 1, the neck brace 2 is connected to the mounting base 1, the first motor 7 is located on the side wall of the mounting base 1, and the end effector 5 is slidably connected to the arc motion frame 3.

[0055] Furthermore, the end effector 5 includes a positioning base 51, a second motor 53, and a spherical rotary joint 52. The positioning base 51 is slidably connected to the arc motion frame 3. The second motor 53 is located on the top of the positioning base 51 and is electrically connected to the processor. The spherical rotary joint 52 is located on the side wall of the positioning base 51.

[0056] Furthermore, the end effector 5 also includes a locking knob 54, which is located at the bottom of the positioning base 51.

[0057] Furthermore, the end effector is also connected to a first calibration plate 4, which is located on top of the positioning base 51.

[0058] Furthermore, the mounting base 1 is also connected to a second calibration plate 6, which is located on the side wall of the mounting base 1.

[0059] Mounting base 1 is used to support the arc motion frame 3 and is a key component for installing the arc motion frame 3 and other subsequent components.

[0060] The arc motion frame 3 has a hemispherical structure and is rotatably connected to the mounting base 1. Its design allows the end effector 5 to move flexibly within the hemispherical space, providing sufficient operating space and providing a basis for the positioning and orientation of the end effector 5.

[0061] The first motor 7 is connected to the circular motion frame 3 and is used to drive the circular motion frame 3 to rotate around the axis to realize the robot's posture adjustment.

[0062] The positioning seat 51, which supports the spherical rotating pair 52, the second motor 53 and the locking knob 54, can slide along the arc motion frame 3, enhancing the flexibility of surgical operations.

[0063] Neck brace 2 is used to support the patient's head, providing stable support and ensuring that the patient's head remains in a relatively fixed position during surgery, thus avoiding any impact on the precision of the surgical procedure due to head movement.

[0064] The second motor 53 is connected to the positioning seat 51 and is used to drive the positioning seat 51 to make circular motion on the circular motion frame 3, so as to assist in the automated operation of the surgical robot and improve the accuracy and efficiency of the surgery.

[0065] The spherical rotating joint 52, located on the side wall of the positioning seat 51, has three equivalent degrees of freedom, allowing manual adjustment of the orientation and angle of the surgical robot to ensure the precise guidance of instruments such as puncture needles; its center is provided with a through channel to guide the insertion path of surgical instruments such as puncture needles.

[0066] The locking knob 54, located at the bottom of the positioning seat 51, is used to lock the position of the spherical rotating joint 52 and the second motor 53, ensuring the stability of the surgical robot during the operation, preventing instrument deviation, and ensuring the safety and accuracy of the operation.

[0067] A 3D camera is used to acquire three-dimensional point cloud data of the surgical area, providing real-time visual feedback and navigation support for surgical procedures.

[0068] The processor, electrically connected to the 3D camera, the first motor 7, and the second motor 53, is used to process data and control the movement of the motors, enabling precise positioning and operation of the surgical robot.

[0069] The first calibration plate 4 and the second calibration plate 6 are mainly used as calibration tools in the surgical robot. Their functions include providing feature points with known geometric features, assisting in establishing the mapping relationship between the camera coordinate system and the robot coordinate system, supporting data acquisition and preprocessing, being used for feature point extraction and coordinate determination, helping to measure the relative pose between the 3D camera and the robot, verifying and optimizing calibration results, and assisting in achieving accurate positioning of the patient in the surgical space, thereby providing a basis for the precise operation of surgical instruments. In this embodiment, both the first calibration plate 4 and the second calibration plate 6 have regularly arranged black and white squares (chessboard pattern) on their surfaces, and the first calibration plate 4 has a 90° L-shaped structure.

[0070] In this embodiment, for better illustration, a control method applicable to the aforementioned surgical assistive robot is also provided. A surgical assistive robot control method includes:

[0071] S1. Construct the inverse kinematics solution formula for the robot;

[0072] S2. Based on the inverse kinematics solution formula, construct the forward kinematics solution formula for the robot;

[0073] S3. Construct a hand-eye calibration model;

[0074] S4. Based on the hand-eye calibration model, perform point cloud registration and determine the optimal registration solution through root mean square error;

[0075] S5. Based on the optimal registration solution and the robot's forward and inverse kinematics calculation formulas, determine the puncture and guide points and calculate the corresponding motion paths.

[0076] Further, S1 includes the following steps:

[0077] S1.1 Set the coordinate system and initial parameters, and construct the reference spherical coordinate system;

[0078] S1.2. Determine the relevant vectors and planes after the robot's motion;

[0079] S1.3. Solve for the relevant vectors;

[0080] S1.4. Construct the inverse kinematics solution formula through geometric and vector operations.

[0081] Specifically as follows:

[0082] S1.1 Set the coordinate system and initial parameters, and construct the reference spherical coordinate system;

[0083] Inverse kinematics calculation determines the corresponding joint motor angle by using the position of the end effector 5;

[0084] It should be noted that in this embodiment, the end tool refers to the spherical revolute joint 52, and the center point of the central through-channel of the spherical revolute joint 52 is the end tool point, which will not be described again later;

[0085] The 0° angle of the first motor 7 is set to the horizontal position of the circular motion frame 3, and the 0° angle of the second motor 53 is set to the center position of the circular motion frame 3. The rotation angle of the first motor 7 is α, and the rotation angle of the second motor 53 is β.

[0086] like Figure 2 As shown, the initial horizontal position of the arc structure component is denoted as arc C', and the end tool point is denoted as point p'. Arc C is the position of arc C' after rotating by an angle α, and the corresponding position of point p' after rotation is point p. In the actual scenario, the limit angle of the first motor 7 is... The limit of the second motor at angle 53 is .

[0087] S1.2. Determine the relevant vectors and planes after the robot's motion;

[0088] like Figure 2 As shown, after rotating arc C' towards arc C by an angle α, arc C and the y-coordinate axis form plane A. The spherical coordinates of point p are given by [formula missing]. The corresponding Cartesian coordinates are [formula missing]. Er coordinates Thus we obtain ,vector The unit vector is denoted as v. x e y、 e z These are the unit vectors for the X, Y, and Z axes, respectively. We obtain:

[0089] Formula 1,

[0090] Plane A can be represented by two non-collinear vectors, the y-axis and vector v. Therefore, any point Q on plane A can be represented as:

[0091] Equation 2,

[0092] In the formula:

[0093] a and b are arbitrary constants;

[0094] e y It is a unit vector along the y-axis.

[0095] like Figure 2 As shown, the vector defined by the intersection of plane A and plane zox is denoted as l, which is a special vector. vector.

[0096] Where, vector It can also be represented by Equation 2.

[0097] S1.3. Solve for the relevant vectors;

[0098] Let vector Since y=0, it is easy to obtain ; , We can obtain:

[0099] Formula 3,

[0100] In the formula:

[0101] e x It is a unit vector along the x-axis;

[0102] α is the angle rotated by the first motor 7;

[0103] β is the angle rotated by the second motor 53.

[0104] S1.4. Construct the inverse kinematics solution formula through geometric and vector operations.

[0105] α and β can be obtained:

[0106] Equation 4,

[0107] Equation 5,

[0108] In the calculation results of Equations 4 and 5 The sign of β is related to the sign of φ, and the sign of α is related to whether θ is greater than φ. Related. In practical applications, the motor angle range obtained from Equations 4 and 5 is based on the position points in the actual spherical coordinate system. It conforms to the motor's angle limit;

[0109] The specific process of positioning, orientation, alignment, and motor angle calculation for any point within the workspace using the end effector is as follows:

[0110] Let C be the spherical workspace enclosed by the circular arc motion frame 3 in the robot coordinate system, and select the puncture point. A (x1, y1, z1), guide point B If (x2, y2, z2) ∈ C, then the direction vector is... For the puncture direction of the end-effector, this forward kinematics method can convert the vector The corresponding motor angle is calculated so that the end effector can locate and orient the alignment point. A The end effector can be directed to any point within the spherical workspace enclosed by the circular motion frame 3, forming an end effector point and a point-to-point connection. A and points B A straight line between three points.

[0111] Because the end tool point on the circular motion frame 3 is at the origin O Since the distance R is a fixed value, the following equation 6 represents the range of motion trajectory D that the end tool point can reach:

[0112] Formula 6

[0113] Here, x>0 is because the range of motion of the circular motion frame 3 is limited to quadrants I, IV, V, and VIII of the robot coordinate system.

[0114] Point A (x1, y1, z1) and point B Connect (x2, y2, z2) with a straight line, let The equation of the line containing the parameter t is:

[0115] Equation 7,

[0116] Will p(t) Substituting into equation D, we get:

[0117] Formula 8,

[0118] This equation has a solution. t 1. Corresponding line AB Intersection with equation D C (x) t1 ,y t1 ,z t1 ), the intersection C It is along Direction aligned AThe position of the tool point at the end of the point.

[0119] Then point C Converting to spherical coordinates (r1, θ1, φ1), and then substituting θ1 and φ1 into equations 4 and 5, yields the solution for the alignment point of the end-effector. A The motor angles α and β.

[0120] Further, S2 includes the following steps:

[0121] S2.1. Clearly define the known joint motor angles and the fixed position relationship of the end effector;

[0122] S2.2. Perform vector decomposition;

[0123] S2.3. Combine the equations in the inverse kinematics solution to complete the construction of the forward kinematics solution formula.

[0124] Specifically as follows:

[0125] S2.1. Determine the relationship between the known joint motor angles and the fixed position of the end effector;

[0126] like Figure 2 As shown, the forward kinematics is obtained by solving for the spherical coordinate system position of the end effector 5 using known joint motor angles (α and β). r, θ, φ Since the end effector 5 is fixed on the rigid circular arc motion frame 3, the value of r is the radius of the circular arc motion frame 3.

[0127] S2.2. Perform vector decomposition;

[0128] like Figure 2 As shown, the position of the end effector 5 in the spherical coordinate system is first solved for θ. Figure 2 Normalize vector l to obtain vector w. Then, v is orthogonally decomposed in plane A into vector w and components along the y-axis:

[0129] Formula 9,

[0130] Depend on Figure 2 It can be seen that, ,

[0131] Calculation yields .

[0132] S2.3. Combine the inverse kinematics solution formula to complete the forward kinematics solution formula.

[0133] Based on the given conditions and Equation 10, we can deduce that:

[0134] Formula 10,

[0135] Therefore, the expression for θ can be obtained as:

[0136] Formula 11,

[0137] The obtained This satisfies the range of values ​​for its spherical coordinate system under actual conditions.

[0138] By combining equations 4 and 10, we can find... φ :

[0139] Formula 12

[0140] Among them, the obtained The sign of φ is determined by the sign of β, and finally... This satisfies the range of values ​​for its spherical coordinate system under actual conditions.

[0141] Further, S3 includes the following steps:

[0142] S3.1. Based on the correspondence between feature points of real objects in space under different coordinate systems, construct a mathematical mapping model;

[0143] S3.2. Perform data acquisition and preprocessing;

[0144] S3.3. Perform feature point extraction and coordinate determination;

[0145] S3.4. Construct a robot coordinate system and obtain the robot's coordinates in the coordinate system;

[0146] S3.5. Solve for the rigid body transformation parameters.

[0147] Specifically as follows:

[0148] S3.1. Based on the correspondence between feature points of real objects in space under different coordinate systems, construct a mathematical mapping model;

[0149] Based on the integration of visual information, a mathematical mapping model is constructed by examining the correspondence between feature points of real objects in space under different coordinate systems. In practical applications, a real object is represented by several feature points in space, and their positions in different coordinate systems have a one-to-one correspondence. Let P be the representation of these feature points in the camera coordinate system. cam， In the robot coordinate system, it is represented as P. rob， By obtaining multiple sets of corresponding point pairs {P cam， P rob This establishes a spatial mapping between the two coordinate systems. According to the classical rigid body transformation model, this mapping can be expressed as:

[0150] Equation 13,

[0151] In the formula:

[0152] R∈SO(3) denotes the rotation matrix, t∈R 3 This represents the translation vector.

[0153] It should be noted that in three-dimensional coordinate system transformation, rigid transformation refers to the process of transforming an object from one coordinate system to another while keeping its shape and size unchanged.

[0154] The optimal values ​​of R and t can be obtained using classical rigid body registration algorithms such as SVD (Singular Value Decomposition). Specifically:

[0155] Let point P in the 3D camera coordinate system cam The set is Point P in the robot coordinate system rob The set is Let N be the number of points. To register points in two coordinate systems with minimal error, we need to find an optimal rotation matrix R and translation vector t that minimizes the following objective function:

[0156] Formula 14

[0157] The following are the specific steps for solving R and t based on Equation 14:

[0158] (1) Calculate the centroid

[0159] Formula 15,

[0160] In the formula:

[0161] The centroid (center point) of all points in the 3D camera coordinate system;

[0162] The centroid (center point) of all points in the robot coordinate system.

[0163] (2) Remove the center of mass

[0164] Formula 16

[0165] In the formula:

[0166] For points in the 3D camera coordinate system after centroid removal;

[0167] This refers to a point in the robot's coordinate system after centroid removal.

[0168] (3) Construct the covariance matrix H and perform singular value decomposition (SVD).

[0169] Equation 17,

[0170] In the formula:

[0171] H is the covariance matrix, used to represent the covariance relationship between the centroid-decentrated 3D camera coordinate system points and the robot coordinate system points.

[0172] (4) Calculate the rotation matrix R

[0173] Equation 18,

[0174] (5) Calculate the translation vector

[0175] Equation 19

[0176] The obtained R and t are what we are looking for;

[0177] S3.2. Perform data acquisition and preprocessing;

[0178] A second calibration plate 6 is fixedly installed on the side of the robot. An RGB image and a depth image are captured by the RGB module of the 3D camera and the depth camera module. First, the depth information in the depth image is subjected to a mean filter. Then, the RGB image and the depth image are aligned and a mapping table of pixel coordinate positions between the two is output.

[0179] In this embodiment, both the first calibration plate 4 and the second calibration plate 6 have regularly arranged black and white squares (chessboard pattern) on their surfaces.

[0180] S3.3. Perform feature point extraction and coordinate determination;

[0181] The algorithm in OpenCV is used to identify the inner corner points of the first calibration plate 4 in the RGB image and output the corresponding pixel coordinates in the RGB image. Then, the corresponding pixel positions on the depth map are output through the mapping table. The depth information Z in the filtered depth map is then read, and the three-dimensional coordinates P of the inner corner point in the 3D camera coordinate system can be obtained by using Equations 21, 22, and 23. cam .

[0182] Equation 20,

[0183] Equation 21,

[0184] Formula 22

[0185] In the formula:

[0186] Z represents depth information, and fx and fy represent focal lengths.

[0187] cx and cy are the pixel coordinates of the center of the depth image;

[0188] x, y, z are the three-dimensional coordinates of the points in the 3D camera coordinate system.

[0189] S3.4. Construct a robot coordinate system and obtain the robot's coordinates in the coordinate system;

[0190] In this embodiment, P is solved by measuring CAD design data. rob ;

[0191] By directly defining the robot coordinate system using the CAD model, the coordinates of the six corner points of the corresponding second calibration plate in the robot coordinate system can be obtained.

[0192] S3.5. Solve for the rigid body transformation parameters.

[0193] Get P cam and P rob Then, R and t are calculated by using the classic rigid body transformation estimation algorithm, thus completing the establishment of the hand-eye calibration model.

[0194] It should be noted that R and t here are the medium for converting points in the 3D camera coordinate system to points in the robot coordinate system, transforming the entire point cloud obtained in the 3D camera coordinate system to the robot coordinate system. This means that the calibration is complete.

[0195] Further, S4 includes the following steps:

[0196] S4.1. Collect data and generate point clouds;

[0197] S4.2. Perform preliminary rotation and alignment of the point cloud;

[0198] S4.3. Perform point cloud segmentation;

[0199] S4.4. Perform point cloud registration to determine the optimal registration solution;

[0200] Specifically as follows:

[0201] S4.1. Collect data and generate point clouds;

[0202] In the process of cerebral hematoma surgery, the localization and orientation first require the localization of the patient in the surgical space, then the localization of the patient's internal structures (including the lesion), and finally the localization of the lesion within the patient's internal space.

[0203] The patient's positioning in the operating space is achieved using a 3D camera. The 3D camera automatically starts and acquires a frame of RGB image and a depth map. To improve the accuracy of the point cloud data, the system performs median filtering on the acquired depth map and fuses it with the corresponding RGB image to generate a high-quality color point cloud of the patient's surface contour (hereinafter referred to as "camera point cloud").

[0204] The location of the patient's internal structures (including internal lesions) is achieved through CT scans; a three-dimensional point cloud map of the patient's internal structures (including internal lesions) (hereinafter referred to as "CT point cloud") is obtained through CT scan reconstruction.

[0205] In this embodiment, a high-precision structured light 3D camera is used for spatial positioning. The 3D camera can capture the patient's surface contour and form point cloud data. Since the 3D camera is fixed on the mounting platform of the surgical robot, the point cloud can be converted into a point cloud of the space where the surgical robot is located.

[0206] It should be noted that the scanning error test of CT point cloud can be carried out by preparing a phantom and placing an object of known size inside the phantom; similarly, the phantom or the patient's internal organs and lesions can also be located in this way.

[0207] The location of the patient's internal lesions in the surgical space is achieved through subsequent registration of camera point clouds and CT point clouds.

[0208] Magnetic resonance imaging (MRI) can also be used instead of CT scans to acquire point clouds.

[0209] S4.2. Perform preliminary rotation and alignment of the point cloud;

[0210] Based on the mapping relationship between the camera coordinate system and the robot coordinate system obtained in step S3, the camera point cloud acquired by the 3D camera and the CT point cloud reconstructed by CT scan are rotated and aligned. That is, the CT point cloud is transformed into a rigid body by using the camera point cloud as a fixed reference system.

[0211] S4.3. Perform point cloud segmentation;

[0212] CT point clouds contain the complete outline of the entire head, while camera point clouds only cover a portion of the head. Because there are many non-overlapping areas between the two, these irrelevant points can significantly interfere with the registration process, thus increasing errors. Therefore, to improve the accuracy and stability of registration, it is necessary to extract the overlapping areas of the CT and camera point clouds.

[0213] The specific method for cutting the point cloud is as follows: Based on the point cloud after initial alignment in step S4.2, take the coordinates of each point in the camera point cloud as the center of a sphere, set the radius of the sphere to 2mm, construct a spherical filtering range, and retain only the CT point cloud data within the spherical range.

[0214] This processing method can narrow the registration data range to the main structural region, which can not only effectively reduce registration errors, but also significantly improve the registration success rate and convergence speed.

[0215] S4.4. Perform point cloud registration to determine the optimal registration solution;

[0216] In this embodiment, the registration algorithm employs the Iterative Closest Point (ICP) method to evaluate the following three basic variants: point-to-point registration, point-to-plane registration, and plane-to-plane registration.

[0217] The ICP (Iterative Closest Point) algorithm is a classic algorithm for point cloud registration. Its core idea is to iteratively find the closest point pair between two point clouds and calculate the optimal rigid transformation (including rotation and translation) to align the source point cloud with the target point cloud in the coordinate system. This algorithm is existing technology and will not be further described here.

[0218] The optimized camera point cloud and CT point cloud exhibit spatial consistency. The registration accuracy is quantified by the root mean square error (RMSE), with a lower value indicating a higher degree of spatial fusion of the point cloud. This process achieves the mapping from the CT coordinate system to the surgical robot's operating space and aligns it with the patient's real-time position.

[0219] By systematically evaluating the RMSE performance metrics of different registration strategies, the optimal registration solution is determined. This enables accurate mapping of the patient's internal lesion location into the surgical robot's operating space, ensuring that surgical instruments can be precisely positioned and manipulated based on the lesion location in the CT image, thereby improving the precision and safety of the surgery.

[0220] S5. Based on the optimal registration solution and the robot's forward and inverse kinematics calculation formulas, the puncture and guidance points are determined, and the corresponding motion paths are calculated and adjusted in real time through visual feedback.

[0221] To better illustrate the technical solution of the present invention, the workflow of the present invention is described herein, as follows:

[0222] 1. Perform preoperative preparations and robot initialization;

[0223] (1) Fix the surgical robot to the operating table, and support and fix the patient’s head with the neck brace 2 to ensure the stability of the robot and the patient’s head during the operation.

[0224] (2) Turn on the processor, acquire images of calibration board 5 through the 3D camera, extract feature points, and construct the mapping relationship between the camera coordinate system and the robot coordinate system.

[0225] (3) Perform a CT scan, import the patient's CT scan data into the processor, reconstruct a three-dimensional lesion model, and provide a basis for surgical planning and instrument positioning.

[0226] 2. Positioning and registration during the surgical procedure;

[0227] (1) Real-time acquisition of point cloud data of the surgical area by 3D camera to reflect the surface contour and spatial position of the patient’s head.

[0228] (2) Based on the point cloud registration technology in the control method, the CT point cloud and the 3D camera point cloud are initially rotated, aligned, cut and iterated nearest point (ICP) registration is performed to determine the optimal registration solution, and the lesion location is accurately mapped to the robot operating space, so that the doctor can confirm the puncture point and guide point in the lesion location.

[0229] (3) Based on the hand-eye calibration model, determine the transformation relationship between the coordinate system of the 3D camera and the robot; combine the puncture point and the guide point with the forward and inverse kinematics to calculate the angle between the first motor 7 and the second motor 53, plan the movement path of the instrument, and make the end of the instrument accurately point to the lesion.

[0230] 3. Operation and execution of surgical instruments.

[0231] (1) The processor sends instructions to the robot motors based on the calculated angles of the first motor 7 and the second motor 53, driving the circular motion frame 3 and the end effector 5 to move. The first motor 7 controls the rotation of the circular motion frame 3, and the second motor 53 drives the positioning seat 51 to move in an arc along the circular motion frame 3. The spherical rotary joint 52 allows manual adjustment of the instrument's orientation and angle, enabling flexible and multi-angle movement of the instrument. When the spherical rotary joint 52 rotates to the appropriate position, it can be locked and fixed by the locking knob 54.

[0232] (2) The 3D camera continuously collects the checkerboard corner information of the first calibration plate 4 above the end effector 5. The processor monitors the position of the end tool in real time and calculates the deviation based on the registered coordinate system. If a deviation occurs, the processor adjusts the motor commands in real time according to the forward and inverse kinematics calculation formulas and the hand-eye calibration model to correct the position and posture of the instrument and ensure accurate operation.

[0233] It should be noted that this invention achieves real-time visual feedback by using a 3D camera in conjunction with the first calibration plate 4, thereby enabling high-precision positioning of the robot's end effector 5. The principle of real-time visual feedback in this invention is as follows:

[0234] 1. Robust estimation of the actual position of end effector 5;

[0235] The coordinates of the inner corner points of the first calibration plate 4 in the camera coordinate system are obtained by using a 3D camera. The coordinates of the inner corner points are then transformed into the robot coordinate system using R and t obtained by hand-eye calibration and registration.

[0236] Define the reference point of the robot's end effector 5 as the end-effector point P. When the robot moves in space, let the end-effector point P be defined at that moment. r The actual coordinates are (x, y, z); at the same time, let A be the center point of all corner points of the first calibration board 4 chessboard under the same state, and its actual coordinates are (a, b, c). Point A can be obtained by taking the average value after the camera identifies the corner points of the chessboard.

[0237] Equation 23,

[0238] in, r i For each interior corner point, the position of the end tool point is calculated in reverse. N is the number of interior corner points in the chessboard grid of the first calibration board 4.

[0239] Based on the robot's structure, the actual value of point A obtained through the camera can be mapped to the actual coordinates of point P.

[0240] (x, y, z). The specific mapping relationship is as follows:

[0241] Equation 24,

[0242] Equation 25,

[0243] Equation 26

[0244] By simultaneously solving equations 25, 26, and 27 above, the actual point P under this motion state can be obtained. r (x,y,z).

[0245] 2. Error quantification and feedback control.

[0246] Let P be the ideal target position of the robot's end-effector point P. t (x t ,y t ,z t The positioning error vector for a single visual feedback is:

[0247] Equation 27,

[0248] in:

[0249] j is the number of feedback iterations;

[0250] It is the error between the expected position and the robustly estimated position of the current end-point tool center point.

[0251] Error The coordinates will be used as input for the next feedback iteration control.

[0252] Equation 28,

[0253] Will The inputs to the kinematic model (forward and inverse kinematics formulas) can calculate the motion angle required for the next feedback center point. Based on this motion angle, the robot will move, and the camera will continuously monitor the calibration board checkerboard and report the error. This iterative process corrects the robot's motor angles, achieving dynamic compensation for the positioning and orientation error of the end-effector's center point.

[0254] To quantify the magnitude of the error after each feedback, the Euclidean distance of the error vector is used as the evaluation of the feedback result:

[0255] Equation 29,

[0256] Ultimately, ||e j The error converges continuously to its minimum value, which is the obtained system accuracy error. When ||e j When the convergence reaches the minimum value, the visual feedback is complete.

[0257] The above description is not intended to limit the present invention in any way. Although the present invention has been disclosed above through embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for controlling a surgical robot, characterized in that, include: S1. Construct the inverse kinematics solution formula for the robot; S2. Based on the inverse kinematics solution formula, construct the forward kinematics solution formula for the robot; S3. Construct a hand-eye calibration model; S4. Based on the hand-eye calibration model, perform point cloud registration and determine the optimal registration solution through root mean square error; S5. Based on the optimal registration solution and the robot's forward and inverse kinematics calculation formulas, determine the puncture and guidance points and calculate and adjust the corresponding motion paths in real time through visual feedback; S1 includes the following steps: S1.1 Set the coordinate system and initial parameters, and construct the reference spherical coordinate system; S1.

2. Determine the relevant vectors and planes after the robot's motion; S1.

3. Solve for the relevant vectors; S1.

4. Construct the inverse kinematics solution formula through geometric and vector operations; S2 includes the following steps: S2.

1. Determine the relationship between the known joint motor angles and the fixed position of the end effector; S2.

2. Perform vector decomposition; S2.

3. Construct the forward kinematics solution formula by combining the inverse kinematics solution formula; S3 includes the following steps: S3.

1. Based on the correspondence between feature points of real objects in space under different coordinate systems, construct a mathematical mapping model; S3.

2. Perform data acquisition and preprocessing; S3.

3. Perform feature point extraction and coordinate determination; S3.

4. Construct a robot coordinate system and obtain the robot's coordinates in the coordinate system; S3.

5. Solve for rigid body transformation parameters; S4 includes the following steps: S4.

1. Collect data and generate point clouds; S4.

2. Perform preliminary rotation and alignment of the point cloud; S4.

3. Perform point cloud segmentation; S4.

4. Perform point cloud registration to determine the optimal registration solution; The surgical assistive robot includes a robot body and a processor and a 3D camera electrically connected to the robot body. The robot body includes a mounting base (1), an arc motion frame (3), a neck brace (2), a first motor (7), and an end effector (5). The arc motion frame (3) is rotatably connected to the mounting base (1), the neck brace (2) is connected to the mounting base (1), the first motor (7) is located on the side wall of the mounting base (1), and the end effector (5) is slidably connected to the arc motion frame (3). The end effector (5) includes a positioning seat (51), a second motor (53), and a spherical rotary joint (52). The positioning seat (51) is slidably connected to the arc motion frame (3). The second motor (53) is located on the top of the positioning seat (51) and is electrically connected to the processor. The spherical rotary joint (52) is located on the side wall of the positioning seat (51).

2. The method according to claim 1, characterized in that, The end effector (5) also includes a locking knob (54) located at the bottom of the positioning base (51).

3. The method according to claim 2, characterized in that, The end effector (5) is also connected to a first calibration plate (4), which is located on top of the positioning base (51).

4. The method according to claim 1, characterized in that, The mounting base (1) is also connected to a second calibration plate (6), which is located on the side wall of the mounting base (1).

Images