Laparoscopic master-slave robot calibration device and method

Abstract

The present application discloses a laparoscopic master-slave robot calibration device and method. The method includes: obtaining the initial pose of the micro manipulator in the retracted state and the central positions of a plurality of pre-planned surgical targets, determining the target area reachable by the micro manipulator in the motion plane, and controlling the manipulator to move to a predetermined position in the target area; based on the target area, adopting a spatial stratification strategy to generate each spatial stratification structure and the surgical targets it contains; based on the pre-planned execution order of the surgical targets, sequentially traversing the position information of the surgical targets and the spatial stratification structure to generate a target movement strategy for the current micro manipulator to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequences corresponding to the target movement path; controlling the micro manipulator to move to the surgical target for alignment based on the target movement strategy. Thereby, the accuracy and efficiency of laparoscopic surgery calibration are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of micro-robot positioning, and particularly to a calibration device and method for laparoscopic master-slave robots. Background Art

[0002] In laparoscopic surgery, doctors need to insert surgical instruments such as micro-manipulators into the abdominal cavity through small incisions for operation. Traditional surgical robot solutions often use large manipulators, which, although having flexible joints and high degrees of freedom, are complex to operate, costly to use, and prone to causing surgical time delays. Micro-manipulators, due to their small size and high flexibility, are more suitable for high-precision positioning and operation in local areas. However, traditional positioning and alignment methods require frequent manual movement and fixation of the manipulator, which not only increases the operation time and risk, but also reduces the precision and efficiency of the surgery.

[0003] Chinese invention patent application No. 202210902656.7 discloses a positioning and alignment method for a micro-manipulator, which reduces frequent manual movement and fixation operations by determining the target area and controlling the micro-manipulator to move to this area at one time; through precise calculation and planning, it ensures that the micro-manipulator can accurately align multiple targets, improving the precision and efficiency of the surgery.

[0004] However, in laparoscopic surgery, local area positioning of the micro-manipulator at the end of the master-slave robot faces multiple technical challenges. First, the micro-manipulator needs to move frequently in the limited surgical space to operate on different targets, and the positioning accuracy of the manipulator must be high enough to ensure the precision of the surgical operation. However, due to the physical environment of the surgical area may change dynamically due to organ displacement or tissue deformation, the manipulator needs to quickly adjust its own pose to adapt to these changes. In addition, the manipulator may encounter physical space limitations or interference with other instruments during movement, which further increases the complexity of path planning and may affect the positioning accuracy and calibration efficiency. Thus, it constitutes the core technical problem of local area positioning of the master-slave robot in laparoscopic surgery. Summary of the Invention

[0005] The present application provides a calibration device for laparoscopic master-slave robots, improving the precision and efficiency of laparoscopic surgery calibration.

[0006] The present application provides a calibration method for laparoscopic master-slave robots, including:

[0007] S101, obtaining the initial pose of the micro-manipulator in the retracted state and the central positions of multiple pre-planned surgical targets, determining the target area reachable by the micro-manipulator in the motion plane, and controlling the manipulator to move to a predetermined position in the target area to complete rough positioning;

[0008] S102. Based on the target area, adopt a preset spatial stratification strategy to classify and store surgical targets, generating each spatial stratification structure and the surgical targets it contains;

[0009] S103. Based on the pre-planned execution order of surgical targets, sequentially traverse the position information and spatial stratification structure of the surgical targets to generate a target movement strategy for the current micromanipulator to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequences corresponding to the target movement path;

[0010] S104. Control the micromanipulator to move to the surgical target based on the target movement strategy for alignment;

[0011] S105. After the manipulator aligns with the surgical target and completes the surgical operation, obtain the next surgical target, and repeat steps S103 to S104 until the precise positioning and operation of all surgical targets are completed.

[0012] Preferably, the preset spatial stratification strategy specifically includes:

[0013] C1. Divide the target area space into multiple layer areas according to the anatomical structure and operation risk level of laparoscopic surgery. Each layer area corresponds to a spatial stratification structure, and each spatial stratification structure is assigned a unique number;

[0014] C2. Match the surgical target according to its position information and the regional space range to the corresponding spatial stratification structure, and generate a hierarchical position distribution map of the surgical target through all spatial stratification structure information.

[0015] Preferably, C1 specifically includes:

[0016] Based on the target area, establish a three-dimensional model of the surgical workspace. According to the spatial form and organ distribution of the three-dimensional model, use a three-dimensional reconstruction algorithm to extract the organ contour from the patient's CT / MRI data and define the stratification boundary:

[0017] The mathematical representation of each layer area is:

[0018]

[0019] Among them, is the spatial constraint function of the i-th layer, set as the Euclidean distance from the organ contour surface, is the safety threshold of the i-th layer, set according to expert experience.

[0020] Preferably, in S103, generating a target movement strategy for the current micromanipulator to move to the surgical target specifically includes:

[0021] According to the hierarchical position distribution map, using the pre-set path cost algorithm, obtain the target movement path for the robotic arm to move to the surgical target;

[0022] The pre-set path cost algorithm specifically includes:

[0023] S201, adopt a pre-set path planning algorithm to generate at least one smooth motion trajectory from the current position to the surgical target in the robotic arm joint space;

[0024] S202, perform discretization processing on the generated at least one smooth motion trajectory, extract the position information of each discrete point, record the position of the end effector, and through the trajectory length calculation method, accumulate the Euclidean distance between adjacent discrete points to obtain the total length of the entire path, denoted as the path length;

[0025] S203, perform curvature analysis on the discrete trajectory, detect local curvature mutation points, determine the position information of the turning points, and record the number of turning points;

[0026] S204, store the path length, turning point pose information and its quantity of at least one smooth motion trajectory in association with the surgical target to form a surgical path planning database;

[0027] S205, based on the surgical path planning database, obtain the data associated with the surgical target, including the path length, turning point pose information and its quantity of each smooth motion trajectory, and calculate the cost value of each smooth motion trajectory according to the following formula:

[0028]

[0029] where L is the path length, is the number of turning points, n is the number of hierarchical structure crossings in the path, is the crossing weight of the i-th hierarchical structure crossing, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost, respectively, and are set according to the requirements of the actual situation.

[0030] S206, select the smooth motion trajectory with the minimum cost value as the target movement path for the robotic arm to move to the surgical target, and generate the target movement parameters of the target movement path.

[0031] Preferably, the crossing weight Obtained from a preset inter-layer transfer cost matrix, obtain the transfer direction of the i-th hierarchical structure crossing in the smooth motion trajectory. The transfer direction includes the initial hierarchical structure number and the target hierarchical structure number of the transfer. Based on the transfer direction, obtain the value corresponding to the transfer direction in the preset inter-layer transfer cost matrix as the crossing weight of the i-th hierarchical structure crossing , the preset inter-layer transfer cost matrix is as follows:

[0032]

[0033] Among them, W represents the inter-layer transfer cost matrix, and the number of layers into which the target area is divided is k, represents the crossing weight from hierarchical structure k to hierarchical structure 1, represents the crossing weight from hierarchical structure 1 to hierarchical structure k, and the specific value is determined according to the situation of the surgical target within the hierarchical structure and expert experience.

[0034] Preferably, the S104 further includes:

[0035] During the movement of the robotic arm, obtain the pose information of the robotic arm in real time, and judge whether there is a deviation from the target movement strategy. If there is a deviation and the calibration condition is met, perform pose calibration, specifically including:

[0036] D1. During the movement of the robotic arm, obtain the pose information of the end of the robotic arm in real time, compare the real-time pose information with the pose sequence in the target movement strategy, and calculate the minimum position deviation Δd and its corresponding attitude deviation Δθ = max(Δα, Δβ, Δγ);

[0037] D2. According to the safety threshold of the hierarchical structure where the robotic arm is currently located , set the dynamic calibration trigger condition. The dynamic calibration trigger condition includes the rough calibration trigger condition and the fine calibration trigger condition, which is: , which is: ;

[0038] Among them, is the preset adjustment factor corresponding to the safety threshold of the layer. Each hierarchical structure's safety threshold is set with a preset adjustment factor, which is set according to the actual situation.

[0039] Preferably, in the S105, after the robotic arm is aligned with the surgical target, it further includes:

[0040] Obtain the spatial environment information of the local surgical area of the surgical target, and judge whether the pose of the robotic arm adapts to the change of the spatial environment. If not, perform secondary pose calibration, specifically including:

[0041] E1. Collect depth images of the local surgical area through a laparoscopic lens device, combine preoperative CT / MRI data, extract spatial environmental features, and construct a real-time 3D point cloud model;

[0042] E2. Use the ICP algorithm to register the real-time point cloud with the preoperative model and calculate the organ displacement E = , if E is greater than the safety threshold corresponding to the layer where the point cloud data is located , it is determined as significant deformation and secondary calibration is triggered;

[0043] E3. Reclassify the displaced surgical target to the corresponding layer, update the layer position distribution map, based on the new layer structure, use the path cost algorithm to re-plan the path, update the target movement strategy of the robotic arm, and re-execute steps D1 to D2 to achieve secondary calibration.

[0044] Preferably, in the E3, when using the path cost algorithm to re-plan the path, it further includes:

[0045] Introduce a dynamic crossing weight ' = f( E, ), update the path cost algorithm:

[0046]

[0047]

[0048] where L is the path length, is the number of turning points, n is the number of layer structures crossed by the path, ' is the dynamic crossing weight from the i-th layer to the i + 1-th layer, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost respectively.

[0049] Preferably, after the S205, it further includes:

[0050] F1. Based on at least one smooth motion trajectory, use the equal-interval discretization method to discretize it into a series of discrete points, denoted as , where q = 1, 2,..., N, and N is the total number of discrete points, generate the pose information of each discrete point, and obtain the pose sequence of the smooth motion trajectory;

[0051] F2. Calculate the motion smoothness index of the smooth motion trajectory according to the following formula:

[0052]

[0053] Among them, f is the motion smoothing index of the smooth motion trajectory path, is the change rate of adjacent discrete pose information, is the average value of all change rates, and N is the number of WeChat messages in the pose sequence;

[0054] F3. Based on each smooth motion trajectory path, perform a weighted sum of its motion smoothing index and the generated cost value to obtain a new cost value to replace the original cost value.

[0055] This application also provides a laparoscopic master-slave robot calibration device, including: a positioning module, a layering module, a strategy determination module, and a calibration module;

[0056] The positioning module is used to obtain the initial pose of the micro manipulator in the retracted state and the central position of multiple pre-planned surgical targets, determine the target area reachable by the micro manipulator in the motion plane, control the manipulator to move to a predetermined position in the target area, and complete rough positioning;

[0057] The layering module is used to classify and store the surgical targets based on the target area by using a preset spatial layering strategy, and generate each spatial layering structure and the surgical targets it contains;

[0058] The strategy determination module is used to sequentially traverse the position information and spatial layering structure of the surgical targets based on the pre-planned surgical target execution order, and generate a target movement strategy for the current micro manipulator to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequence corresponding to the target movement path;

[0059] The calibration module is used to control the micro manipulator to move to the surgical target based on the target movement strategy for alignment; after the manipulator aligns with the surgical target and completes the surgical operation, obtain the next surgical target, and jump to the foregoing modules until the precise positioning and operation of all surgical targets are completed.

[0060] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0061] Classify and store the position information of the surgical targets according to the spatial range. This hierarchical differential management mechanism helps to more precisely manage the position information of the surgical targets, improves the positioning accuracy during the operation; the hierarchical management method generates a hierarchical position distribution map of the surgical targets, which is convenient for intuitively understanding the distribution of the surgical targets, helps the surgeon to have a comprehensive understanding of the surgical targets before the operation, and thus formulate a more reasonable surgical plan.

[0062] Through one-time rough positioning, the frequent manual movement and fixation operations of the micro manipulator are reduced, and the operation time and risk are lowered; through precise calculation and planning, it is ensured that the micro manipulator can accurately align with multiple surgical targets, improving the precision and efficiency of the surgery; through hierarchical management of surgical targets, the risks during the surgery are reduced and the safety of the surgery is improved; by comprehensively considering the path length, the number of turns, and the crossed hierarchical structures, a movement path more in line with the regulations of the surgical operation sequence and the requirements of surgical safety is generated, improving the efficiency and accuracy of path planning; by optimizing the movement path through the dynamic programming algorithm, the shortest path and the fewest turns are achieved, reducing the risks of collision and interference of the manipulator during the surgery and improving the safety of the surgery.

[0063] Through the generation of the target movement strategy, the accuracy and stability of the micro manipulator during movement are ensured, improving the surgical precision and efficiency; through the spatial hierarchical strategy, it helps with subsequent path optimization and pose calibration, reducing the risks during the surgery; through the automated and intelligent positioning and alignment method, the manual operations and decision-making burdens of the doctor during the surgery are reduced, improving the success rate of the surgery and the satisfaction of the patient. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic flowchart of the calibration method of the laparoscopic master-slave robot according to an embodiment of the present invention;

[0065] Figure 2 It is a structural block diagram of the calibration device of the laparoscopic master-slave robot according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] To facilitate the understanding of the present invention, the present application will be described more comprehensively with reference to the relevant drawings; the preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0067] It should be noted that the terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are only for the purpose of illustration and do not represent the only embodiment.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs; the terms used in the description of the present invention in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0069] Embodiment 1: Figure 1 It is a schematic flowchart of the calibration method for the laparoscopic master-slave robot according to the embodiment of the present invention.

[0070] As Figure 1 shown, a calibration method for a laparoscopic master-slave robot includes the following steps:

[0071] S101. Obtain the initial pose of the micro manipulator in the retracted state and the central positions of multiple pre-planned surgical targets, determine the target area of the micro manipulator in the motion plane, and control the manipulator to move to a predetermined position in the target area to complete rough positioning.

[0072] Among them, the target area is the area where the instrument clamping front end of the micro manipulator can reach at least two surgical targets when the micro manipulator is in its initial pose; the determination of the predetermined position can be set to any position in the target area, as long as the manipulator can complete rough positioning in the target area, so that it is located in the target area that can cover multiple surgical targets.

[0073] Specifically, the initial pose of the micro manipulator includes initial position information and initial attitude information, and the acquisition method is specifically as follows:

[0074] A1. Detect position markers: Detect the physical markers (including reflective points, two-dimensional codes or radio frequency identification tags) attached to the micro manipulator in the retracted state through a vision sensor or an electromagnetic encoder, and combine the physical space coordinate system (base coordinate system) of the operating table to convert the three-dimensional coordinates of the markers ( ) into the initial position of the manipulator clamping front end;

[0075] A2. Determine the initial attitude: Obtain the initial attitude angles of the end effector of the manipulator ( ) through an inertial measurement unit (IMU) or a joint angle sensor, which represents the rotation angles around x, y, and z.

[0076] Specifically, the central position of the surgical target is obtained as follows: Based on preoperative medical image data (including computed tomography or magnetic resonance imaging), extract the central coordinates of multiple surgical targets ( ) and map them to the operating table coordinate system through coordinate registration.

[0077] In some embodiments, the method for determining the target area specifically includes:

[0078] B1. According to the manipulator kinematic model and the extended length, obtain the maximum movement radius R of the manipulator. In actual situations, the maximum movement radius of the manipulator is usually pre-set, and the present invention will not elaborate on this;

[0079] B2. Project the central coordinates of the surgical target onto the motion plane of the robotic arm (defined as the plane parallel to the laparoscope mirror surface) to determine the projection point ( );

[0080] B3. Calculate the Euclidean distances from the initial position of the robotic arm to the projection points of each surgical target. If the distance is less than or equal to the maximum movement radius, mark it as an accessible target. If it is greater than the maximum movement radius, prompt to adjust the pose of the robotic arm or the layout of the surgical targets;

[0081] B4. With the initial position as the center, draw covering circles (with a radius of R) for all accessible targets, calculate the common intersection area of multiple covering circles as the target area, and extract its geometric center as the predetermined position of the target area ( ), or the predetermined position can also be randomly determined.

[0082] S102. Based on the target area, adopt a preset spatial stratification strategy to classify and store the surgical targets, and generate each spatial stratification structure and the surgical targets it contains.

[0083] In some embodiments, the preset spatial stratification strategy specifically includes:

[0084] C1. According to the anatomical structure of laparoscopic surgery (such as the peritoneal layer, vascular layer, target organ layer) and the operation risk level (high risk, medium risk, safe), divide the target area space into multiple layer area spaces, and each layer area space corresponds to a spatial stratification structure (including the area space range).

[0085] Specifically, based on the target area, establish a three-dimensional model of the surgical workspace. According to the spatial form and organ distribution of the three-dimensional model, use a three-dimensional reconstruction algorithm (such as Marching Cubes) to extract the organ contours from the patient's CT / MRI data, and define the stratification boundaries:

[0086] The mathematical representation of each layer area is: , where is the spatial constraint function of the i-th layer (such as the Euclidean distance from the organ contour surface), is the safety threshold of the i-th layer. For example, the spatial constraint function of the vascular layer may be the spatial range represented by the Euclidean distance from the vascular contour surface, and the safety threshold is set to 2 millimeters;

[0087] Thus, the surgical workspace is divided into multiple sub-areas, and each sub-area corresponds to a spatial stratification structure.

[0088] In another embodiment, according to the target area, the spatial range is divided into three layers, namely the core layer, the middle layer and the outer layer, to generate area stratification information. The spatial range of each layer is set according to the actual surgical structure and expert experience. For example, the target area of the robotic arm is a spherical spatial range with a radius of 800 millimeters. The core layer is set as a spatial range with a radius of 0 - 300 millimeters: high-risk targets (such as tumors), the middle layer is set as a spatial range with a radius of 300 - 600 millimeters: medium-risk targets (such as blood vessel branches), and the outer layer is set as a spatial range with a radius of 600 - 800 millimeters: safe operation area. The core layer, the middle layer and the outer layer respectively include different anatomical structures.

[0089] C2. According to the position information of the surgical target and the regional spatial range, match it to the corresponding spatial stratification structure.

[0090] Store the position information of the surgical target in the corresponding spatial stratification structure, and generate a stratification position distribution map of the surgical target through all spatial stratification structure information.

[0091] Thus, clear hierarchical management of the surgical target is provided, which helps with subsequent path optimization and pose calibration; through hierarchical management, the risks during the surgical process are reduced and the safety of the surgery is improved.

[0092] S103. Based on the pre-planned execution order of the surgical targets, sequentially traverse the position information of the surgical targets and the spatial stratification structure to generate a target movement strategy for the current micro-robotic arm to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters (the pose sequence corresponding to the target movement path, including a position sequence and an attitude sequence).

[0093] In some embodiments, generating a target movement strategy for the current micro-robotic arm to move to the surgical target specifically includes:

[0094] According to the stratification position distribution map, use the pre-set path cost algorithm to obtain the target movement path for the robotic arm to move to the surgical target.

[0095] The pre-set path cost algorithm specifically includes:

[0096] S201. Adopt a pre-set path planning algorithm (for example, the Rapidly-exploring Random Tree RRT algorithm) to generate at least one smooth motion trajectory from the current position to the surgical target in the robotic arm joint space, avoiding collisions and singularities.

[0097] S202. Discretize the generated at least one smooth motion trajectory, extract the position information of each discrete point, record the position of the end effector, and calculate the total length of the whole path by accumulating the Euclidean distances between adjacent discrete points through the trajectory length calculation method, denoted as the path length.

[0098] S203. Conduct curvature analysis on the discrete trajectory, detect local curvature mutation points, determine the position information of the turning points. If the number of turning points exceeds the preset turning threshold, re-plan the path, use an optimization algorithm to reduce the number of turning points, and record the number of turning points.

[0099] S204. Store the path length, turning point pose information and its quantity of at least one smooth motion trajectory in association with the surgical target to form a surgical path planning database.

[0100] S205. Based on the surgical path planning database, obtain the data associated with the surgical target, including the path length, turning point pose information and its quantity of each smooth motion trajectory, and calculate the cost value of each smooth motion trajectory according to the following formula:

[0101]

[0102] where L is the path length, is the number of turning points, n is the number of hierarchical structure crossings in the path, is the crossing weight of the i-th hierarchical structure crossing, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost respectively. For example, = 0.5, = 0.3, = 0.2.

[0103] The crossing weight of the i-th hierarchical structure crossing is obtained from the preset inter-layer transfer cost matrix, and the transfer direction of the i-th hierarchical structure crossing in the smooth motion trajectory is obtained. The transfer direction includes the initial hierarchical number and the target hierarchical number of the transfer. Based on the transfer direction, obtain the value corresponding to the transfer direction in the preset inter-layer transfer cost matrix as the crossing weight of the i-th hierarchical structure crossing , and the preset inter-layer transfer cost matrix is:

[0104]

[0105] where W represents the inter-layer transfer cost matrix, and the number of layers into which the target area is divided is k, represents the crossing weight from hierarchical structure k to hierarchical structure 1, represents the crossing weight from hierarchical structure 1 to hierarchical structure k, and the others are not elaborated here. The specific values are determined according to the situation of the surgical target within the hierarchical structure and expert experience. The greater the crossing weight, the greater the risk cost of the hierarchical structure transfer.

[0106] Specifically, the motion plane of the robotic arm is discretized into a grid map, and each grid is assigned a different weight according to the layer it belongs to. For example, the blood vessel layer may have a higher weight because the surgical risk is greater; while the peritoneum layer may have a lower weight because the surgical operation is relatively simple. Therefore, the setting of the preset inter-layer transfer cost matrix is based on actual surgical experience and expert knowledge to ensure that the path selection not only complies with the surgical operation sequence regulations but also guarantees surgical safety.

[0107] S206, select the smooth motion trajectory with the minimum cost value as the target movement path for the robotic arm to move to the surgical target, and generate the target movement parameters (pose sequence) of the target movement path.

[0108] Among them, the way to obtain the target movement parameters is as follows:

[0109] Discretize the target movement path, extract the position information of each discrete point, and form a position sequence;

[0110] Based on the target movement path, calculate the joint angle sequence of the robotic arm through inverse kinematics as the pose sequence;

[0111] Specifically, first obtain the coordinate system data of the current pose of the robotic arm and the surgical target pose (indicating the pose information when the robotic arm reaches the center position of the surgical target, including the center position information of the surgical target and the joint angle information of the robotic arm, that is, the pose information, obtained based on the pre-planned surgical target execution sequence, and the pose information of each surgical target is calibrated by preoperative planning). Assume the current pose is (x1, y1, z1, α1, β1, γ1), and the surgical target pose is (x2, y2, z2, α2, β2, γ2). Through the inverse kinematics algorithm, calculate the joint angle change to ensure that the end effector of the robotic arm can accurately reach the surgical target position. For example, use the Jacobian matrix method to solve the joint angle change. Assume the joint angle change is Δθ1 = 10°, Δθ2 = 15°, Δθ3 = 20°, and use the path planning algorithm to generate at least one smooth motion trajectory from the current position to the target position.

[0112] As an example, assume that there are two potential paths from the starting point of layer L1 to the tumor target of layer L3: Path 1 is L1→L2→L3, the path length L = 120 mm, the number of turns = 2, and the total weight of the crossings is ∑w = + = 3.5; Path 2 is L1→(directly cross L2)→L3, the path length L = 130 mm, the number of turns = 1, and the total weight of the crossings is ∑w = = 3. According to the path cost algorithm, the cost values of the two paths are calculated. By comparing the cost values of the two paths, the path with the lower cost value is selected as the target movement path.

[0113] Thus, by comprehensively considering the path length, the number of turns, and the crossed hierarchical structures, a movement path that better conforms to the surgical operation sequence regulations and surgical safety requirements is generated, improving the efficiency and accuracy of path planning. It not only considers the direct cost of the path but also the indirect cost of the path (such as the risk of crossing different hierarchical structures), thereby improving the efficiency and accuracy of path planning; ensuring the safety and feasibility of the surgical process. By introducing the crossing weight and the total cost function, it is ensured that the selected path is both efficient and safe during the surgical process, conforms to the surgical operation sequence regulations, and reduces the surgical risk.

[0114] S104, control the micromanipulator to move to the surgical target based on the target movement strategy for alignment.

[0115] Specifically, according to the target movement strategy, drive the manipulator to move to the surgical target. Through laser ranging or visual feedback, verify that the distance error Δd between the execution end of the manipulator and the center of the surgical target is ≤ 0.3 mm and the angle error Δθ is ≤ 0.1. If the verification passes, perform alignment.

[0116] S105, after the manipulator aligns with the surgical target and completes the surgical operation, obtain the next surgical target, and repeat steps S103 to S104 until the precise positioning and operation of all surgical targets are completed.

[0117] The technical solutions in the embodiments of the present application described above have at least the following technical effects or advantages:

[0118] Classify and store the position information of the surgical targets according to the spatial range. This hierarchical differentiation management mechanism helps to more precisely manage the position information of the surgical targets, improving the positioning accuracy during the surgical process; the hierarchical management method generates a hierarchical position distribution map of the surgical targets, facilitating an intuitive understanding of the distribution of the surgical targets, and helping the surgeon to have a comprehensive understanding of the surgical targets before the operation, thereby formulating a more reasonable surgical plan.

[0119] Through a rough positioning, the frequent manual movement and fixation operations of the micro manipulator are reduced, and the operation time and risk are lowered; through precise calculation and planning, it is ensured that the micro manipulator can accurately align with multiple surgical targets, improving the precision and efficiency of the operation; through the hierarchical management of surgical targets, the risks during the operation are reduced and the safety of the operation is improved; by comprehensively considering the path length, the number of turns, and the crossed hierarchical structures, a movement path that better conforms to the regulations of the surgical operation sequence and the requirements of surgical safety is generated, improving the efficiency and accuracy of path planning; by optimizing the movement path through a dynamic programming algorithm, the shortest path and the fewest turns are achieved, reducing the risk of collision and interference of the manipulator during the operation and improving the safety of the operation.

[0120] Through the generation of the target movement strategy, the accuracy and stability of the micro manipulator during movement are ensured, improving the surgical precision and efficiency; through the spatial hierarchical strategy, it helps with subsequent path optimization and pose calibration, reducing the risks during the operation; through automated and intelligent positioning and alignment methods, the manual operations and decision-making burdens of doctors during the operation are reduced, improving the success rate of the operation and the satisfaction of patients.

[0121] Example 2: During laparoscopic surgery, respiration, heartbeat, or instrument contact can cause organ displacement (typical displacement 2 - 5 mm), and the spatial positions of the surgical target and its surrounding tissues may change, which requires the manipulator to be able to adjust its pose in real time to adapt to this change, and the static hierarchical strategy in Example 1 cannot adapt.

[0122] Therefore, the embodiments of the present application are optimized based on the above embodiments.

[0123] In some embodiments, step S104 further includes:

[0124] During the movement of the manipulator, the pose information of the manipulator is obtained in real time, and it is judged whether there is a deviation from the target movement strategy. If there is a deviation and the deviation is greater than the hierarchical threshold (the hierarchical threshold is set as the product of the safety threshold of the layer where the manipulator is located and a preset adjustment factor), then pose calibration is performed.

[0125] Specifically, it includes:

[0126] D1. During the movement of the manipulator, the pose information of the end of the manipulator is obtained in real time through a joint angle sensor, an IMU, and a laparoscopic vision feedback system, and the real-time pose information is compared with the pose sequence in the target movement strategy to calculate the minimum position deviation Δd and its corresponding attitude deviation Δθ = max(Δα, Δβ, Δγ).

[0127] D2. According to the safety threshold of the layer where the manipulator is currently located , set the dynamic calibration trigger condition:

[0128]

[0129] Among them, is a preset adjustment factor corresponding to the hierarchical safety threshold. Each hierarchical structure's safety threshold is set with a preset adjustment factor. The smaller the safety threshold, the smaller the adjustment factor, which is set according to the actual situation, so as to combine the static division and dynamic calibration of hierarchical management. For example: the blood vessel layer (with a smaller safety threshold and high risk) is set with a more stringent calibration threshold ( = 0.5, adjustment factor is smaller), and the outer layer (with a larger safety threshold and safety zone) uses a loose threshold ( = 0.8, adjustment factor is larger), reducing unnecessary interventions.

[0130] D3. Coarse calibration: Adjust the joint angles through inverse kinematics to make the deviation return within the threshold (for example, the joint 3 rotates 3°, reducing Δd from 1.5 mm to 0.8 mm); Fine calibration: Enable visual servo control, capture the feature points of the surgical target through the laparoscopic RGB-D camera, calculate the pose deviation between the end effector and the surgical target, and dynamically adjust using a PID controller until Δd ≤ 0.3 mm and Δθ ≤ 0.1°.

[0131] Thereby, the accuracy and stability of the robotic arm during movement are ensured. Through real-time pose calibration, the deviation and collision risks during the surgery are reduced.

[0132] In some embodiments, in step S105, after the robotic arm aligns with the surgical target, it further includes:

[0133] Obtain the spatial environment information of the local surgical area of the surgical target (such as organ position, tissue morphology, and spatial structure), determine whether the pose of the robotic arm adapts to the changes in the spatial environment, and if not, perform secondary pose calibration.

[0134] Specifically, it includes:

[0135] E1. Collect the depth images of the local surgical area at a frequency of 30 fps through the laparoscopic lens device (TOF camera), combine with the preoperative CT / MRI data, extract the spatial environment features, and construct a real-time 3D point cloud model.

[0136] E2. Use the ICP algorithm to register the real-time point cloud with the preoperative model (a three-dimensional space model constructed using the preoperative CT / MRI data), and calculate the organ displacement E = , if E is greater than (obtaining the safety threshold corresponding to the layer where the point cloud data is located), it is determined as significant deformation (such as the organ moving 3 mm due to breathing), and secondary calibration is triggered.

[0137] E3. Reclassify the displaced surgical target to the corresponding layer (such as the tumor moving from layer L3 to layer L2), update the layer position distribution map, based on the new layer structure, use the path cost algorithm to re-plan the path, update the target movement strategy of the robotic arm, and re-execute steps D1 to D3 to achieve secondary calibration.

[0138] Example: The tumor position moves by ΔE = 3 mm due to breathing (exceeding the threshold of layer L3, δ3 = 1 mm), and the system automatically reclassifies the target to layer L2 and generates a new path.

[0139] Among them, during the secondary calibration process, combined with the feedback of the six-axis force sensor inside the robotic arm, the contact force between the robotic arm and the tissue is detected. If it exceeds the force safety threshold (such as F > 5 N), admittance control is used to adjust the end pose to avoid tissue damage;

[0140] Thus, the continuity and accuracy of the surgical process are ensured. Through secondary pose calibration, it adapts to the spatial environment changes during the surgical process and improves the success rate of the surgery.

[0141] The technical solutions in the embodiments of the present application described above have at least the following technical effects or advantages:

[0142] Through real-time pose calibration, the consistency between the pose of the robotic arm during movement and the target movement strategy is ensured, reducing the deviation and collision risks; during the surgical process, due to the influence of various factors (such as breathing, organ movement, tissue deformation, etc.), the pose of the robotic arm may deviate. Secondary pose calibration adapts to the spatial environment changes during the surgical process, such as organ displacement, tissue deformation, etc., improving the success rate of the surgery; real-time pose calibration and secondary pose calibration can timely detect and correct the pose deviation of the robotic arm, avoiding surgical risks caused by excessive deviation; by updating the layer position distribution map and re-planning the path, the continuity and accuracy of the surgical process are ensured.

[0143] Real-time pose calibration and secondary pose calibration reduce the manual adjustment and intervention during the surgical process, improving the automation and intelligence level of the surgery.

[0144] Embodiment 3: Due to factors such as breathing and organ movement, the spatial positions of the surgical target and its surrounding tissues may change. When performing secondary calibration in a scenario based on spatial environment changes, although the path and hierarchical structure information are updated based on the path cost algorithm, the cross-layer in the path cost algorithm does not consider the influence of the deformation amount caused by factors such as breathing and organ movement during the surgical process on path selection. There is no path cost algorithm targeted at the dynamic spatial environment, and the static crossing weight may not accurately reflect the true risks and costs of crossing the deformed area during the surgical process.

[0145] Therefore, the embodiment of the present application is optimized to a certain extent on the basis of the above embodiments.

[0146] In some embodiments, in step E3, when using the path cost algorithm to re-plan the path, it further includes:

[0147] Introduce a dynamic crossing weight ' = f( E, ), and update the path cost algorithm:

[0148]

[0149]

[0150] where L is the path length, is the number of turning points, n is the number of hierarchical structures crossed by the path, ' is the dynamic crossing weight from the i-th layer to the i + 1-th layer, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost respectively. For example, = 0.5, = 0.3, = 0.2.

[0151] Thus, a dynamic weight is introduced in the path cost value calculation, enabling the path selection to introduce the deformation amount index and avoiding the deformed area as much as possible.

[0152] The technical solutions in the embodiments of the present application at least have the following technical effects or advantages:

[0153] By introducing dynamic crossing weights, path planning can more intelligently adapt to changes in the spatial environment during the operation, especially in the deformed area. Instead of relying solely on static hierarchical information and the surgical target position, it can consider the deformation of the surgical area in real time, making the path selection more in line with the actual surgical needs. Smarter path planning can reduce risks during the operation, such as collisions and accidental resections, thereby improving the safety and success rate of the operation. By considering changes in the spatial environment in real time and adjusting the path planning, the automation and intelligence level of the operation are improved, reducing the manual operation and decision-making burden on doctors.

[0154] Embodiment 4: Further limit the path cost algorithm in Embodiment 1.

[0155] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0156] In some embodiments, after step S205, the following steps are further included:

[0157] F1. Based on at least one smooth motion trajectory, use the equal-interval discretization method to discretize it into a series of discrete points, denoted as , where q = 1, 2,..., N, and N is the total number of discrete points. Generate the pose information of each discrete point to obtain the pose sequence of the smooth motion trajectory.

[0158] F2. Calculate the motion smoothness index of the smooth motion trajectory according to the following formula:

[0159]

[0160] where f is the motion smoothness index of the smooth motion trajectory path, is the change rate of adjacent discrete point pose information (set as the weighted sum of the position information change rate and the attitude information change rate, and the change rate is the absolute value of the difference between adjacent discrete point pose information and the ratio of the pose information of the previous discrete point), is the average value of all change rates, and N is the number of WeChat information in the pose sequence.

[0161] F3. Based on each smooth motion trajectory path, perform a weighted sum of its motion smoothness index and the generated cost value to obtain a new cost value to replace the original cost value.

[0162] Among them, the weight values of the motion smoothness index and the generated cost value are set to 0.7 and 0.3, specifically:

[0163]

[0164] C’ is the new cost value, C is the original cost value, and f is the motion smoothness index.

[0165] By calculating the change rate of adjacent discrete pose information and finding its average value, a motion smoothing index is obtained. This index reflects the smoothness of the path during motion. The smaller the change rate, the smoother the path.

[0166] The technical solutions in the embodiments of the present application at least have the following technical effects or advantages:

[0167] By introducing the motion smoothing index, the path planning algorithm is more inclined to select a smooth path when choosing a path, reducing the jitter and mutation of the robotic arm during movement, improving the stability and safety of the surgery; the smooth path can reduce the risk of collision and interference during the surgery, improving the success rate of the surgery. At the same time, the smooth path also conforms more to the surgeon's operating habits, reducing the operating burden on the surgeon.

[0168] By comprehensively considering various factors (path length, number of turning points, crossing weight, motion smoothness), the path planning algorithm is more intelligent and flexible, and can better adapt to different surgical scenarios and requirements.

[0169] Furthermore, the embodiment of the present invention also provides a laparoscopic master-slave robot calibration device.

[0170] Figure 2 It is a structural block diagram of the laparoscopic master-slave robot calibration device according to the embodiment of the present invention.

[0171] As Figure 2 shown, the laparoscopic master-slave robot calibration device includes: a positioning module, a layering module, a strategy determination module, and a calibration module;

[0172] The positioning module is used to obtain the initial pose of the micro-robotic arm in the retracted state and the central position of a plurality of pre-planned surgical targets, determine the target area reachable by the micro-robotic arm in the motion plane, control the robotic arm to move to a predetermined position in the target area, and complete rough positioning;

[0173] The layering module is used to classify and store the surgical targets based on the target area, adopt a preset space layering strategy, and generate each space layering structure and the surgical targets it contains;

[0174] The strategy determination module is used to sequentially traverse the position information and space layering structure of the surgical targets based on the pre-planned surgical target execution order, and generate a target movement strategy for the current micro-robotic arm to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequence corresponding to the target movement path;

[0175] The calibration module is used to control the micro manipulator to move to the surgical target based on the target movement strategy for alignment; after the manipulator aligns with the surgical target and completes the surgical operation, the next surgical target is acquired, and the foregoing module is jumped to until the precise positioning and operation of all surgical targets are completed.

[0176] It should be noted that other specific implementation contents of the embodiments of the present invention can refer to the above-mentioned laparoscopic master-slave robot calibration method.

[0177] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A calibration method for laparoscopic master-slave robots, characterized in that, Including: S101. Obtain the initial pose of the micro manipulator in the retracted state and the central positions of multiple pre-planned surgical targets, determine the target area reachable by the micro manipulator in the motion plane, control the manipulator to move to a predetermined position in the target area to complete rough positioning; S102. Based on the target area, adopt a preset spatial stratification strategy to classify and store the surgical targets, and generate each spatial stratification structure and the surgical targets it contains; S103. Based on the pre-planned surgical target execution sequence, sequentially traverse the position information of the surgical targets and the spatial stratification structure to generate a target movement strategy for the current micro manipulator to move to the surgical target. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequences corresponding to the target movement path; The target movement path is obtained by using a path cost algorithm according to the hierarchical position distribution map: S201. Generate at least one smooth motion trajectory from the current position to the surgical target in the manipulator joint space; S202. Discretize at least one smooth motion trajectory, extract the position information of each discrete point, record the position of the end effector, and accumulate the Euclidean distance between adjacent discrete points to obtain the path length; S203. Analyze the curvature of the discrete trajectory, detect local curvature mutation points, and determine the turning point pose information and quantity; S204. Associatively store the path length, turning point pose information and its quantity of at least one smooth motion trajectory with the surgical target to form a surgical path planning database; S205. According to the path length, the number of turning points of each smooth motion trajectory associated with the surgical target in the surgical path planning database, and the crossing weight of the hierarchical structure crossed in the path, obtain the cost value of each smooth motion trajectory, and use the smooth motion trajectory with the minimum cost value as the target movement path; S104. Control the micro manipulator to move to the surgical target based on the target movement strategy for alignment; S105. After the manipulator aligns with the surgical target and completes the surgical operation, obtain the next surgical target, and repeat steps S103 to S104 until the precise positioning and operation of all surgical targets are completed.

2. The laparoscopic master-slave robot calibration method according to claim 1, wherein The preset spatial stratification strategy specifically includes: C1. According to the anatomical structure and operation risk level of laparoscopic surgery, divide the target area space into multiple layer area spaces. Each layer area space corresponds to a spatial stratification structure, and each spatial stratification structure is assigned a unique number; C2. Match the position information of the surgical target with the area space range to the corresponding spatial stratification structure, and generate a hierarchical position distribution map of the surgical target through all spatial stratification structure information.

3. The laparoscopic master-slave robot calibration method according to claim 2, wherein, The specific content of C1 includes: Based on the target area, establish a three-dimensional model of the surgical workspace, extract the organ contours from the patient's CT / MRI data using a three-dimensional reconstruction algorithm according to the spatial form and organ distribution of the three-dimensional model, and define the stratification boundary: The mathematical representation of each layer area is: ； Among them, is the spatial constraint function of the i-th layer, which is set to the Euclidean distance from the surface of the organ contour. is the safety threshold of the i-th layer, which is set according to expert experience.

4. The laparoscopic master-slave robot calibration method according to claim 2, wherein, In S103, the cost value is calculated according to the following formula: ； where L is the path length, is the number of turning points, n is the number of hierarchical structure crossings in the path, is the crossing weight of the i-th hierarchical structure crossing, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost, respectively, and are set according to the requirements of the actual situation. S206. Select the smooth motion trajectory with the minimum cost value as the target movement path for the robotic arm to move to the surgical target, and generate the target movement parameters of the target movement path.

5. The laparoscopic master-slave robot calibration method according to claim 4, wherein The spanning weight spanned by the i-th hierarchical structure is obtained from a preset inter-layer transfer cost matrix. The transfer direction of the i-th hierarchical structure spanned in the smooth motion trajectory is obtained. The transfer direction includes the initial hierarchical structure number and the target hierarchical structure number of the transfer. Based on the transfer direction, the value corresponding to the transfer direction in the preset inter-layer transfer cost matrix is obtained as the spanning weight of the i-th hierarchical structure , and the preset inter-layer transfer cost matrix is as follows: ； Among them, W represents the inter-layer transfer cost matrix, and the number of layers into which the target area is divided is k. represents the crossing weight from layer structure k to layer structure 1. represents the crossing weight from layer structure 1 to layer structure k, and the specific value is determined according to the situation of the surgical target within the layer structure and expert experience.

6. The laparoscopic master-slave robot calibration method according to claim 3, characterized in that, The S104 further includes: During the movement of the robotic arm, obtain the pose information of the robotic arm in real time, and determine whether there is a deviation from the target movement strategy. If there is a deviation and the calibration condition is met, perform pose calibration, specifically including: D1. During the movement of the robotic arm, obtain the pose information of the end of the robotic arm in real time, compare the real-time pose information with the pose sequence in the target movement strategy, and calculate the minimum position deviation Δd and its corresponding attitude deviation Δθ = max(Δα, Δβ, Δγ); D2. According to the safety threshold of the current hierarchical structure where the robotic arm is located , set the dynamic calibration trigger conditions, which include the rough calibration trigger condition and the fine calibration trigger condition, as follows: , as follows: ; Among them, is a preset adjustment factor corresponding to the hierarchical security threshold. A preset adjustment factor is set for the security threshold of each hierarchical structure and is set according to the actual situation.

7. The laparoscopic master-slave robot calibration method according to claim 6, characterized in that In S105, after the robotic arm aligns with the surgical target, it further includes: Obtain the spatial environment information of the local surgical area of the surgical target, and determine whether the pose of the robotic arm adapts to the change of the spatial environment. If not, perform secondary pose calibration, specifically including: E1. Collect the depth image of the local surgical area through the laparoscopic lens device, combine the preoperative CT / MRI data, extract the spatial environment features, and construct a real-time 3D point cloud model; E2. Use the ICP algorithm to register the real-time point cloud with the preoperative model and calculate the organ displacement E = , if E is greater than the safety threshold corresponding to the layer where the point cloud data is located , it is determined as significant deformation and secondary calibration is triggered; E3. Reclassify the displaced surgical target into the corresponding layer, update the layer position distribution map, based on the new layer structure, use the path cost algorithm to re-plan the path, update the target movement strategy of the robotic arm, and re-execute steps D1 to D2 to achieve secondary calibration.

8. The laparoscopic master-slave robot calibration method according to claim 7, wherein, In E3, when using the path cost algorithm to re-plan the path, it further includes: Introduce dynamic spanning weights '=f( E, ), update the path cost algorithm: ； ； where L is the path length, is the number of turning points, n is the number of hierarchical structures crossed by the path, ' is the dynamic crossing weight from the i-th layer to the (i + 1)-th layer, , and are the influence degrees of the path length, the number of turning points, and the crossing weight on the path cost, respectively.

9. The laparoscopic master-slave robot calibration method according to claim 4, characterized in that, After S205, it further includes: F1. Based on at least one smooth motion trajectory, using the equal-distance discretization method to discretize it into a series of discrete points, denoted as , where q = 1, 2,..., N, and N is the total number of discrete points, generating the pose information of each discrete point to obtain the pose sequence of this smooth motion trajectory; F2. Calculate the motion smoothness index of this smooth motion trajectory according to the following formula: ； where f is the motion smoothing exponent of the smooth motion trajectory path, is the change rate of adjacent discrete point pose information, is the average value of all change rates, and N is the number of WeChat messages in the pose sequence; F3. Based on each smooth motion trajectory path, perform a weighted sum of its motion smoothness index and the generated cost value to obtain a new cost value to replace the original cost value.

10. A laparoscopic master-slave robot calibration device, which is applied to a laparoscopic master-slave robot calibration method according to any one of claims 1 to 9, and is characterized in that, It includes: A positioning module, a layering module, a strategy determination module, and a calibration module; The positioning module is used to obtain the initial pose of the micro-robotic arm in the retracted state and the central positions of multiple pre-planned surgical targets, determine the target area that the micro-robotic arm can reach in the motion plane, control the robotic arm to move to a predetermined position in the target area, and complete rough positioning; The layering module is used to classify and store the surgical targets based on the target area, adopt a preset spatial layering strategy, and generate each spatial layering structure and the surgical targets it contains; The strategy determination module is used to generate the target movement strategy for the current micro-robotic arm to move to the surgical target based on the pre-planned surgical target execution order, sequentially traverse the position information and spatial layering structure of the surgical targets. The target movement strategy includes a target movement path and target movement parameters, and the target movement parameters are the pose sequence corresponding to the target movement path. The target movement path is obtained according to the hierarchical position distribution map by using the path cost algorithm: S201, generate at least one smooth movement trajectory from the current position to the surgical target in the robotic arm joint space; S202, discretize at least one smooth movement trajectory, extract the position information of each discrete point, record the position of the end effector, and accumulate the Euclidean distance between adjacent discrete points to obtain the path length; S203, perform curvature analysis on the discrete trajectory, detect local curvature mutation points, and determine the pose information and quantity of the turning points; S204, store the path length, the pose information of the turning points and their quantity of at least one smooth movement trajectory in association with the surgical target to form a surgical path planning database; S205, obtain the cost value of each smooth movement trajectory according to the path length, the number of turning points of each smooth movement trajectory associated with the surgical target in the surgical path planning database, and the crossing weight of the hierarchical structure crossed in the path, and use the smooth movement trajectory with the minimum cost value as the target movement path; The calibration module is used to control the micro-robotic arm to move to the surgical target based on the target movement strategy for alignment; After the robotic arm aligns with the surgical target to complete the surgical operation, obtain the next surgical target and jump to the foregoing module until the precise positioning and operation of all surgical targets are completed.

Images