Method, device, electronic device, storage medium and computer program product for positioning and moving surgical tools for surgical robots

Abstract

The present disclosure relates to a method, apparatus, electronic device, storage medium, and computer program product for positioning and moving surgical tools for a surgical robot, including: calculating a travel path direction based on the position of a sensor in a base coordinate system at each viewing angle, a first transformation matrix, a point cloud collected at each viewing angle, and a point cloud of preoperative medical images; obtaining a second transformation matrix and a target position; calculating a fourth transformation matrix based on the second transformation matrix, the target position, the travel path direction, and a third transformation matrix; and controlling the tip of the surgical tool to move from a current position to a target position based on the fourth transformation matrix. Thus, by mounting the sensor above the end of a robotic arm component, the robotic arm connecting rod can be used to pull the sensor around the surgical object, thereby enabling close-range, multi-viewpoint acquisition of high-precision point clouds of the surgical object, thereby improving the accuracy and efficiency of the surgery.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly, to a method, apparatus, electronic device, storage medium, and computer program product for positioning and moving surgical tools applied to a surgical robot. Background Art

[0002] Brain surgery typically involves positioning, opening holes, and access procedures, making precise positioning crucial for successful surgery. Furthermore, surface matching technology is commonly used during brain surgery. This technology aligns point clouds in different coordinate systems by registering 3D point cloud data.

[0003] In related technologies, "surface matching technology" typically uses an "eyes outside the hand" configuration, meaning the sensor is typically fixed in a certain location in the operating room and cannot be moved. However, if the sensor is fixed, the imaging distance is too large and the viewing angle is fixed, which limits the point cloud resolution and coverage. This makes it difficult to obtain comprehensive, high-precision head information, resulting in poor surgical accuracy and low efficiency. Summary of the Invention

[0004] The present disclosure provides a method, device, electronic device, storage medium and computer program product for positioning and moving surgical tools applied to a surgical robot, so as to at least solve the problem in the above-mentioned related technologies that the surgical accuracy and efficiency are poor due to the fixed sensors.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for positioning and moving a surgical tool applied to a surgical robot is provided, wherein the surgical robot comprises a robotic arm component, a sensor mounted above the end of the robotic arm component, and a surgical tool mounted at the end of the robotic arm component, wherein the robotic arm component comprises a plurality of robotic arm links connected end to end, and the method for positioning and moving the surgical tool comprises: determining the position of the sensor in a base coordinate system at each of a plurality of pre-acquired view angles; , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple perspectives are multiple perspectives obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; obtain the second transformation matrix from the robotic arm end coordinate system to the base coordinate system in the current state. and the target position that the tip of the surgical tool is expected to reach , wherein the target location is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of reaching the target point from the opening point of the surgical object; based on the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is the coordinate system at the tip of the surgical tool; based on the fourth transformation matrix , controlling the tip of the surgical tool to move from the current position to the target position .

[0006] Optionally, the position and posture of the sensor in the base coordinate system at each of the multiple perspectives acquired in advance , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , including: for the i-th perspective among the multiple perspectives, based on the posture , the first transformation matrix and the point cloud , calculate the point cloud in the base coordinate system at the i-th perspective , where i=1, 2, ..., N, where N is the number of the multiple perspectives, and N is an integer greater than 1; a plurality of point clouds corresponding to the multiple perspectives Perform stitching to obtain a complete point cloud in the base coordinate system ; Based on the complete point cloud The preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , wherein the image coordinate system is used to obtain the preoperative medical image point cloud The coordinate system corresponding to the device; based on the transformation relationship matrix , the preoperative medical imaging point cloud The location of the opening point included and the location of the target , calculate the direction of the travel path .

[0007] Optionally, the posture-based , the first transformation matrix and the point cloud , calculate the point cloud in the base coordinate system at the i-th perspective , including: calculating the point cloud by the following formula :

[0008] .

[0009] Optionally, the complete point cloud The preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , including: calculating the point cloud of the preoperative medical image through a point cloud registration algorithm The transformation relationship matrix , so that the transformation relationship matrix Transformed preoperative medical image point cloud With the complete point cloud The degree of overlap between them is greater than or equal to a preset threshold.

[0010] Optionally, the transformation relationship matrix , the preoperative medical imaging point cloud The location of the opening point included and the location of the target , calculate the direction of the travel path , including: calculating the transformation relationship matrix and the location of the opening point The product of , obtains the position of the opening point in the base coordinate system. ; Calculate the transformation relationship matrix and the location of the target The product of the target point and the base coordinate system is obtained. Based on the location and the location , calculate the direction of the travel path .

[0011] Optionally, the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , including: calculating the fourth transformation matrix by the following formula :

[0012]

[0013] in, Contains translation part and rotating part ;

[0014]

[0015]

[0016]

[0017]

[0018] in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

[0019] According to a second aspect of an embodiment of the present disclosure, a surgical tool positioning and moving device for a surgical robot is provided, wherein the surgical robot comprises a robotic arm component, a sensor mounted above the end of the robotic arm component, and a surgical tool mounted at the end of the robotic arm component, wherein the robotic arm component comprises a plurality of robotic arm links connected end to end, and the surgical tool positioning and moving device comprises: a path direction calculation module configured to calculate the position of the sensor in a base coordinate system at each of a plurality of pre-acquired view angles. , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple perspectives are multiple perspectives obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; the target position acquisition module is configured to obtain a second transformation matrix from the robotic arm end coordinate system to the base coordinate system in the current state. and the target position that the tip of the surgical tool is expected to reach , wherein the target location The tip of the surgical tool is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of reaching the target point from the opening point of the surgical object; the transformation matrix calculation module is configured to calculate the position of the tip of the surgical tool based on the second transformation matrix. , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is the coordinate system at the tip of the surgical tool; the moving module is configured to be based on the fourth transformation matrix , controlling the tip of the surgical tool to move from the current position to the target position .

[0020] Optionally, the path direction calculation module is configured to: for the i-th perspective among the multiple perspectives, based on the posture , the first transformation matrix and the point cloud , calculate the point cloud in the base coordinate system at the i-th perspective , where i=1, 2, ..., N, where N is the number of the multiple perspectives, and N is an integer greater than 1; a plurality of point clouds corresponding to the multiple perspectives Perform stitching to obtain a complete point cloud in the base coordinate system ; Based on the complete point cloud The preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , wherein the image coordinate system is used to obtain the preoperative medical image point cloud The coordinate system corresponding to the device; based on the transformation relationship matrix , the preoperative medical imaging point cloud The location of the opening point included and the location of the target , calculate the direction of the travel path .

[0021] Optionally, the travel path direction calculation module is configured to calculate the point cloud by the following formula: :

[0022] .

[0023] Optionally, the path direction calculation module is configured to: calculate the direction of the path direction of the preoperative medical image point cloud by a point cloud registration algorithm The transformation relationship matrix , so that the transformation relationship matrix Transformed preoperative medical image point cloud With the complete point cloud The degree of overlap between them is greater than or equal to a preset threshold.

[0024] Optionally, the travel path direction calculation module is configured to: calculate the transformation relationship matrix and the location of the opening point The product of , obtains the position of the opening point in the base coordinate system. ; Calculate the transformation relationship matrix and the location of the target The product of the target point and the base coordinate system is obtained. Based on the location and the location , calculate the direction of the travel path .

[0025] Optionally, the transformation matrix calculation module is configured to calculate the fourth transformation matrix by the following formula: :

[0026]

[0027] in, Contains translation part and rotating part ;

[0028]

[0029]

[0030]

[0031]

[0032] in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

[0033] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the surgical tool positioning and movement method applied to a surgical robot according to the present disclosure.

[0034] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the surgical tool positioning and movement method applied to a surgical robot according to the present disclosure.

[0035] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the surgical tool positioning and movement method applied to a surgical robot according to the present disclosure.

[0036] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0037] This disclosure utilizes an "eyes in hand" approach, specifically, sensors mounted above the distal end of the robotic arm of a surgical robot. This allows the robotic arm's linkage to pull the sensor along a predetermined trajectory around the surgical object, enabling close-range, multi-view acquisition of surgical object information. This enables high-precision point cloud capture of the surgical object from close range and multiple perspectives, improving the comprehensiveness and accuracy of the acquired surgical object information, increasing point cloud resolution and expanding coverage, thereby enhancing surgical precision and efficiency.

[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0040] Figure 1 is a schematic structural diagram illustrating a surgical robot according to an exemplary embodiment of the present disclosure;

[0041] Figure 2 is a flowchart illustrating a surgical tool positioning and moving method applied to a surgical robot according to an exemplary embodiment of the present disclosure;

[0042] Figure 3 is a block diagram illustrating a surgical tool positioning and moving apparatus applied to a surgical robot according to an exemplary embodiment of the present disclosure;

[0043] Figure 4 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0045] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0046] It should be noted that the phrase "at least one of the several items" in this disclosure includes three types of parallel situations: "any one of the several items", "a combination of any multiple of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" means the following three parallel situations: (1) performing step 1; (2) performing step 2; and (3) performing both step 1 and step 2.

[0047] In brain surgery, accurate positioning is crucial for the success of the operation. The positioning methods in related technologies have the following main defects:

[0048] 1. Limitations of traditional mechanical positioning methods: For example, if a stereotactic instrument is used, the patient's head needs to be fixed on a rigid frame, which limits the flexibility of surgical operations and increases the patient's discomfort and pain, especially in complex operations.

[0049] 2. Disadvantages of landmark-based positioning:

[0050] (1) Rigid landmarks: Metal screws need to be implanted before surgery, which increases the risk of surgical trauma and infection.

[0051] (2) Non-rigid landmarks: Use markers attached to the patient’s skin surface for positioning. Due to the deformability of the skin, this method has low positioning accuracy and is difficult to meet the high-precision requirements of intraoperative positioning.

[0052] 3. Limitations of Traditional Face Matching Technology: "Face matching technology" generally refers to methods that utilize registration between 3D point cloud data to align point clouds in different coordinate systems. Related technologies typically employ an "eyes outside the hand" configuration when performing face matching, meaning the sensor is typically fixed in a certain location in the operating room and cannot be moved. However, if the sensor is fixed, the imaging distance is excessive and the viewing angle is fixed, which limits the point cloud resolution and coverage, making it difficult to obtain comprehensive, high-precision head information.

[0053] 4. Lack of an integrated surgical system: The robot-assisted system in related technologies has a single function and fails to achieve automated and integrated operations from positioning, opening holes to entry, resulting in cumbersome and inefficient surgical procedures.

[0054] To address the aforementioned issues in related technologies, the present disclosure provides a method, device, electronic device, storage medium, and computer program product for positioning and moving surgical tools for a surgical robot. These methods utilize an "eye-in-hand" approach, specifically, sensors mounted above the distal end of a robotic arm within the surgical robot. This allows the robotic arm's connecting rod to guide the sensor along a predetermined trajectory around the surgical object, enabling close-range, multi-perspective acquisition of surgical object information. This allows for close-range, multi-perspective acquisition of high-precision point clouds of the surgical object, improving the comprehensiveness and accuracy of the acquired surgical object information. This improves point cloud resolution and expands coverage, thereby enhancing surgical precision and efficiency.

[0055] Figure 1 1 is a schematic structural diagram illustrating a surgical robot according to an exemplary embodiment of the present disclosure. Figure 1The surgical robot provided by the present disclosure may mainly include a robotic arm component (1), a sensor (2) installed above the end of the robotic arm component (1), and a surgical tool (3) installed at the end of the robotic arm component (1). Furthermore, the surgical robot may also include a quick-change disk (4) installed at the end of the robotic arm component (1), and the surgical tool (3) may be installed on the quick-change disk (4). The above-mentioned robotic arm component (1) may include a plurality of robotic arm links connected end to end. Exemplarily, the robotic arm component (1) may include four robotic arm links connected end to end, namely: a robotic arm link (11), a robotic arm link (12), a robotic arm link (13), and a robotic arm link (14).

[0056] The above-mentioned robotic arm component (1) may have multiple degrees of freedom, and illustratively, may have 6 or more degrees of freedom. It is a mechanical device that can be precisely controlled by a computer to achieve movement and positioning, and is mainly used to control the precise movement and positioning of the above-mentioned sensor (2) and surgical tool (3).

[0057] The above-mentioned sensor (2) may be, but is not limited to, a 3D sensor. For example, it may be a structured light camera, a laser scanner, etc., which is mainly used to collect high-precision point cloud data of the patient's head. In addition, in the present disclosure, a color camera may be introduced to obtain texture information of the patient's head surface, and the accuracy of surface matching and registration may be improved by combining it with the 3D point cloud; alternatively, a real-time imaging device such as an ultrasound device may be introduced to enhance the accuracy of intraoperative tissue identification and safety monitoring.

[0058] The above-mentioned surgical tools (3) can also be called end tools, which are mainly used to be precisely controlled to enter the target point in the brain for treatment or research operations. For example, the surgical tools (3) may include but are not limited to: cranial drill tools, drug syringes, electrode implantation needles, laser cutters, endoscopes, etc. Moreover, as mentioned above, these surgical tools (3) can be quickly switched through the aforementioned quick-change disk (4), thereby meeting the needs of different surgeries.

[0059] The quick-change disc (4) is used to quickly replace the surgical tool (3) installed at the end of the robotic arm, that is, it is mainly used to achieve rapid switching and reliable connection between different surgical tools (3).

[0060] In this disclosure, a cranial drill or needle tool can be integrated into a surgical robot, and a quick-change tray can be used to automatically integrate and quickly switch between various surgical tools. Furthermore, precise robotic arm control can be used to complete integrated, automated operations such as positioning, opening a hole, and entering the surgical area, achieving integrated integration of the entire surgical process, thereby simplifying the surgical process and improving surgical efficiency and precision.

[0061] Figure 2The present invention is a flowchart showing a method for positioning and moving a surgical tool applied to a surgical robot according to an exemplary embodiment of the present invention. As described above, the surgical robot may include a robotic arm component (1), a sensor (2) mounted above the end of the robotic arm component (1), and a surgical tool (3) mounted at the end of the robotic arm component (1). The robotic arm component (1) may include a plurality of robotic arm links connected end to end. Exemplarily, the robotic arm component (1) may include four robotic arm links connected end to end. It should be noted that the method for positioning and moving a surgical tool applied to a surgical robot provided by the present invention may be applicable to, but not limited to, the following scenarios: intracranial drug delivery / implantation system for large animals.

[0062] Reference Figure 2 In step 201, the position and orientation of the sensor (2) in the base coordinate system at each of the multiple perspectives acquired in advance can be obtained. , the first transformation matrix between the coordinate system of sensor (2) and the coordinate system of the end of the robot arm , point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the direction of the travel path of the tip of the surgical tool (3) The above-mentioned multiple viewing angles can be multiple viewing angles obtained by the sensor (2) when the sensor (2) is pulled by multiple robotic arm links to move around the surgical object according to a predetermined trajectory. " can be a point cloud of the skin surface of the surgical object obtained based on CT or MRI images through threshold segmentation, point cloud extraction, etc.

[0063] The above-mentioned "base coordinate system" may be the coordinate system of the target joint farthest from the end of the robotic arm component (1) among the multiple joints connecting the multiple robotic arm links. For example, return to reference Figure 1 , the three joints connecting the four robotic arm links: robotic arm link (11), robotic arm link (12), robotic arm link (13) and robotic arm link (14) can be respectively: joint A for connecting robotic arm link (11) and robotic arm link (12); joint B for connecting robotic arm link (12) and robotic arm link (13); and joint C for connecting robotic arm link (13) and robotic arm link (14). In this case, the "base coordinate system" can be the coordinate system of the joint A that is farthest from the end of the robotic arm component (1) among the three joints connecting the above four robotic arm links. Specifically, the "base coordinate system" can be a coordinate system with the center of the circle of joint A as the origin and the radial direction of joint A as the Z axis.

[0064] The above-mentioned "robot arm end coordinate system" may be a coordinate system at the end of the robot arm component (1). For example, the "robot arm end coordinate system" may be a coordinate system with the center of the end circular end of the robot arm component (1) as the origin and the radial direction of the end circular end as the Z axis.

[0065] It should be noted that the 3D sensor can be pulled by the robotic arm and moved around the head of the surgical object according to a predetermined trajectory to perform multi-view, high-resolution point cloud data acquisition. At this time, the surgical tool (3) is not installed at the end of the robotic arm to avoid obstruction and interference. Thus, in the present disclosure, by adopting the "eye on hand" solution, that is, by installing the high-resolution 3D sensor above the end of the robotic arm, it is possible to achieve close-range, multi-angle acquisition of high-precision point clouds of the surgical object's head, thereby achieving non-invasive, high-precision surface matching positioning.

[0066] In addition, before the operation begins, the preoperative medical imaging point cloud of the surgical object can be obtained. (MRI, CT, etc.), and then the 3D model of the head skin of the surgical object can be reconstructed, that is, the point cloud or mesh data of the head skin of the surgical object can be generated. Manual or automatic marking of intracranial targets Next, the cranial entry point can be determined based on the surgical requirements. , that is, the location of the opening on the surface of the skull of the surgical subject Then, you can generate the To the target The direction of the travel path of the tip of the surgical tool (3) .

[0067] It should be noted that before the operation begins, the hand-eye calibration method can also be used to determine the coordinate transformation relationship between the coordinate system of the 3D sensor (2) and the coordinate system of the end of the robotic arm. In addition, if preoperative calibration is not performed, hand-eye calibration can also be completed during surgery through multi-view shooting and algorithm estimation.

[0068] According to an exemplary embodiment of the present disclosure, for the i-th perspective among the above-mentioned multiple perspectives, the posture , the first transformation matrix and point cloud , calculate the point cloud in the base coordinate system under the i-th perspective , where i=1, 2, ..., N, where N can be the number of multiple view angles, and N can be an integer greater than 1. That is, in the present disclosure, the position and orientation of the 3D sensor in the base coordinate system at each view angle can be obtained through the kinematic information of the manipulator. , and then the point cloud collected from each perspective can be Transform to the base coordinate system.

[0069] Then, multiple point clouds corresponding to the above multiple perspectives can be Perform stitching to obtain a complete point cloud in the base coordinate system That is, in this disclosure, point cloud data from multiple perspectives can be The point cloud registration algorithm can be used to correct the accumulated error and generate a complete head skin point cloud. It should be noted that point cloud registration algorithms can include, but are not limited to, the Iterative Closest Point (ICP) algorithm, a commonly used algorithm for point cloud registration, and global registration algorithms. Furthermore, in addition to the aforementioned algorithms, more advanced algorithms, such as deep learning algorithms, can be introduced during point cloud stitching and registration to further improve computational efficiency and registration accuracy.

[0070] Next, based on the complete point cloud Preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , where the image coordinate system can be used to obtain the preoperative medical image point cloud That is, in the present disclosure, the three-dimensional head model obtained before surgery can be Point cloud collected during surgery Perform registration to solve the transformation relationship from the image coordinate system to the base coordinate system , where point cloud registration can adopt feature-based global registration algorithm or ICP algorithm.

[0071] Then, based on the above transformation relationship matrix , Preoperative medical imaging point cloud The location of the included opening points and the location of the target , calculate the direction of the travel path .

[0072] According to an exemplary embodiment of the present disclosure, the point cloud can be calculated by the following formula :

[0073] .

[0074] According to an exemplary embodiment of the present disclosure, a point cloud registration algorithm can also be used to calculate the preoperative medical image point cloud. The transformation relationship matrix , so that the transformation matrix Transformed preoperative medical image point cloud With the complete point cloud The degree of overlap between them is greater than or equal to the preset threshold. That is, a point cloud registration algorithm can be used to find a point cloud registration algorithm that acts on The transformation relationship matrix , so that the transformed The overlap is best.

[0075] Specifically, the above registration process can include two parts:

[0076] (1) First, use Fast Point Feature Histograms (FPFH) feature matching combined with Random Sample Consensus (RANSAC) algorithm for global coarse registration;

[0077] (2) Then use the point-to-plane iterative closest point algorithm (ICP) for local precise registration.

[0078] For "global coarse registration", the following three steps can be included:

[0079] 1) Downsample the point cloud to a certain isotropic resolution to reduce the amount of calculation and improve processing efficiency;

[0080] 2) Estimate the normal vector of each point and calculate the FPFH descriptor, where the FPFH descriptor is a multidimensional vector that can be used to describe the local geometric features of each point;

[0081] 3) Use the RANSAC algorithm to obtain the initial transformation. Specifically, first, we can get Three points are randomly selected from . Then, we can The FPFH vector in [ ] is used to find the most similar points to estimate the transformation matrix, which can be applied to the source point cloud. Next, the effectiveness of the transformation can be evaluated by calculating the proportion of overlapping points between the transformed source and target point clouds within a set threshold radius. By repeating this process multiple times, the transformation that results in the highest point cloud overlap can be selected as the initial transformation for local fine registration.

[0082] For “local fine registration”, the goal of point-to-plane ICP is to transform the source point cloud Points in and target point cloud The nearest plane is matched, and the optimization goal can be expressed as:

[0083]

[0084] in, yes The normal vector (length 1), superscript is the transpose symbol, yes Middle distance The nearest point.

[0085] Thus, in the present disclosure, through hand-eye calibration, tool calibration and multi-view point cloud stitching technology, the conversion relationship between each coordinate system can be accurately established, thereby ensuring high-precision registration between the preoperative image and the intraoperative coordinate system, thereby realizing the travel path direction of the tip of the surgical tool. Accurate planning.

[0086] According to an exemplary embodiment of the present disclosure, the key points in the travel path direction of the tip of the surgical tool (3) planned before surgery can be transformed into the base coordinate system. Specifically, the transformation relationship matrix can be calculated and the location of the opening point The product of , obtains the position of the opening point in the base coordinate system : ; You can also calculate the transformation relationship matrix and the location of the target The product of , obtains the position of the target in the base coordinate system : Then, based on the location and location , calculate the direction of the travel path For example, the direction of the travel path can be calculated by the following formula :

[0087]

[0088] In this way, the surgical tool (3), i.e., the drill bit of the cranial drilling tool, can be aligned with the starting point of the travel path direction. , and make the travel direction follow the planned path direction The movement is performed so that the tip of the surgical tool (3) gradually moves to the intracranial target of the surgical subject.

[0089] In step 202, the second transformation matrix from the robot end coordinate system to the base coordinate system in the current state can be obtained. and the target position that the tip of the surgical tool (3) is expected to reach , where the target position The target position may be one of the multiple positions that the tip of the surgical tool (3) is expected to reach during the process of reaching the target point from the opening point of the surgical object, that is, the "target position" " can be a position among a plurality of positions that the needle tip of the surgical tool (3) is expected to reach in the direction of the path from the opening point of the surgical object to the target point. "The second transformation matrix ” can be directly read from the robotic arm included in the surgical robot in its current state.

[0090] In step 203, based on the second transformation matrix , target location , travel path direction And the third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate the movement of the tip of the surgical tool (3) to the target position The fourth transformation matrix between the manipulator end coordinate system and the base coordinate system is , wherein the tool coordinate system may be a coordinate system at the tip of the surgical tool (3). Exemplarily, the tool coordinate system may be a coordinate system with the tip of the surgical tool (3) as the origin and the axial direction of the surgical tool (3) as the Z axis.

[0091] It should be noted that, in the present disclosure, the surgical tool (3) connected to the end of the robotic arm component (1) included in the surgical robot can be calibrated in advance to determine the fixed transformation matrix between the tool coordinate system and the robotic arm end coordinate system. .

[0092] According to an exemplary embodiment of the present disclosure, the fourth transformation matrix can be calculated by the following formula , that is, the terminal transformation after moving can be expressed as:

[0093]

[0094] in, It can be the transformation of the tool coordinate system from the initial position to the target position, which may only contain the unknown translation part and the unknown rotation part For example, let's assume 4 4-size matrix, then Can include 3 of the upper left corner of the element Matrix of size 3, Can be included 3 of the upper right corner of the element A vector of size 1. Specifically:

[0095] 1. Calculate the unknown translation part :

[0096] The origin of the tool coordinate system can be . Then according to the transformation relationship: , to get the translation vector:

[0097]

[0098] in, The third transformation matrix can be The inverse matrix of .

[0099] 2. Calculate the unknown rotation part :

[0100] The standard axis in the tool coordinate system can be , then the target direction in the tool initial coordinate system can be expressed as:

[0101]

[0102] Rotation Matrix Can satisfy:

[0103]

[0104] Among them, as mentioned above, The third transformation matrix can be The inverse matrix of Can be included For example, assume that 4 4-size matrix, then Can be included 3 of the upper left corner of the element Matrix of size 3. Can be included For example, assume that 4 4-size matrix, then Can be included 3 of the upper left corner of the element Matrix of size 3.

[0105] In addition, the rotation matrix can also be expressed in the form of "rotation axis-angle" :

[0106] The "rotation axis" can be expressed as:

[0107]

[0108] The "rotation angle" can be expressed as:

[0109]

[0110] Then, by using the Rodrigues rotation formula we can get :

[0111]

[0112] in, For the above rotation matrix The inverse matrix of is the identity matrix, The rotation axis The antisymmetric matrix of :

[0113]

[0114] In step 204, based on the fourth transformation matrix , controls the tip of the surgical tool (3) to move from the current position to the target position . That is, based on the fourth transformation matrix Align the axial direction (Z-axis direction of the tool coordinate system) of the surgical tool (3) with the target direction and gradually move the tip of the surgical tool (3) to the target position .

[0115] It should be noted that the surgical tool positioning and movement method provided by this disclosure for a surgical robot can be applied not only in brain surgery scenarios, but also in other surgical scenarios, including but not limited to spinal surgery scenarios, orthopedic surgery scenarios, etc. Moreover, in the corresponding surgical scenario, only the surgical tools need to be adjusted.

[0116] The surgical robot provided by the present disclosure can use multiple robotic arm links connected end to end and carry high-resolution three-dimensional sensors. By utilizing surface matching technology, it can achieve high-precision three-dimensional reconstruction and positioning of the patient's or experimental animal's head during surgery. Moreover, by acquiring high-resolution point clouds from multiple perspectives and combining them with preoperative medical imaging point clouds (e.g., MRI, CT, etc.), it can achieve accurate alignment between the preoperative imaging coordinate system and the intraoperative coordinate system, thereby accurately calculating the surgical path. In addition, the surgical robot provided by the present disclosure can be integrated with a cranial drill tool or a needle-like tool (e.g., a drug syringe, an electrode implant needle, etc.), and can achieve an integrated, automated surgical process from positioning, opening a hole, to entry by controlling the robotic arm, which can improve the accuracy and efficiency of the surgery and reduce damage to the patient or experimental animal.

[0117] Figure 31 is a block diagram showing a surgical tool positioning and moving device 300 applied to a surgical robot according to an exemplary embodiment of the present disclosure. The surgical robot may include a robotic arm component (1), a sensor (2) mounted above the end of the robotic arm component (1), and a surgical tool (3) mounted at the end of the robotic arm component (1). The robotic arm component (1) may include a plurality of robotic arm links connected end to end. Exemplarily, the robotic arm component (1) may include four robotic arm links connected end to end. It should be noted that the surgical tool positioning and moving method applied to a surgical robot provided by the present disclosure may be applicable to, but not limited to, the following scenarios: intracranial drug delivery / implantation system for large animals.

[0118] Reference Figure 3 The surgical tool positioning and moving device 300 applied to a surgical robot may include a travel path direction calculation module 301 , a target position acquisition module 302 , a transformation matrix calculation module 303 and a movement module 304 .

[0119] The path direction calculation module 301 can be based on the position and posture of the sensor (2) in the base coordinate system at each of the multiple perspectives acquired in advance. , the first transformation matrix between the coordinate system of sensor (2) and the coordinate system of the end of the robot arm , point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the direction of the travel path of the tip of the surgical tool (3) The above-mentioned multiple viewing angles can be multiple viewing angles obtained by the sensor (2) when the sensor (2) is pulled by multiple robotic arm links to move around the surgical object according to a predetermined trajectory. " can be a point cloud of the skin surface of the surgical object obtained based on CT or MRI images through threshold segmentation, point cloud extraction, etc.

[0120] The above-mentioned “base coordinate system” may be a coordinate system at a target joint farthest from the end of the robotic arm component (1) among a plurality of joints connecting a plurality of robotic arm links; the above-mentioned “robotic arm end coordinate system” may be a coordinate system at the end end of the robotic arm component (1).

[0121] It should be noted that the 3D sensor can be pulled by the robotic arm and moved around the head of the surgical object according to a predetermined trajectory to perform multi-view, high-resolution point cloud data acquisition. At this time, the surgical tool (3) is not installed at the end of the robotic arm to avoid obstruction and interference. Thus, in the present disclosure, by adopting the "eye on hand" solution, that is, by installing the high-resolution 3D sensor above the end of the robotic arm, it is possible to achieve close-range, multi-angle acquisition of high-precision point clouds of the surgical object's head, thereby achieving non-invasive, high-precision surface matching positioning.

[0122] According to an exemplary embodiment of the present disclosure, the path direction calculation module 301 can calculate the direction of the i-th perspective of the above-mentioned multiple perspectives based on the posture , the first transformation matrix and point cloud , calculate the point cloud in the base coordinate system under the i-th perspective , where i=1, 2, …, N, N can be the number of multiple perspectives, and N can be an integer greater than 1.

[0123] Then, the path direction calculation module 301 can calculate the multiple point clouds corresponding to the multiple perspectives. Perform stitching to obtain a complete point cloud in the base coordinate system .

[0124] Next, the path direction calculation module 301 can calculate the direction of the path based on the complete point cloud. Preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , where the image coordinate system can be used to obtain the preoperative medical image point cloud That is, in the present disclosure, the three-dimensional head model obtained before surgery can be Point cloud collected during surgery Perform registration to solve the transformation relationship from the image coordinate system to the base coordinate system , where point cloud registration can adopt feature-based global registration algorithm or ICP algorithm.

[0125] Then, the path direction calculation module 301 can calculate the direction of the path based on the above transformation matrix , Preoperative medical imaging point cloud The location of the included opening points and the location of the target , calculate the direction of the travel path .

[0126] According to an exemplary embodiment of the present disclosure, the travel path direction calculation module 301 can calculate the point cloud by the following formula: :

[0127] .

[0128] According to an exemplary embodiment of the present disclosure, the path direction calculation module 301 can also calculate the direction of the preoperative medical image point cloud by using a point cloud registration algorithm. The transformation relationship matrix , so that the transformation matrix Transformed preoperative medical image point cloud With the complete point cloud The degree of overlap between them is greater than or equal to the preset threshold. That is, the path direction calculation module 301 can find a point cloud registration algorithm that acts on The transformation relationship matrix , so that the transformed The overlap is best.

[0129] Thus, in the present disclosure, through hand-eye calibration, tool calibration and multi-view point cloud stitching technology, the conversion relationship between each coordinate system can be accurately established, thereby ensuring high-precision registration between the preoperative image and the intraoperative coordinate system, thereby realizing the travel path direction of the tip of the surgical tool. Accurate planning.

[0130] According to an exemplary embodiment of the present disclosure, the path direction calculation module 301 can transform the key points in the path direction of the tip of the surgical tool (3) planned before surgery into the base coordinate system. Specifically, the path direction calculation module 301 can calculate the transformation relationship matrix and the location of the opening point The product of , obtains the position of the opening point in the base coordinate system : ; The path direction calculation module 301 can also calculate the transformation relationship matrix and the location of the target The product of , obtains the position of the target in the base coordinate system : Then, the travel path direction calculation module 301 can calculate the direction of the vehicle based on the position. and location , calculate the direction of the travel path For example, the direction of the travel path can be calculated by the following formula :

[0131]

[0132] In this way, the surgical tool (3), i.e., the drill bit of the cranial drilling tool, can be aligned with the starting point of the travel path direction. , and make the travel direction follow the planned path direction The movement is performed so that the tip of the surgical tool (3) gradually moves to the intracranial target of the surgical subject.

[0133] The target position acquisition module 302 can obtain the second transformation matrix from the robot end coordinate system to the base coordinate system in the current state and the target position that the tip of the surgical tool (3) is expected to reach , where the target position The target position may be one of the multiple positions that the tip of the surgical tool (3) is expected to reach during the process of reaching the target point from the opening point of the surgical object, that is, the "target position" " can be a position among a plurality of positions that the needle tip of the surgical tool (3) is expected to reach in the direction of the path from the opening point of the surgical object to the target point. "The second transformation matrix ” can be directly read from the robotic arm included in the surgical robot in its current state.

[0134] The transformation matrix calculation module 303 can be based on the second transformation matrix , target location , travel path direction And the third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate the movement of the tip of the surgical tool (3) to the target position The fourth transformation matrix between the manipulator end coordinate system and the base coordinate system is , wherein the tool coordinate system may be a coordinate system at the tip of the surgical tool (3). Exemplarily, the tool coordinate system may be a coordinate system with the tip of the surgical tool (3) as the origin and the axial direction of the surgical tool (3) as the Z axis.

[0135] It should be noted that, in the present disclosure, the surgical tool (3) connected to the end of the robotic arm component (1) included in the surgical robot can be calibrated in advance to determine the fixed transformation matrix between the tool coordinate system and the robotic arm end coordinate system. .

[0136] According to an exemplary embodiment of the present disclosure, the transformation matrix calculation module 303 can calculate the fourth transformation matrix by the following formula: , that is, the terminal transformation after moving can be expressed as:

[0137]

[0138] in, It can be the transformation of the tool coordinate system from the initial position to the target position, which may only contain the unknown translation part and the unknown rotation part For example, let's assume 4 4-size matrix, then Can be included 3 of the upper left corner of the element Matrix of size 3, Can be included 3 of the upper right corner of the element A vector of size 1. Specifically:

[0139] 1. Calculate the unknown translation part :

[0140] The origin of the tool coordinate system can be . Then according to the transformation relationship: , to get the translation vector:

[0141]

[0142] in, The third transformation matrix can be The inverse matrix of .

[0143] 2. Calculate the unknown rotation part :

[0144] The standard axis in the tool coordinate system can be , then the target direction in the tool initial coordinate system can be expressed as:

[0145]

[0146] Rotation Matrix Can satisfy:

[0147]

[0148] Among them, as mentioned above, The third transformation matrix can be The inverse matrix of Can be included For example, assume that 4 4-size matrix, then Can be included 3 of the upper left corner of the element Matrix of size 3. Can be included For example, assume that 4 4-size matrix, then Can be included 3 of the upper left corner of the element Matrix of size 3.

[0149] The movement module 304 can be based on the fourth transformation matrix , controls the tip of the surgical tool (3) to move from the current position to the target position That is, the movement module 304 can be based on the fourth transformation matrix Align the axial direction (Z-axis direction of the tool coordinate system) of the surgical tool (3) with the target direction and gradually move the tip of the surgical tool (3) to the target position .

[0150] It should be noted that the surgical tool positioning and movement method provided by this disclosure for a surgical robot can be applied not only in brain surgery scenarios, but also in other surgical scenarios, including but not limited to spinal surgery scenarios, orthopedic surgery scenarios, etc. Moreover, in the corresponding surgical scenario, only the surgical tools need to be adjusted.

[0151] Figure 4 is a block diagram illustrating an electronic device 400 according to an exemplary embodiment of the present disclosure.

[0152] Reference Figure 4 The electronic device 400 includes at least one memory 401 and at least one processor 402, wherein the at least one memory 401 stores instructions. When the instructions are executed by the at least one processor 402, a surgical tool positioning and moving method applied to a surgical robot according to an exemplary embodiment of the present disclosure is executed.

[0153] As an example, electronic device 400 may be a PC, tablet device, personal digital assistant, smartphone, or other device capable of executing the aforementioned instructions. Here, electronic device 400 is not necessarily a single electronic device, but may also be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 400 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with local or remote devices (e.g., via wireless transmission).

[0154] In electronic device 400, processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0155] The processor 402 can execute instructions or codes stored in the memory 401, wherein the memory 401 can also store data. Instructions and data can also be sent and received over the network via the network interface device, wherein the network interface device can use any known transmission protocol.

[0156] Memory 401 may be integrated with processor 402, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, memory 401 may comprise a separate device, such as an external disk drive, a storage array, or any other storage device usable by a database system. Memory 401 and processor 402 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, or the like, such that processor 402 can access files stored in memory.

[0157] In addition, the electronic device 400 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 400 may be connected to each other via a bus and / or a network.

[0158] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned surgical tool positioning and movement method applied to a surgical robot. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as a multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device configured to store a computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0159] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program, which, when executed by a processor, implements the surgical tool positioning and moving method applied to a surgical robot according to the present disclosure.

[0160] The disclosed method, apparatus, electronic device, storage medium, and computer program product for positioning and moving surgical tools for a surgical robot employ an "eye-in-hand" approach. Specifically, the disclosed method can mount a sensor above the distal end of a robotic arm component included in the surgical robot. This allows the robotic arm's connecting rod to pull the sensor around the surgical object along a predetermined trajectory, enabling close-range, multi-perspective acquisition of surgical object information. This enables close-range, multi-perspective acquisition of high-precision point clouds of the surgical object, improving the comprehensiveness and accuracy of the acquired surgical object information. This in turn increases the resolution of the point cloud and expands its coverage, thereby enhancing surgical precision and efficiency.

[0161] According to exemplary embodiments of the present disclosure, a cranial drill or needle tool can be integrated into a surgical robot, and a quick-change tray can be used to automatically integrate and rapidly switch between various surgical tools. Furthermore, precise robotic arm control can be used to achieve integrated, automated operations for positioning, opening holes, and access, achieving integrated integration of the entire surgical process, thereby simplifying the surgical process and improving efficiency and precision.

[0162] According to an exemplary embodiment of the present disclosure, by adopting the "eyes on hands" solution, that is, by installing a high-resolution 3D sensor above the end of the robotic arm, high-precision point cloud of the surgical object's head can be collected at close range and from multiple angles, thereby achieving non-invasive, high-precision surface matching positioning.

[0163] According to an exemplary embodiment of the present disclosure, through hand-eye calibration, tool calibration and multi-view point cloud stitching technology, the conversion relationship between each coordinate system can be accurately established, thereby ensuring high-precision alignment between the preoperative image and the intraoperative coordinate system, thereby achieving accurate planning of the travel path direction of the tip of the surgical tool.

[0164] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0165] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A surgical tool positioning and moving device for a surgical robot, the surgical robot comprising a robotic arm component, a sensor mounted above the distal end of the robotic arm component, and a surgical tool mounted at the distal end of the robotic arm component, the robotic arm component comprising a plurality of robotic arm links connected end to end, characterized in that: The surgical tool positioning and moving device comprises: The path direction calculation module is configured to calculate the position of the sensor in the base coordinate system at each of the multiple perspectives acquired in advance. , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple viewing angles are multiple viewing angles obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; The target position acquisition module is configured to obtain the second transformation matrix from the end coordinate system of the manipulator to the base coordinate system in the current state and the target position that the tip of the surgical tool is expected to reach , wherein the target location It is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of moving from the opening point of the surgical object to the target point; A transformation matrix calculation module is configured to calculate the transformation matrix based on the second transformation matrix. , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is a coordinate system at the tip of the surgical tool; A moving module is configured to move the , controlling the tip of the surgical tool to move from the current position to the target position ; The transformation matrix calculation module is specifically configured as follows: The fourth transformation matrix is ​​calculated by the following formula : in, Contains translation part and rotating part ; in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

2. The surgical tool positioning and moving device according to claim 1, wherein: The path direction calculation module is configured to: for the i-th perspective among the multiple perspectives, based on the posture , the first transformation matrix and the point cloud , calculate the point cloud in the base coordinate system at the i-th perspective , where i=1, 2, ..., N, where N is the number of the multiple perspectives, and N is an integer greater than 1; a plurality of point clouds corresponding to the multiple perspectives Perform stitching to obtain a complete point cloud in the base coordinate system ; Based on the complete point cloud The preoperative medical imaging point cloud , calculate the transformation relationship matrix between the image coordinate system and the base coordinate system , wherein the image coordinate system is used to obtain the preoperative medical image point cloud The coordinate system corresponding to the device; based on the transformation relationship matrix , the preoperative medical imaging point cloud The location of the opening point included and the location of the target , calculate the direction of the travel path .

3. The surgical tool positioning and moving device according to claim 2, wherein: The path direction calculation module is configured to calculate the point cloud by the following formula: : 。 4. The surgical tool positioning and moving device according to claim 2, wherein: The path direction calculation module is configured to: calculate the direction of the preoperative medical image point cloud through the point cloud registration algorithm The transformation relationship matrix , so that the transformation relationship matrix Transformed preoperative medical image point cloud With the complete point cloud The degree of overlap between them is greater than or equal to a preset threshold.

5. The surgical tool positioning and moving device according to claim 2, wherein: The travel path direction calculation module is configured to: calculate the transformation relationship matrix and the location of the opening point The product of , obtains the position of the opening point in the base coordinate system. ; Calculate the transformation relationship matrix and the location of the target The product of the target point and the base coordinate system is obtained. Based on the location and the location , calculate the direction of the travel path .

6. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the following surgical tool positioning and movement method applied to a surgical robot: The surgical robot includes a robotic arm component, a sensor mounted above the distal end of the robotic arm component, and a surgical tool mounted at the distal end of the robotic arm component. The robotic arm component includes a plurality of robotic arm links connected end to end. Based on the position and posture of the sensor in the base coordinate system at each of the multiple perspectives acquired in advance , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple viewing angles are multiple viewing angles obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; Get the second transformation matrix from the robot end coordinate system to the base coordinate system in the current state and the target position that the tip of the surgical tool is expected to reach , wherein the target location It is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of moving from the opening point of the surgical object to the target point; Based on the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is a coordinate system at the tip of the surgical tool; Based on the fourth transformation matrix , controlling the tip of the surgical tool to move from the current position to the target position ; Wherein, the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is ,include: The fourth transformation matrix is ​​calculated by the following formula : in, Contains translation part and rotating part ; in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

7. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the following surgical tool positioning and moving method applied to a surgical robot: The surgical robot includes a robotic arm component, a sensor mounted above the distal end of the robotic arm component, and a surgical tool mounted at the distal end of the robotic arm component. The robotic arm component includes a plurality of robotic arm links connected end to end. Based on the position and posture of the sensor in the base coordinate system at each of the multiple perspectives acquired in advance , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple viewing angles are multiple viewing angles obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; Get the second transformation matrix from the robot end coordinate system to the base coordinate system in the current state and the target position that the tip of the surgical tool is expected to reach , wherein the target location It is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of moving from the opening point of the surgical object to the target point; Based on the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is a coordinate system at the tip of the surgical tool; Based on the fourth transformation matrix , controlling the tip of the surgical tool to move from the current position to the target position ; Wherein, the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is ,include: The fourth transformation matrix is ​​calculated by the following formula : in, Contains translation part and rotating part ; in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the following surgical tool positioning and moving method applied to a surgical robot is implemented: The surgical robot includes a robotic arm component, a sensor mounted above the distal end of the robotic arm component, and a surgical tool mounted at the distal end of the robotic arm component. The robotic arm component includes a plurality of robotic arm links connected end to end. Based on the position and posture of the sensor in the base coordinate system at each of the multiple perspectives acquired in advance , the first transformation matrix between the coordinate system of the sensor and the coordinate system of the end of the robot arm , the point cloud collected at each perspective And preoperative medical imaging point cloud , calculate the travel path direction of the tip of the surgical tool , wherein the multiple viewing angles are multiple viewing angles obtained by the sensor when the sensor is pulled by the multiple robotic arm links to move around the surgical object according to a predetermined trajectory, the base coordinate system is the coordinate system of the target joint farthest from the end of the robotic arm component among the multiple joints connecting the multiple robotic arm links, and the robotic arm end coordinate system is the coordinate system at the end of the robotic arm component; Get the second transformation matrix from the robot end coordinate system to the base coordinate system in the current state and the target position that the tip of the surgical tool is expected to reach , wherein the target location It is one of the multiple positions that the tip of the surgical tool is expected to reach in the process of moving from the opening point of the surgical object to the target point; Based on the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is , wherein the tool coordinate system is a coordinate system at the tip of the surgical tool; Based on the fourth transformation matrix , controlling the tip of the surgical tool to move from the current position to the target position ; Wherein, the second transformation matrix , the target location , the direction of the travel path and a third transformation matrix between the pre-acquired tool coordinate system and the robot end coordinate system , calculate and move the tip of the surgical tool to the target position The fourth transformation matrix between the robot end coordinate system and the base coordinate system is ,include: The fourth transformation matrix is ​​calculated by the following formula : in, Contains translation part and rotating part ; in, is the third transformation matrix The inverse matrix of To include The matrix of the upper left corner of the element, To include The matrix of the upper left corner element.

Images