Tumor ablation system

Abstract

According to the tumor ablation system provided by the invention, the image processing subsystem is adopted to perform intelligent segmentation on the case image, and an ablation target segmentation result with a clear boundary can be obtained, so that in the subsequent puncture ablation process, an ablation target can be accurately positioned, damage to healthy tissues is avoided, and in addition, the tumor ablation efficiency is improved. The optical navigation subsystem tracks a first mark on an ablation object and a second mark on a mechanical arm of the execution robot, so that accurate coordinate conversion among the execution robot, the image processing subsystem and the optical navigation subsystem is realized, and the accuracy of ablation target positioning in puncture ablation can be realized; the method is especially suitable for a small target area.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and particularly to a tumor ablation system. Background Art

[0002] Liver cancer is the sixth most common cancer in the world, and hepatocellular carcinoma (HCC) is one of the most common subtypes of cancer. Surgical treatment, radiofrequency ablation, chemotherapy, and liver transplantation are traditional treatment methods for cancer patients.

[0003] Percutaneous radiofrequency ablation (RFA) has the advantages of minimally invasive and good survival rate, and has become the preferred and effective alternative treatment strategy for cancer. Specifically, a needle electrode is percutaneously punctured into the tumor, and radiofrequency current is released. The tumor is thermally destroyed and completely burned into coagulative necrosis. The tumor tissue within the ablation coverage area can achieve complete necrosis, effectively control the development of the tumor, and improve the survival rate of patients. RFA is an emerging treatment method for minimally invasive treatment of liver malignancies, especially for patients who cannot undergo surgery.

[0004] However, there are still some bottleneck problems in the treatment of liver tumors by percutaneous radiofrequency ablation. First, the tumor boundaries defined based on CT, ultrasound, and other imaging methods are not clear, resulting in inaccurate puncture and damage to healthy tissues. Second, the accuracy of manual puncture depends to a large extent on the doctor's spatial awareness and subjective experience. Radiofrequency ablation of small nodules is technically challenging due to the difficulty of reaching small target areas. Third, for large cancer nodules that require multi-point ablation and overlapping inactivation, complete coverage cannot be achieved, resulting in tumor residue and recurrence. Summary of the Invention

[0005] The present application proposes a tumor ablation system that can solve one of the problems existing in the background art.

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] There is provided a tumor ablation system, which acts on an ablation object marked with a first mark, and the first mark is relatively stationary with respect to the ablation object. The system includes:

[0008] An execution robot configured with a first coordinate system, a robotic arm provided with a second mark, an ablation needle assembled on the robotic arm, the relative position of the ablation needle with respect to the second mark is determined, and the second mark has a first mark coordinate in the first working coordinate system;

[0009] An image processing subsystem configured with a second coordinate system, which is used to obtain a pathological image, the ablation object and the first marker are imaged in the pathological image, and the pathological image is intelligently segmented to obtain the ablation target positioning information of the ablation object in the second coordinate system, calculate the ablation strategy determined according to the ablation target positioning information, and the first marker has a second marker coordinate in the second coordinate system;

[0010] An optical navigation subsystem configured with a third coordinate system, which is used to track the first marker and the second marker, the first marker has a third marker coordinate in the third coordinate system, and the second marker has a fourth marker coordinate in the third coordinate system; and

[0011] A robot control subsystem, which is used to realize the coordinate system conversion of the ablation target positioning information based on the first marker coordinate, the second marker coordinate, the third marker coordinate and the fourth marker coordinate, and control the execution of the robot action based on the coordinate system conversion result.

[0012] Based on the above technical solution, the intelligent segmentation of the case image by the image processing subsystem can obtain a clear boundary ablation target segmentation result, so that in the subsequent puncture ablation process, the ablation target can be accurately positioned, avoiding damage to healthy tissues. Moreover, the optical navigation subsystem tracks the first marker on the ablation object and the second marker on the robotic arm of the execution robot, so as to realize the accurate coordinate conversion among the execution robot, the image processing subsystem and the optical navigation subsystem, and can realize the accuracy of the ablation target positioning in the puncture ablation, which is particularly applicable to small target areas.

[0013] In a possible design, the first coordinate system includes: a robot base coordinate system, an end effector coordinate system and a tool coordinate system, and there are conversion relationships between the robot base coordinate system and the end effector coordinate system, between the robot base coordinate system and the tool coordinate system, between the robot base coordinate system and the third coordinate system, between the end effector coordinate system and the tool coordinate system, and between the tool coordinate system and the third coordinate system.

[0014] In a possible design, the robot control subsystem is specifically used for:

[0015] Using the second marker coordinate and the third marker coordinate to realize the conversion of the ablation target positioning information from the second coordinate system to the third coordinate system; and

[0016] Using the fourth marker coordinate and the first marker coordinate to realize the conversion of the ablation target positioning information from the third coordinate system to the first coordinate system.

[0017] In a possible design approach, coordinate system transformation is achieved based on the singular value decomposition method.

[0018] In a possible design approach, the image processing subsystem adopts a multi-task UNet network architecture. The multi-task UNet network architecture employs at least two segmentation algorithms for different categories of organs. The first segmentation algorithm includes: thresholding, region growing, and partial differential equations. The second segmentation algorithm is a deep segmentation network based on active contours.

[0019] In a possible design approach, the multi-task UNet network architecture specifically includes: a first 3×3 convolutional layer, a second 3×3 convolutional layer, a first 2×2 max pooling layer, a third 3×3 convolutional layer, a fourth 3×3 convolutional layer, a second 2×2 max pooling layer, a fifth 3×3 convolutional layer, a sixth 3×3 convolutional layer, a third 2×2 max pooling layer, a seventh 3×3 convolutional layer, an eighth 3×3 convolutional layer, a fourth 2×2 max pooling layer, a ninth 3×3 convolutional layer, a tenth 3×3 convolutional layer, a first 2×2 upsampling layer, an eleventh 3×3 convolutional layer, a twelfth 3×3 convolutional layer, a second 2×2 upsampling layer, a thirteenth 3×3 convolutional layer, a fourteenth 3×3 convolutional layer, a third 2×2 upsampling layer, a fifteenth 3×3 convolutional layer, a sixteenth 3×3 convolutional layer, a fourth 2×2 upsampling layer, a seventeenth 3×3 convolutional layer, an eighteenth 3×3 convolutional layer, and an output layer, deployed in sequence from input to output. The output of the second 3×3 convolutional layer is skip-connected to the input of the fourth 2×2 upsampling layer. The output of the fourth 3×3 convolutional layer is skip-connected to the input of the third 2×2 upsampling layer. The output of the sixth 3×3 convolutional layer is skip-connected to the input of the second 2×2 upsampling layer. The output of the eighth 3×3 convolutional layer is skip-connected to the input of the first 2×2 upsampling layer.

[0020] In a possible design approach, the image processing subsystem is specifically configured to:

[0021] Adopt a branch-and-cut algorithm and a greedy algorithm. Under the condition that the ablation range is limited in a single time, with the goal of minimizing the number of punctures, obtain the ablation strategy including the ablation path.

[0022] Based on the above technical solution, by adopting a multi-task UNet network architecture, it is possible to completely cover the ablation target that requires multi-point ablation and overlapping inactivation with the least number of punctures.

[0023] In a possible design approach, the parameters to be optimized in the multi-task UNet network architecture include: k i , x i and r i , where k iThe value is 0 or 1, where 1 indicates that the i-th ablation sphere actually participates in tumor ablation, and 0 indicates that the corresponding puncture ablation process is not performed; x i is the center of the sphere of the ablation sphere, and r i is the radius of the sphere of the ablation sphere.

[0024] In a possible design, the first marker and the second marker are four reflection spheres. Brief Description of the Drawings

[0025] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings in the following description are only some embodiments of the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] Figure 1 is a schematic diagram of a high-precision liver tumor ablation system based on optical navigation provided by an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of a puncture body with 4 reflection spheres provided by an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of three coordinate system conversions provided by an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of the positions of 4 reflection spheres in the WCS coordinate system provided by an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of the target attitude of the TCP tracking target selection provided by an embodiment of the present application;

[0031] Figure 6 is a schematic diagram of the multi-task UNet network architecture provided by an embodiment of the present application;

[0032] Figure 7 is a schematic diagram of the experimental results of image segmentation provided by an embodiment of the present application. Detailed Embodiments

[0033] In order to make the purpose, technical solutions and advantages of the present application more clear, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

[0034] It should be noted that although the functional modules are divided in the schematic diagram of the device and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the description, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0036] Based on optical positioning and navigation technologies, this application has developed a method for establishing a reference coordinate system based on fiducial points and applied this method to the research on the registration method between different spatial coordinate systems in a navigation system, thereby constructing a software and hardware platform for a liver puncture navigation robot system.

[0037] Through hand-eye calibration, this project obtained the transformation relationship between the ablation robot and the measurement device, namely the hand-eye matrix. This matrix describes the position and pose of the robot hand in the camera coordinate system, enabling the robot to accurately perform tasks under the guidance of camera vision. The final accuracy test result after calibration shows an error of 1.733 mm. Combining with the specific position of the tumor in CT image segmentation technology, under hand-eye optical positioning and navigation technologies, the robot arm can accurately puncture to the specified position and apply energy to induce coagulative necrosis.

[0038] In tumor ablation applications, the maximum diameter of a single ablation spherical region is usually less than 4 cm. When the maximum diameter of the tumor size is less than 4 cm, a single ablation spherical region can achieve complete coverage of the tumor. However, for tumors with a larger diameter, a single ablation cannot cover the tumor, so multiple punctures and ablations are required to cover the tumor area. The design of the tumor ablation plan can be abstracted as the problem of optimally covering an irregular object with multiple spheres. The principle of ablating a large tumor is to cover the entire area of the tumor with the fewest number of punctures while ensuring that normal tissues are not damaged. To solve this problem, the present application adopts a branch and cut algorithm, which combines a branch and bound algorithm and a cutting plane algorithm. Through simulation experiments, we also found that by selecting different ablation radii, such as 10 mm and 15 mm, for tumor ablation, a satisfactory tumor coverage rate can be achieved, but more punctures are required. For large tumors, our principle is to cover the entire area with the fewest number of punctures while ensuring that normal tissues are not damaged. The desired output is the minimum number of overlapping ablation spheres required to ablate each tumor and its margin. To address this problem, the present application also studied the ablation plan for large tumors when the unit ablation range is not restricted. The inventors discovered a heuristic algorithm for finding a locally optimal feasible solution, and the algorithm design was inspired by the idea of the greedy algorithm. The principle of this algorithm is to first cover the largest possible area and then sequentially complete the coverage of the remaining areas. Finally, 4 large tumor samples were verified, and the results of the ablation plan design also achieved complete coverage of the large tumors. The ablation frequency was 2 - 4 times, which was significantly reduced compared with the case of limited single ablation volume. It not only reduced the number of punctures, achieved a satisfactory tumor coverage rate, but also protected normal tissues from damage as much as possible.

[0039] The branch and cut method is a combinatorial optimization method for solving integer linear problems (ILPs), that is, linear programming (LP) problems where some or all unknowns are integer-valued. Based on the branch and bound method, this method uses cutting planes to tighten the linear programming relaxation. If the cutting planes are only used to tighten the initial LP relaxation, it is called the cutting and branching method.

[0040] The ability to achieve overall tumor coverage and eliminate viable tumor cells during ablation therapy is the key to improving patient survival rate and reducing the recurrence rate. By combining different functional components, the above challenges are solved, and CT image segmentation technology can effectively, stably, and accurately distinguish liver tumors from normal tissues. Then, the branch and cut algorithm and the greedy algorithm are optimized to obtain the best puncture and ablation plan. Finally, under the hand-eye optical positioning and navigation technology, the robotic arm can accurately puncture to the designated position and apply energy to induce coagulative necrosis.

[0041] The present application uses a hand-eye optical positioning and navigation system and a UR robotic arm to achieve CT-guided thermal ablation surgery for liver tumors.

[0042] In the phantom experiment, after positioning, the robotic arm will automatically move its position so that the ablation needle at the end of the robotic arm reaches the planned position.

[0043] In the simulated tumor puncture experiment, it was also verified again that by combining CT with optical positioning information, the ablation needle at the end of the robotic arm can accurately avoid key structures (major blood vessels, bones, and other organs) and puncture into the tumor position.

[0044] The intelligent tumor precise ablation system in this application is expected to fundamentally solve the above problems. It can completely replace cancer surgery, with less trauma and lower cost. For patients with overly large advanced tumors or those with severe cardiovascular and cerebrovascular diseases who are contraindicated for surgery, it is expected to significantly improve their survival rate and quality of life. This technology effectively reduces the heavy burden on social medical insurance and family finances. This technology can also be further extended to the treatment of tumors in other organs and other fields. It has great economic and social value.

[0045] The tumor ablation system of this application will be exemplarily described below through a specific application example.

[0046] The specific solution of this invention: The hardware system consists of a UR5e robot (device code: 20225501534), an optical navigation system FusionTrack 500 (FTK 500), and an image processing system, as Figure 1 shown. The robot includes a host computer, a control cabinet, a robot body, a tool holder, a TCP (tool center point) tracking target, and a puncture ablation needle. The puncture ablation needle, as Figure 2 shown, can be directly connected to a set of ablation systems. This hardware system uses the real-time passive / active optical tracking system FTK 500 developed by the Swiss company Atrasys, which can detect and track reflection spheres, reflection planes, and infrared light in real time in the video. In this system, the tracking targets used are all based on four reflection spheres as the positioning reference, and the spatial postures of the puncture body (ablation object, human or animal body) and the puncture needle (ablation needle) are described in the WCS working coordinate system composed of the centers of the four spheres.

[0047] 1. Hand-eye system

[0048] The host computer of the robot uses the optical measurement device FTK 500 (FusionTrack 500) to obtain the image information of the puncture body and the puncture needle in the Cartesian space, as Figure 3 shown.

[0049] Taking the WCS tracking target as an example, the coordinates of the centers of the four spheres in the WCS coordinate system are W P1 = [46.98, 0, 0]T , W P2 = [0, 44.32, 0] T , W P3 = [-41.02, 0, 0] T , W P4 = [0, -28.59, 0] T , as Figure 4 shown.

[0050] By using a vision measurement device, the coordinates of the four center points in the camera coordinate system are respectively C P1, C P2, C P3 and C P4. It is necessary to find a set of Euclidean transformations R, t, where R is a rotation matrix and t is a translation vector, such that:

[0051]

[0052] where i = 1, 2, 3, 4....

[0053] Define the centroids of two sets of points:

[0054]

[0055] Note that in the cross - project section, C P i - C P1 - R( W P i - W P1) becomes 0 after summation. Therefore, the optimization objective function can be simplified to:

[0056]

[0057] where J is the Jacobian matrix, which is used to describe the derivative of a vector - valued function with respect to the input variables.

[0058] The actual optimization objective function becomes:

[0059]

[0060] where T is the transformation matrix, C q i is a point in the camera coordinate system, W P i is a point in the WCS coordinate system, and tr(trace) is the trace of the matrix, which refers to the sum of all elements on the main diagonal of the matrix.

[0061] Define the matrix W:

[0062]

[0063] W is a 3x3 matrix, and its singular value decomposition (SVD) gives:

[0064] W = U∑V T (6)

[0065] where ∑ is a diagonal matrix composed of singular values, and the diagonal elements are arranged in ascending order, while U and V are orthogonal matrices. When W is full rank, R is:

[0066] R = UV T (7)

[0067] Then, t can be obtained as:

[0068] t = C P - R· W P (8)

[0069] The homogeneous transformation matrix composed of R and t C T w is the pose description of the WCS tracking target (the marker on the puncture body, i.e., the reflective sphere) in the camera coordinate system, while the TCP (tool center point) tracking target (the marker on the puncture needle, i.e., the reflective sphere) in the camera coordinate system is C T t , and the calculation method is the same, where:

[0070]

[0071] Define the TCP coordinate system as O t and the camera coordinate system as O c and the robot base coordinate system as O b and the end effector coordinate system as O e Each transformation matrix has the following relationship:

[0072]

[0073] where b T t is the transformation matrix from the tool TCP coordinate system O t to the robot base coordinate system O b . b T C is the transformation matrix from the camera coordinate system O c to the robot base coordinate system O b , b T e is the transformation matrix from the end effector coordinate system O e to the robot base coordinate system O b , e T tFor the transformation matrix from the tool coordinate system O t to the end - effector coordinate system O e , c T t is the transformation matrix from the camera coordinate system O c to the tool TCP coordinate system. e is the end - effector, that is, the robot end device.

[0074] Robots moving to different poses in space can obtain multiple sets of C T i t and b T i e values, and then multiple sets of matrix transformation relationships can be established:

[0075]

[0076] respectively represent the i - th and j - th movements, where i≠j. Define:

[0077]

[0078] Equation (11) can be transformed into:

[0079] AX = XB (13)

[0080] Among them, A and B are known quantities, and X is the quantity to be solved. They are all homogeneous transformation matrices, composed of a 3×3 rotation matrix and a 3×1 translation matrix:

[0081]

[0082] Among them, R bij is the rotation matrix representing from the robot base coordinate system i to the robot base coordinate system j, and t bij is the translation vector from the robot base coordinate system i to the robot base coordinate system j; R cij is the rotation matrix from the camera coordinate system i to the camera coordinate system j, and t cij is the translation vector from the camera coordinate system i to the camera coordinate system j; R be is the rotation matrix from the robot base coordinate system b to the end - effector coordinate system e, and t be is the translation vector representing from the robot base coordinate system b to the end - effector coordinate system e.

[0083] Equation (13) can be expanded as:

[0084] R bij R be = R be R cij (15)

[0085] R bij t be +t bij =R be t cij +t be (16)

[0086] Equation (15) gives R bij =R be ·R cij ·R be -1 ,R bij and R cij corresponding to equal rotation angles, denoted as θ. Equation (15) can be written as:

[0087]

[0088] where k b and k c are the rotation axes of R bij and R cij respectively, and Rot(k c ,θ) represents the rotation matrix for rotating by an angle θ about the axis k c . Equation (16) can be written as:

[0089] (R bij -I)t be =R be t cij -t bij (18)

[0090] where I is the identity matrix.

[0091] The properties of the above rotation matrix include:

[0092]

[0093] If the robot moves to three different positions during calibration, the following four relationships can be obtained:

[0094]

[0095] Taking the WCS tracking target as the reference coordinate system, select the target postures w T1, w T2 and w T3 of the three TCP tracking targets, as Figure 5 shown.

[0096] The puncture needle needs to be calibrated for the target posture. The robot needs to adjust the posture and position of the puncture needle so that the TCP aligns with the target point or follows the movement of the target point.

[0097] Determine the pose of the WCS tracking target through the measuring device as c T w . And b T c · c T w · w T1, b T c · c T w · w T2 and b T c · c T w · w T3 are converted into motion commands in the robot URScript format, and the robot is controlled to run towards three target points. The absolute accuracy of calibrating the robot can be obtained:

[0098]

[0099] Wherein, c Ti* is the calibrated transformation matrix from the camera coordinate system to the i-th target coordinate system.

[0100] By fixing the puncture body and the WCS tracking target together, the complete shape of the puncture body and the relative position of the WCS tracking target with respect to the puncture body can be obtained through CT scanning. In addition, the coordinates of the target points on the puncture body can be obtained in the WCS coordinate system w T. The target point is the specific position on the puncture body where the puncture operation needs to be performed. Through CT scanning, the coordinates of the target point in the WCS tracking target coordinate system can be obtained w T. Through coordinate transformation, w T is mapped to the robot base coordinate system to generate motion instructions. b T c · c T w · w T can be converted into motion instructions in the URScript format to control the robot to move towards the target points on the puncture body, thereby controlling the robot to complete the expected puncture action.

[0101] 2. Abdominal CT dataset analysis and CT image segmentation method

[0102] The data used in this part comes from many abdominal CT datasets published on the Internet. As shown in Table 1, there are a total of 212 CT samples.

[0103] Table 1: Source of experimental data

[0104]

[0105] All samples have corresponding manually marked liver labels, which can be used for neural network training and performance testing. CT data processing is carried out on a dedicated artificial intelligence (AI) server, and the core software and hardware configurations are shown in Table 2.

[0106] Table 2: Computing Environment Configuration

[0107]

[0108] A total of 212 abdominal CT samples published on the Internet were used for neural network training. Abdominal CT images were used as the data source for modeling to identify and segment the liver and its adjacent organs, blood vessels, bile ducts, tumor tissues, skin, fat, muscle, bone, etc., and to establish a 3D digital model. Among them, classical image processing algorithms such as thresholding, region growing, and PDE can achieve satisfactory segmentation results for regions with significant differences in CT values between the skin, fat, muscle, bone, and adjacent regions. The inventor integrated the advantages of variational segmentation algorithms and the latest deep network design methods in classical medical images, effectively reducing the amount of data required for CT liver segmentation. The image processing system includes a multi-task UNet network architecture, using the VENet, UAMT segmentation method, and SASSNet segmentation method respectively, as Figure 6 shown.

[0109] The multi-task UNet network architecture specifically includes: a first 3×3 convolutional layer, a second 3×3 convolutional layer, a first 2×2 max-pooling layer, a third 3×3 convolutional layer, a fourth 3×3 convolutional layer, a second 2×2 max-pooling layer, a fifth 3×3 convolutional layer, a sixth 3×3 convolutional layer, a third 2×2 max-pooling layer, a seventh 3×3 convolutional layer, an eighth 3×3 convolutional layer, a fourth 2×2 max-pooling layer, a ninth 3×3 convolutional layer, a tenth 3×3 convolutional layer, a first 2×2 upsampling layer, an eleventh 3×3 convolutional layer, a twelfth 3×3 convolutional layer, a second 2×2 upsampling layer, a thirteenth 3×3 convolutional layer, a fourteenth 3×3 convolutional layer, a third 2×2 upsampling layer, a fifteenth 3×3 convolutional layer, a sixteenth 3×3 convolutional layer, a fourth 2×2 upsampling layer, a seventeenth 3×3 convolutional layer, an eighteenth 3×3 convolutional layer, and an output layer deployed in sequence from the input to the output direction. The output of the second 3×3 convolutional layer is skip-connected to the input of the fourth 2×2 upsampling layer, the output of the fourth 3×3 convolutional layer is skip-connected to the input of the third 2×2 upsampling layer, the output of the sixth 3×3 convolutional layer is skip-connected to the input of the second 2×2 upsampling layer, and the output of the eighth 3×3 convolutional layer is skip-connected to the input of the first 2×2 upsampling layer.

[0110] 3. Complete Tumor Ablation Plan

[0111] Since large tumors require multiple ablation covers, the ablation radius is uniformly fixed at the maximum value of ablation. The optimization goal is to minimize the number of punctures and ablations as much as possible. In this embodiment, the branch cutting algorithm is used to solve this problem, combining the branch and bound algorithm and the cutting plane algorithm.

[0112] Specifically, given a finite set (tumor) in three-dimensional space, the set of ablation spheres is

[0113] X = {B(x1, r1), B(x2, r2), …, B(x n , r n )}, where x i ∈R 3 , r i ∈R, and B(x i , r i )

[0114] where T represents a finite set, representing the position of the tumor in three-dimensional space, R represents the set of real numbers, and R 3 represents three-dimensional real space, that is, the set of all possible three-dimensional coordinates (x, y, z). In B(x i , r i ), x i serves as the center of the sphere, and r i serves as the radius of the sphere. The goal is to create K is the ablation sphere used in the actual ablation process, and n is the maximum number of puncture ablations, which can be taken as a sufficiently large fixed value according to the actual situation. This problem can be expressed as a constrained optimization problem, and its goal is to minimize the number of ablation spheres / punctures used:

[0115]

[0116] where k i takes the value of 0 or 1, where 1 means the i-th sphere actually participates in tumor ablation, and 0 means no corresponding puncture ablation process is performed; p is the tip of the tumor; ||p - x i || is the Euclidean distance from p to the center of the sphere x i ; T represents a finite set, representing the position of the tumor in three-dimensional space, a represents the distance between the point p and the center of the ablation sphere x i , and b represents the radius r i of the i-th ablation sphere. The parameters to be optimized include k i , x i , and r i . The objective function in the first line of formula (23) represents minimizing the number of punctures, while the constraint in the second line represents that each point in the tumor is covered by at least one ablation sphere.

[0117] 4. Establishment of the Wooden Frame for Animal Experiments

[0118] To verify the accuracy of puncture, live pigs were used in this experiment for puncture. Therefore, after anesthesia, the animals needed to be fixed on a special bracket. Thus, we made a wooden bracket according to the size of the pigs so that the pigs could remain stable on the bracket throughout the experiment. The WCS tracking target was fixed on the wooden frame, and the relative positions of the complete body shape of the pig and the WCS tracking object could be obtained through CT scanning. After CT scanning the pig, the CT data of the pig liver could be obtained. The CT data was processed and the coordinates of the optical navigation marker points were obtained to generate an ablation plan, including the puncture target points and the puncture path. The navigation system guided the robotic arm to automatically insert the puncture needle into the animal's body according to the preset path based on the coordinates of the marker points and the ablation plan, completing the ablation process. Another CT scan of the animal (with the puncture needle remaining in place) was performed to verify the correctness of the puncture process.

[0119] 5. Operating Principle of the Hardware

[0120] The high-precision liver tumor ablation system based on optical navigation consists of an ablation robot, an optical navigation system, a CT image data processing system, and marker points fixed on the experimental object.

[0121] The process of one ablation experiment is as follows:

[0122] The experimental object was scanned by CT. During the scanning process, the marker points moved synchronously with the experimental object on the CT table, and the two remained relatively stationary, and the marker points and the experimental object were imaged in the CT at the same time.

[0123] After the scanning was completed, the CT image was transmitted to the CT image data processing system. The image processing software completed the segmentation of the liver and other relevant organs of the experimental object in the CT image, calculated the coordinates of the ablation target points in the CT coordinate system and the optimal path for the ablation needle to penetrate, as well as the coordinates of the marker points in the CT coordinate system.

[0124] The CT image data processing system transmitted the target points and the optimal path for the ablation needle to penetrate, as well as the marker point coordinates to the robot control system. Note that the above coordinates are all in the CT coordinate system.

[0125] Throughout the experiment, the optical navigation system continuously tracked the marker points on the experimental object and the marker points on the robotic arm, and transmitted the coordinates of the above two sets of marker points in the coordinate system of the optical navigation system to the robot control system. Among them, the marker points in the system are multiple reflective spheres that can be recognized and tracked by the navigation system, and the relative position of the ablation needle to the marker points on the robotic arm is known.

[0126] The robot control system uses the coordinates of the marked points on the experimental subject in the CT coordinate system and the optical navigation system coordinate system to achieve the conversion from the CT coordinate system to the optical navigation coordinate system; uses the coordinates of the marked points on the robotic arm in the optical navigation coordinate system and the robot coordinate system to achieve the conversion from the optical navigation coordinate system to the robot coordinate system; finally obtains the ablation needle insertion path and the coordinates of the ablation target point in the robot coordinate system, and then guides the robotic arm to achieve the ablation needle insertion.

[0127] 6. Software implementation path and pseudocode

[0128] The software of the high-precision liver tumor ablation system based on optical navigation includes the CT data processing software located on the CT image processing device and the robot control software located on the robot.

[0129] The robot control software uses the method based on singular value decomposition (SVD) to calculate the transformation matrix between different coordinate systems, and improves the accuracy of the transformation process by sampling the marked points multiple times under different position and attitude conditions.

[0130] The main functions of the CT data processing software are: to accurately segment organs such as skin, bone, liver, gallbladder, etc. and tumors in the liver related to the ablation experiment according to the CT image; to calculate an ablation plan that can ensure 100% ablation of the tumor and minimize the damage to adjacent normal tissues, and give the final position and the entry path that the ablation needle needs to reach; to determine the position of the marker center in the CT image.

[0131] In the process of CT image segmentation, the system adopts corresponding processing algorithms for different types of organs. For parts such as skin, fat, muscle, bone, etc. with obvious differences in CT values from adjacent regions, satisfactory segmentation effects can be obtained by using classic image processing algorithms such as thresholding, region growing, and partial differential equations (PDE). For organs with relatively blurred boundaries such as the liver, gallbladder, and tumors, a deep segmentation network based on active contours (DNN with ACWE) is proposed by integrating the variational-based segmentation algorithm in classic medical image processing and the deep network design method, which effectively reduces the amount of data required for training the segmentation model, and the average accuracy of the segmentation reaches 95.3%, meeting the actual use requirements.

[0132] In the process of designing the optimal ablation plan, m alternative ablation points are preset. For example, they can be obtained by uniformly sampling the tumor and its adjacent regions. Note that the alternative points located in areas that cannot be punctured such as blood vessels need to be excluded, and the area to be ablated (expanded from the segmented tumor area) is discretized into n tetrahedral meshes, and the optimization objective function is obtained as:

[0133]

[0134] where k i takes values of 1 / 0, indicating whether the i-th candidate ablation point is selected, and c ij takes values of 0 / 1, indicating whether the i-th ablation covers the j-th tetrahedron. The system uses the Branch and Cut algorithm to solve this problem and obtain the exact positions of the ablation points. Finally, the ablation needle insertion path is determined according to the principle of avoiding key positions (bones, large blood vessels, etc.) and having the shortest path (the line connecting the skin puncture point and the ablation point). Using the above Branch and Cut algorithm, the integer programming problem or the mixed-integer programming problem can be solved. It combines the advantages of the branch and bound method and the cutting plane method, aiming to obtain the exact positions of the ablation points.

[0135] The processing flow of the CT data processing software is as follows:

[0136] Input: D: CT Value Data

[0137] 1. Segmentation

[0138] a) Normalize D → DN ∈ {-1, 1}

[0139] b) Thresholding + PDE(DN) → Segmentation mask of bone, skin, fat,...

[0140] c) DNN with ACWE(DN) → Segmentation mask of liver, tumor,...

[0141] 2. Planning

[0142] a) Sampling (mask of tumor,...) → coordinate of candidate ablation points

[0143] b) Mesh Generation (mask of tumor) → vertices

[0144] c) Branch and Cut (candidate ablation points, vertices) → k i

[0145] d) k i , mask of skin, fat, bone,... → ablation points

[0146] 7. CT Image Segmentation Experiment

[0147] To achieve the full - automatic and high - precision recognition and segmentation of the liver and its adjacent organs, blood vessels, bile ducts, tumor tissues in the liver, as well as skin, fat, muscle, bones, etc. in the puncture path, this embodiment uses abdominal CT images as the modeling data source to establish a 3D digital model required for the ablation process. There are a total of 212 CT samples. 208 images with an accuracy greater than 0.9 account for 98% of the total samples, and 4 images with an accuracy less than 0.9 account for 2% of the total number of samples, which indicates that the automatic segmentation results are basically consistent with the true values. The final segmentation accuracy (Dice coefficient, 1 represents perfect accuracy) is as shown in Figure 7 shown. The average segmentation accuracy rate reaches 95.25%.

[0148] 8. Tumor Ablation Coverage Plan

[0149] The inactivated area during a single radiofrequency ablation process is spherical. Therefore, the design of a complete tumor ablation plan can be abstracted as a mathematical optimization problem, that is, using multiple spheres (single ablation area) to completely cover an irregular body (tumor). The optimization goal is to completely cover the lesion area with ablation areas. The design of the tumor ablation plan can be abstracted as the problem of the best coverage of an irregular object using multiple spheres. During the test process, tumors larger than 3 cm were selected, and different ablation radii, such as 10 mm and 15 mm, were chosen to verify the effectiveness of the algorithm in covering large tumors. The principle of ablating large tumors is to cover the entire area of the tumor with the fewest number of punctures while ensuring that normal tissues are not damaged. In the case of limited single - ablation range, more punctures are required to achieve complete coverage ablation of large tumors. In response to this problem, considering the future development of ablation technology, this embodiment also studied the ablation plan for large tumors when the single - ablation range is not limited. The inventors discovered a heuristic algorithm for finding a locally optimal feasible solution, and the algorithm design inspiration comes from the idea of the greedy algorithm. The principle of this algorithm is to first cover the largest possible area and then sequentially complete the coverage of the remaining areas. This algorithm was also verified in 4 large - tumor samples. The results of the ablation plan design achieved complete coverage of large tumors, and the number of ablations was 2 - 4 times, significantly reduced compared with the case of limited single - ablation volume, as shown in Table 3. The test data is as shown in the following figure, and the maximum tumor size reaches 8 cm. The ablation plan can be generated on a test laptop (CPU: Intel 12700H 14 - core, 64GB memory) in no more than 80 seconds, which can meet the actual ablation requirements.

[0150] Table 3: Results of Tumor Coverage Algorithm

[0151]

[0152] Among them, Oi : Represents the origin position of coordinate system i, R i : Represents the rotation matrix of coordinate system i. S / T approaching 1 indicates that the effect reaches the expected value.

[0153] 5. Phantom Experiment

[0154] To test the accuracy of the robotic puncture needle, 10 different position points were selected within the WCS tracking target range for testing. Among them, x, y, and z are the coordinates of the center of each TCP tracking target measured by the measuring device. The relative position of the WCS tracking target w The T coordinate can be detected by CT scan. Tx, Ty, and Tz are the coordinates of the end flange returned by the robot control, while r x , r y and r z are the end postures recorded by the robot control in pitch angle format. Coordinate b T c · c T w · w T can be converted into a motion command in URScript format to control the robot to move towards the target point on the puncture body, thereby controlling the robot to complete the expected puncture action. The specific data recorded during the hand-eye calibration process are shown in the supplementary Table 4-7. The final accuracy test result after calibration shows an error of 1.733 mm. Through hand-eye calibration, the conversion relationship between the ablation robot and the measuring device, that is, the hand-eye matrix, was obtained in this project. This matrix describes the position and posture of the robot hand in the camera coordinate system, enabling the robot to accurately execute tasks under the guidance of camera vision.

[0155] Table 4: Original data of hand-eye calibration

[0156]

[0157]

[0158] Table 5: Original data of hand-eye calibration.

[0159]

[0160]

[0161]

[0162] Table 6: Attitude data of hand-eye calibration.

[0163]

[0164]

[0165] Attitude data obtained by the measurement device and the robot

[0166]

[0167] 6. System verification through in vivo porcine liver experiments

[0168] Previous in vitro simulation experiments have verified that the robot can accurately reach the specified position under optical navigation. Therefore, in order to verify the feasibility and accuracy of the entire system when operating on animals, we conducted experiments using pigs. First, the pigs were anesthetized and fixed on a dedicated bracket. Since lipiodol shows high density on CT, we injected a needle containing lipiodol into the liver under the guidance of ultrasound to form a lipiodol-containing area to simulate a tumor in the liver (cigurw10A). After injection, we used a spiral CT (OptimaTM CT680) to scan the entire pig and the wooden bracket to obtain the CT data of the animal's liver. As shown in the figure, the specific position of the lipiodol in the area was obtained through optical navigation. Then, through CT data processing, the coordinates of the central area of the lipiodol could be obtained, and through the precise segmentation of the tumor in the liver and other adjacent tissues, an ablation plan was generated according to the branch cutting algorithm. The corresponding puncture target points and puncture paths were generated by the CT data processing software in the system. After obtaining the specific position, the robotic arm could accurately avoid key structures such as large blood vessels, bones, and other organs, and pierce the puncture point for ablation. Finally, the puncture needle was fixed in the animal's body, and another CT scan of the animal was performed according to the specific position of the puncture needle and the lipiodol to verify the accuracy of the puncture. After planning the puncture path, during actual operation, the puncture needle passed through the middle of the rib and successfully avoided the rib. The distances between the two ribs were 2.592 mm and 3.796 mm respectively.

[0169] The above is the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present application.

Claims

1. A tumor ablation system, characterized in that, The tumor ablation system acts on an ablation object with a first marker, and the first marker is relatively stationary with respect to the ablation object. The system includes: An execution robot configured with a first coordinate system, a robotic arm provided with a second marker, an ablation needle assembled on the robotic arm, the relative position of the ablation needle and the second marker being determined, and the second marker having a first marker coordinate in the first working coordinate system; An image processing subsystem configured with a second coordinate system for obtaining a pathological image, the ablation object and the first marker being imaged in the pathological image, and performing intelligent segmentation on the pathological image to obtain ablation target positioning information of the ablation object in the second coordinate system, and calculating an ablation strategy determined based on the ablation target positioning information, the first marker having a second marker coordinate in the second coordinate system; An optical navigation subsystem configured with a third coordinate system for tracking the first marker and the second marker, the first marker having a third marker coordinate in the third coordinate system, and the second marker having a fourth marker coordinate in the third coordinate system; and A robot control subsystem for realizing coordinate transformation of the ablation target positioning information based on the first marker coordinate, the second marker coordinate, the third marker coordinate, and the fourth marker coordinate, and controlling the action of the execution robot based on the coordinate transformation result.

2. The tumor ablation system according to claim 1, wherein The first coordinate system includes: a robot base coordinate system, an end effector coordinate system, and a tool coordinate system. There are transformation relationships between the robot base coordinate system and the end effector coordinate system, between the robot base coordinate system and the tool coordinate system, between the robot base coordinate system and the third coordinate system, between the end effector coordinate system and the tool coordinate system, and between the tool coordinate system and the third coordinate system.

3. The tumor ablation system according to claim 1, wherein The robot control subsystem is specifically configured to: Use the second marker coordinate and the third marker coordinate to realize the transformation of the ablation target positioning information from the second coordinate system to the third coordinate system; And Use the fourth marker coordinate and the first marker coordinate to realize the transformation of the ablation target positioning information from the third coordinate system to the first coordinate system.

4. The tumor ablation system according to claim 3, wherein Realize coordinate transformation based on the singular value decomposition method.

5. The tumor ablation system according to claim 1, wherein The image processing subsystem adopts a multi-task UNet network architecture, and the multi-task UNet network architecture adopts at least two segmentation algorithms for different categories of organs. The first segmentation algorithm includes: threshold, region growing, and partial differential equations, and the second segmentation algorithm is a depth segmentation network based on active boundaries.

6. The tumor ablation system according to claim 5, wherein The multi-task UNet network architecture specifically includes: a first 3×3 convolutional layer, a second 3×3 convolutional layer, a first 2×2 max pooling layer, a third 3×3 convolutional layer, a fourth 3×3 convolutional layer, a second 2×2 max pooling layer, a fifth 3×3 convolutional layer, a sixth 3×3 convolutional layer, a third 2×2 max pooling layer, a seventh 3×3 convolutional layer, an eighth 3×3 convolutional layer, a fourth 2×2 max pooling layer, a ninth 3×3 convolutional layer, a tenth 3×3 convolutional layer, a first 2×2 upsampling layer, an eleventh 3×3 convolutional layer, a twelfth 3×3 convolutional layer, a second 2×2 upsampling layer, a thirteenth 3×3 convolutional layer, a fourteenth 3×3 convolutional layer, a third 2×2 upsampling layer, a fifteenth 3×3 convolutional layer, a sixteenth 3×3 convolutional layer, a fourth 2×2 upsampling layer, a seventeenth 3×3 convolutional layer, an eighteenth 3×3 convolutional layer, and an output layer, which are deployed in sequence from the input to the output direction. The output of the second 3×3 convolutional layer is skip-connected to the input of the fourth 2×2 upsampling layer, the output of the fourth 3×3 convolutional layer is skip-connected to the input of the third 2×2 upsampling layer, the output of the sixth 3×3 convolutional layer is skip-connected to the input of the second 2×2 upsampling layer, and the output of the eighth 3×3 convolutional layer is skip-connected to the input of the first 2×2 upsampling layer.

7. The tumor ablation system according to claim 5, wherein, The image processing subsystem is specifically used for: Adopting a branch-and-cut algorithm and a greedy algorithm, under the condition that the ablation range is limited in a single time, with the goal of minimizing the number of punctures, the ablation strategy including the ablation path is obtained.

8. The tumor ablation system according to claim 7, wherein The parameters to be optimized in the multi-task UNet network architecture include: k i , x i and r i , where k i takes a value of 0 or 1, where 1 indicates that the i-th ablation sphere actually participates in tumor ablation, and 0 indicates that the corresponding puncture ablation process is not performed; x i is the center of the ablation sphere, and r i is the radius of the ablation sphere.

9. The tumor ablation system according to claim 1, wherein The first marker and the second marker are four reflective balls.

Images