An ultrasound positioning system based on liver tumor ablation and method thereof

Abstract

The application provides an ultrasound positioning system based on liver tumor ablation and a method thereof, which comprises the following steps: performing a first CT scanning to obtain an initial medical image as an initial registration image; obtaining an ablation needle intervention position, an ultrasound scanning head alignment range, and a path and a posture of a robot; obtaining an ultrasound medical image; performing a three-dimensional ultrasound image reconstruction; registering an ultrasound three-dimensional image coordinate system with the initial medical image coordinate system, and registering an end effector coordinate system with the ultrasound three-dimensional image coordinate system; obtaining a registration fusion image; generating a robot trajectory path, and realizing a precise needle entry path planning based on the initial medical image and a safe guidance based on the ultrasound three-dimensional image through a surgical robot, so that the surgical robot executes the planned robot trajectory path. According to the position of a liver tumor of a patient, the application can realize the precise needle entry path planning based on the preoperative image and the safe guidance based on the intraoperative image.

Description

Technical Field

[0001] This invention relates to the field of ultrasound imaging technology, specifically to an ultrasound localization system based on liver tumor ablation and a method for using the system. Background Technology

[0002] According to a 2022 report by the World Health Organization (WHO), approximately 905,000 new cases of liver cancer are diagnosed globally each year, and about 830,000 patients die from liver cancer. Hepatocellular carcinoma (HCC), as a type of malignant tumor, has become one of the world's major health problems. Traditional ablation interventional surgery generally involves the surgeon using an 18G puncture to precisely target and ablate the tumor site, with the reference of preoperative and intraoperative X-ray images. However, in practice, this surgical method has many limitations, such as: the mapping between the image and the actual tumor location is not intuitive enough; surgical precision is difficult to guarantee and requires the surgeon to judge the needle insertion position based on experience; it is highly invasive to the patient; and the surgery is difficult, thus prolonging the operation time and inevitably bringing more radiation to both doctors and patients.

[0003] There are few surgical robot systems for liver tumor ablation interventional surgery in this field. For example, patent application number CN201910737901 describes a liver segment resection auxiliary system based on three-dimensional reconstruction and holographic display of medical images. This system registers the patient's CT / MRI scan sequence images through multiple processing modules, performs preoperative simulated liver segment division and risk analysis, and then conducts preoperative planning. In this technical solution, the system prioritizes using AR glasses technology to display the preoperative planning results to assist medical staff in the surgery. However, AR glasses technology is not yet fully mature and still has significant errors for tumor resection surgery with high precision requirements, lacking safety. In addition, this system cannot be integrated into surgical robots with higher automation and greater convenience for surgeons, and the system uses CT scan images, resulting in high radiation doses to both doctors and patients, which can cause harm to both.

[0004] Patent application CN200910273516 discloses a comprehensive surgical planning simulation method and system for liver tumors based on three-dimensional multimodal imaging. Its design principle is similar to the aforementioned patent, planning the preoperative surgical path through three-dimensional registration and fusion, simulation, and analysis. However, this patent document does not propose an automated hardware surgical system to match this system, and its registration accuracy cannot be guaranteed; the entire surgical procedure is an open-loop operation. Similar to the aforementioned patent, this system uses CT scan images, resulting in prolonged exposure to radiation, which can cause varying degrees of harm to both the surgeon and the patient.

[0005] A precision-positioning tumor surgery robot system, patent application number CN201821342041, achieves positioning and surgical operations solely through a six-degree-of-freedom robotic arm, a tracking marker assembly, and a far-infrared camera. However, this technical solution only emphasizes the hardware of the robot system, failing to provide detailed steps and schemes for how an external computer performs precise positioning. The solution fails to demonstrate the precise registration relationship between the system and medical images, thus failing to guarantee surgical accuracy and protect patient safety. Summary of the Invention

[0006] To address the aforementioned shortcomings of existing technologies, this invention provides an ultrasound positioning system and method for liver tumor ablation, which meets the requirements for clinical operation. This system and method integrate the ultrasound positioning system into the robot. After completing the registration of various coordinate systems, the preoperative medical images and intraoperative ultrasound images are unified under the robot's positioning coordinate system, enabling the robot to achieve precise needle insertion path planning based on preoperative images and safe guidance based on intraoperative images, according to the location of the patient's liver tumor.

[0007] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0008] An ultrasound positioning system based on liver tumor ablation is disclosed. The system includes an electromagnetic positioning and tracking system and a surgical robot. The surgical robot includes a robotic arm trolley module with a series robotic arm module mounted on it. Each series robotic arm module includes a six-degree-of-freedom robotic arm and an end effector connected to each other. The end portion of the end effector is an ultrasound scanning head with a puncture needle. The electromagnetic positioning and tracking system includes an electromagnetic tracking system and an electromagnetic position sensor mounted on the ultrasound scanning head. The puncture needle holder with the electromagnetic position sensor is fixed to the ultrasound scanning head. The movement of the six-degree-of-freedom robotic arm drives the ultrasound scanning head that operates the end effector to acquire real-time ultrasound scanning images. The surgical robot executes a planned robot trajectory path to complete the ultrasound positioning of the puncture needle intervention position assisted by the surgical robot.

[0009] The electromagnetic tracking system integrates an electromagnetic tracking system coordinate system.

[0010] An ultrasound-guided localization method for liver tumor ablation, applied to the aforementioned ultrasound-guided localization system for liver tumor ablation, includes the following steps:

[0011] A first CT scan is performed to acquire initial medical images, which are used as initial registration images.

[0012] The initial medical images are used to obtain the ablation needle insertion location, the ultrasound scanning head alignment range, and the robot's path and posture.

[0013] Perform an ultrasound scan to acquire ultrasound medical images;

[0014] Reconstruction of three-dimensional ultrasound images based on ultrasound medical imaging;

[0015] The reconstructed ultrasound 3D image coordinate system is registered with the initial medical image coordinate system using a registration algorithm between 3D ultrasound and preoperative CT images, and the end effector coordinate system is registered with the ultrasound 3D image coordinate system.

[0016] The coordinate transformation relationship between the end effector coordinate system, the ultrasound 3D image coordinate system, and the initial medical image coordinate system represented in the electromagnetic tracking and positioning coordinate system is obtained, and a registration and fusion image based on the initial medical image and the ultrasound 3D image is obtained.

[0017] Based on the registration and fusion of images, a robot trajectory path is generated. The surgical robot realizes precise needle insertion path planning based on the initial medical images and safe guidance based on ultrasound three-dimensional images, so that the surgical robot can execute the planned robot trajectory path.

[0018] A second CT scan was performed to adjust the robot's trajectory and complete the ultrasound positioning of the surgical robot-assisted puncture needle insertion point.

[0019] According to the present invention, an ultrasound localization method based on liver tumor ablation is used to reconstruct a three-dimensional ultrasound image, comprising:

[0020] Three-dimensional reconstruction of ultrasound medical images was performed using an ultrasound reconstruction algorithm based on weighted least squares estimation of voxels in the ultrasound images. That is, to calculate its blocks in three dimensions to perform three-dimensional reconstruction of ultrasound medical images.

[0021] According to the ultrasound localization method for liver tumor ablation provided by the present invention, the optimal representation of the weighted least squares method in the equation is expressed by matrix as formulas (1)-(4):

[0022]

[0023] Y = [Y1, Y2, ..., Y] p ] T (9)

[0024]

[0025] W x =diag[K h (X0-X),Kh (X1-X),...,K h (X p -X)] (11)

[0026] Where X represents two-dimensional ultrasound scan image data, Y represents preoperative CT image data, and “diag” indicates that the element of the diagonal matrix in (i,i) is Kh(·).

[0027] According to the present invention, an ultrasound localization method based on liver tumor ablation is provided to estimate voxels in ultrasound medical images. That is, to calculate its block in three dimensions, at which point the voxel estimate is optimized based on the weighted least squares method. The expression is given by formula (5):

[0028]

[0029] Where e1 is a column vector in which all elements except the first column are 0. It is an estimate; the characteristic matrix X x Written as formula (6):

[0030]

[0031] According to the ultrasound localization method based on liver tumor ablation provided by the present invention, the registration algorithm of the three-dimensional ultrasound and the preoperative CT image determines the registration relationship by calculating the maximum normalized cross-correlation between the blocks of the reconstructed ultrasound three-dimensional image and the blocks of the initial medical image, thereby obtaining the three-dimensional spatial transformation matrix of the second spatial pose.

[0032] According to the present invention, an ultrasound localization method based on liver tumor ablation is provided, which uses an image registration framework to acquire a reference image and a floating image, wherein the reference image comes from the reconstructed three-dimensional ultrasound image and the floating image comes from the initial medical image of the first CT scan.

[0033] Using a block matching algorithm, a point correspondence is established between the current transformed floating-point image and the reference image by dividing the reference image and the transformed floating-point image into blocks of the same size.

[0034] Each block in the reference image is compared with the corresponding image block in the neighborhood of the transformed floating image, where the matching block is the block with the maximum normalized cross-correlation with the reference block, and its calculation is expressed as formula (7):

[0035]

[0036] Where, r r (X) and r f(X) represents the voxels in the reference and floating images, i.e., the blocks in the reference and floating images, respectively. μ and σ correspond to the mean and standard deviation values ​​within the block, and N is the number of voxels in the block.

[0037] According to the ultrasound localization method for liver tumor ablation provided by the present invention, the LTS regression algorithm is used to calculate the transformation parameters according to the obtained image block correspondence, and the iterative execution includes: optimization starting from a large block search neighborhood, and then moving to a smaller search neighborhood with a coarse-to-fine strategy;

[0038] Obtain the coordinate transformation relationship between the ultrasound three-dimensional image coordinate system and the initial medical image coordinate system.

[0039] The ultrasound positioning method based on liver tumor ablation provided by the present invention obtains the coordinate transformation relationship between the end effector coordinate system, the ultrasound three-dimensional image coordinate system and the initial medical image coordinate system in the electromagnetic tracking positioning coordinate system. At the same time, it also integrates the robot's own coordinate system into the electromagnetic tracking positioning coordinate system. Thus, all coordinate systems can be expressed in the electromagnetic tracking positioning coordinate system, and a registration and fusion image based on the initial medical image and the ultrasound three-dimensional image can be obtained.

[0040] According to the present invention, an ultrasound-guided localization method based on liver tumor ablation receives an indication of movement of a surgical area relative to at least one surgical instrument detected by an electromagnetic position sensor. The electromagnetic position sensor is configured to detect the movement of the surgical area relative to at least one surgical instrument and to automatically control a six-degree-of-freedom robotic arm in response to the received indication, so as to compensate for the movement of the surgical area by maintaining the position of at least one surgical instrument along a planned robot trajectory path.

[0041] Therefore, compared to existing technologies,

[0042] The robotic system and method provided by this invention are designed for the process of planning needle insertion for the surgeon before the procedure, without directly operating on the surgical patient. Before inserting the needle to ablate the tumor site, the robot's end effector operates an ultrasound probe to scan and image. Based on the preoperative CT images, the insertion position of the ablation needle, the alignment range of the ultrasound scanning head, and the path and posture of the robot are planned. This allows for automatic needle insertion path planning, ensuring that the planned path conforms to the actual location of the liver tumor and guaranteeing the accuracy of the ablation.

[0043] Furthermore, all the coordinate systems mentioned in this invention can be expressed through a unified electromagnetic tracking system coordinate system, which can be used to obtain preoperative CT-intraoperative ultrasound registration and fusion images. This allows the robot to achieve precise needle insertion path planning based on preoperative images and safe guidance based on intraoperative images, according to the location of the patient's liver tumor. The robot can then execute the planned movements, and the ablation needle is clamped at the needle holder. The doctor then manually inserts the needle to complete the robot-assisted positioning and ablation. At the same time, a second CT scan allows the robot system to adjust in time during the operation based on the puncture error, ultimately completing the robot-assisted ablation.

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of an embodiment of an ultrasound positioning system based on liver tumor ablation according to the present invention.

[0046] Figure 2 This is a schematic diagram illustrating an application scenario of an ultrasound-guided positioning system based on liver tumor ablation according to an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of the structure of the serially connected robotic arm module and the fixture fixed to the ultrasound scanning head in an embodiment of an ultrasound positioning system based on liver tumor ablation according to the present invention.

[0048] Figure 4 This is a schematic diagram of the structure of the serial robotic arm module and the fixture fixed to the ultrasound scanning head from another perspective in an embodiment of the ultrasound positioning system based on liver tumor ablation of the present invention.

[0049] Figure 5 This is a flowchart of an embodiment of an ultrasound localization method based on liver tumor ablation according to the present invention.

[0050] Figure 6 This is a schematic diagram showing the relationship between various coordinate systems in an embodiment of an ultrasound localization method based on liver tumor ablation according to the present invention.

[0051] Figure 7 This is a schematic diagram of the registration framework used in the registration algorithm of three-dimensional ultrasound and preoperative CT images in an embodiment of an ultrasound localization method based on liver tumor ablation according to the present invention.

[0052] Figure 8 This is a flowchart illustrating the workflow of the ultrasound positioning system in an embodiment of an ultrasound positioning method based on liver tumor ablation according to the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] An embodiment of an ultrasound-guided localization system based on liver tumor ablation

[0055] See Figures 1 to 4 This invention provides an ultrasound positioning system based on liver tumor ablation. The system includes an electromagnetic positioning and tracking system and a surgical robot. The surgical robot includes a robotic arm moving cart module, on which a series robotic arm module is mounted. The series robotic arm module includes a six-degree-of-freedom robotic arm and a robotic arm end effector connected to each other. The end part of the robotic arm end effector is an ultrasound scanning head with a puncture needle. The electromagnetic positioning and tracking system includes an electromagnetic tracking system and an electromagnetic position sensor mounted on the ultrasound scanning head. The puncture needle holder with the electromagnetic position sensor is fixed to the ultrasound scanning head. The movement of the six-degree-of-freedom robotic arm drives the ultrasound scanning head that operates the robotic arm end effector to obtain real-time ultrasound scanning images. The surgical robot executes the planned robot trajectory path to complete the ultrasound positioning of the puncture needle intervention position assisted by the surgical robot. The electromagnetic tracking system integrates an electromagnetic tracking system coordinate system.

[0056] Specifically, such as Figure 1 and 2 As shown, Figure 1 The system's overall composition is shown, including: a robotic arm trolley module, which carries the robotic arm used in surgery; a serial robotic arm module, which includes a six-degree-of-freedom robotic arm and an end effector, representing a general-purpose robotic arm combination; and an operating table module. Figure 3 and Figure 4 The serial-parallel robotic arms shown in the example represent a type of universally applicable robotic arm assembly. This embodiment uses the commercially available UR5 collaborative robotic arm, a self-modeled ultrasound scanning head assembly (including a puncture needle holder 32 and a self-modeled flange connector 33), and a position sensor mounted on the ultrasound scanning head as examples to illustrate the system's composition and working principle. The puncture needle holder 32, equipped with an electromagnetic position sensor, is fixed to the ultrasound scanning head 31; the entire ultrasound scanning head assembly is fixed to the end effector of the two serially connected robotic arms. The flexible movement of the six-degree-of-freedom serially connected robotic arms automatically operates the end effector (ultrasound scanning head) to perform real-time ultrasound scanning of the affected area.

[0057] An Example of an Ultrasound Localization Method Based on Liver Tumor Ablation

[0058] See Figure 5 This invention provides an ultrasound localization method based on liver tumor ablation. This method is applied to the aforementioned ultrasound localization system based on liver tumor ablation, and includes the following steps:

[0059] Step S1: Perform the first CT scan to acquire initial medical images, which are used as initial registration images.

[0060] Step S2: Obtain the ablation needle intervention position through initial medical images, obtain the alignment range of the ultrasound scanning head, and the path and posture of the robot.

[0061] Step S3: Perform an ultrasound scan to acquire ultrasound medical images;

[0062] Step S4: Reconstruct the three-dimensional ultrasound image based on the ultrasound medical image;

[0063] Step S5: Using a registration algorithm between three-dimensional ultrasound and preoperative CT images, the reconstructed ultrasound three-dimensional image coordinate system is registered with the initial medical image coordinate system, and the end effector coordinate system is registered with the ultrasound three-dimensional image coordinate system.

[0064] Step S6: Obtain the coordinate transformation relationship between the end effector coordinate system, the ultrasound three-dimensional image coordinate system and the initial medical image coordinate system represented in the electromagnetic tracking and positioning coordinate system, and obtain the registration and fusion image based on the initial medical image and the ultrasound three-dimensional image.

[0065] Step S7: Generate robot trajectory path based on registration and fusion images. The surgical robot realizes precise needle insertion path planning based on initial medical images and safe guidance based on ultrasound three-dimensional images, so that the surgical robot executes the planned robot trajectory path.

[0066] Step S5: Perform a second CT scan, adjust the robot's trajectory path, and complete the ultrasound positioning of the surgical robot-assisted puncture needle insertion position.

[0067] In step S4 above, the reconstruction of the ultrasound three-dimensional image is performed, including:

[0068] Three-dimensional reconstruction of ultrasound medical images was performed using an ultrasound reconstruction algorithm based on weighted least squares estimation of voxels in the ultrasound images. That is, to calculate its blocks in three dimensions to perform three-dimensional reconstruction of ultrasound medical images.

[0069] In this embodiment, the optimal representation of the weighted least squares method in the equation is expressed in matrix form as formulas (1)-(4):

[0070]

[0071] Where Y = [Y1, Y2, ..., Y] p ] T (16)

[0072]

[0073] W x =diag[K h (X0-X),K h (X1-X),...,K h (X p -X)] (18)

[0074] Where X represents two-dimensional ultrasound scan image data, Y represents preoperative CT image data, and “diag” indicates that the element of the diagonal matrix in (i,i) is Kh(·).

[0075] In this embodiment, voxels of ultrasound medical images are estimated. That is, to calculate its block in three dimensions, at which point the voxel estimate is optimized based on the weighted least squares method. The expression is given by formula (5):

[0076]

[0077] Where e1 is a column vector in which all elements except the first column are 0. It is an estimate; the characteristic matrix X x Written as formula (6):

[0078]

[0079] In this embodiment, the registration algorithm between three-dimensional ultrasound and preoperative CT images determines the registration relationship by calculating the maximum normalized cross-correlation between the blocks of the reconstructed three-dimensional ultrasound image and the blocks of the initial medical image, thereby obtaining the three-dimensional spatial transformation matrix of the second spatial pose.

[0080] In this embodiment, an image registration framework is used to acquire a reference image and a floating image. The reference image is derived from the reconstructed 3D ultrasound image, and the floating image is derived from the initial medical image from the first CT scan.

[0081] Using a block matching algorithm, a point correspondence is established between the current transformed floating-point image and the reference image by dividing the reference image and the transformed floating-point image into blocks of the same size.

[0082] Each block in the reference image is compared with the corresponding image block in the neighborhood of the transformed floating image, where the matching block is the block with the maximum normalized cross-correlation with the reference block, and its calculation is expressed as formula (7):

[0083]

[0084] Where, r r (X) and r f (X) represents the voxels in the reference and floating images, i.e., the blocks in the reference and floating images, respectively. μ and σ correspond to the mean and standard deviation values ​​within the block, and N is the number of voxels in the block.

[0085] Then, the LTS regression algorithm is used to calculate the transformation parameters based on the obtained image patch correspondences. The iterative process includes: optimization starting from the large patch search neighborhood, and then moving to the smaller search neighborhood in a coarse-to-fine strategy.

[0086] Obtain the coordinate transformation relationship between the ultrasound three-dimensional image coordinate system and the initial medical image coordinate system.

[0087] In this embodiment, while obtaining the coordinate transformation relationship between the end effector coordinate system, the ultrasound 3D image coordinate system, and the initial medical image coordinate system represented in the electromagnetic tracking and positioning coordinate system, the robot's own coordinate system is also incorporated into the electromagnetic tracking and positioning coordinate system. This allows all coordinate systems to be expressed in the electromagnetic tracking and positioning coordinate system, thus obtaining a registered and fused image based on the initial medical image and the ultrasound 3D image.

[0088] In this embodiment, an electromagnetic position sensor receives an indication of movement in the surgical area relative to at least one surgical instrument, the electromagnetic position sensor being configured to detect movement in the surgical area relative to at least one surgical instrument, and to automatically control a six-degree-of-freedom robotic arm in response to the received indication, so as to compensate for the movement of the surgical area by maintaining the position of at least one surgical instrument along a planned robot trajectory path.

[0089] like Figure 6 As shown, the system in this embodiment involves five main coordinate systems: the electromagnetic positioning coordinate system {O}, the initial medical image coordinate system (i.e., the CT preoperative image coordinate system {C}), the robot's end effector coordinate system {E}, the ultrasound three-dimensional image coordinate system {S}, and the robot's own coordinate system {B}.

[0090] like Figure 8 As shown in the system workflow diagram, firstly, a first CT scan is performed on the patient's affected area. The surgical plan is then made through the graphical user interface. Based on the preoperative CT images, the ablation needle insertion position is planned, as well as the alignment range of the ultrasound scanning head and the path and posture of the robot are planned.

[0091] Then, the flexible movement of the six-degree-of-freedom tandem robotic arm automatically operates the end effector (ultrasound scanning head) to perform real-time intraoperative ultrasound scanning of the affected area. For example... Figure 6 As shown, the electromagnetic positioning and tracking system is a type of position measurement system. By fixing an electromagnetic position sensor to the ultrasonic probe, real-time tracking of the spatial pose of the end effector of the robotic arm, i.e., the ultrasonic probe, can be achieved. This system uses the local coordinate system of the electromagnetic position sensor itself, mounted on a self-made puncture needle holder. This means that the electromagnetic tracker can directly acquire the position and orientation of the end effector coordinate system {E}. Furthermore, because the electromagnetic position sensor is mounted on the ultrasonic probe, real-time tracking of the spatial pose of the end effector of the robotic arm, i.e., the ultrasonic probe, can be achieved through the electromagnetic tracking sensor during ultrasonic scanning, unifying the end effector coordinate system {E} and the ultrasonic three-dimensional image coordinate system {S} under the electromagnetic tracking system coordinate system {O}.

[0092] To obtain the coordinate transformation relationship between the intraoperative ultrasound three-dimensional image coordinate system {S} and the CT preoperative image coordinate system {C}, this embodiment uses two different algorithms, which are executed in two steps.

[0093] The first step is to use an ultrasound reconstruction algorithm to perform three-dimensional (3D) reconstruction of the intraoperative ultrasound medical images (2D). This ultrasound reconstruction algorithm is based on the weighted least squares method to estimate the voxels of the ultrasound two-dimensional images. That is, to calculate its blocks in three dimensions to perform three-dimensional reconstruction of the intraoperative ultrasound image {S}.

[0094] The reconstruction algorithm is described below:

[0095] The optimal representation of the weighted least squares method in the equation can be expressed by a matrix as shown in formulas (1)-(4) above. Its main objective is to estimate the voxels of two-dimensional ultrasound images. That is, to calculate its block in three dimensions, at which point the voxel estimate is optimized based on the weighted least squares method. The expression can be simplified from the above formula to the above formula (5):

[0096]

[0097] Here, e1 is a column vector where all elements except the first column are 0. This is the final estimate based on local examples. In this simplified representation, the feature matrix X... x It can be written as formula (6).

[0098] Therefore, this embodiment can use this algorithm to estimate the voxels of the intraoperative two-dimensional ultrasound image. This allows for the calculation of its blocks in three dimensions, thus enabling three-dimensional reconstruction of two-dimensional ultrasound images.

[0099] The second step involves registering the reconstructed 3D ultrasound image with the preoperative CT image coordinate system {C} using a registration algorithm between the 3D ultrasound and preoperative CT images. This intraoperative registration algorithm determines the registration relationship by calculating the maximum normalized cross-correlation (NCC) between blocks in the reconstructed ultrasound image and blocks in the preoperative CT image, thus obtaining the 3D spatial transformation matrix between the intraoperative abdominal surface pose and the preoperative spatial pose.

[0100] like Figure 7 As shown, this embodiment adopts a general registration framework, where the reference image comes from the reconstructed three-dimensional ultrasound image and the floating image comes from the preoperative CT image. In this block matching algorithm, a two-step method is used to perform registration. The first step is to establish a point correspondence between the current transformed floating image and the reference image using a block matching strategy. This is achieved by dividing the reference image and the transformed floating image into blocks of the same size. Each block in the reference image is compared with the corresponding image block in the neighborhood of the transformed floating image. The matching block is the block with the maximum normalized cross-correlation (NCC) with the reference block, calculated as shown in the above formula (7).

[0101] The second step involves using the LTS regression method to calculate the transformation parameters corresponding to these points, ensuring robust outlier removal. Since each LTS iteration yields an analytical solution, this optimization technique is also computationally efficient. The two steps are performed iteratively, with optimization starting with large block search neighborhoods (corresponding to coarse displacements) and then moving towards smaller search neighborhoods (corresponding to finer displacements) in a coarse-to-fine strategy.

[0102] Combining the two algorithms, the ultrasonic reconstruction algorithm is first used to calculate the blocks of the ultrasonic 3D image. Then, by using the intraoperative ultrasound-preoperative CT registration algorithm, the maximum normalized cross-correlation (NCC) between the blocks of the ultrasound 3D reconstructed image and the blocks of the preoperative CT image is calculated to determine their registration relationship, thereby obtaining the coordinate transformation relationship between the intraoperative ultrasound image {S} and the preoperative CT image {C}.

[0103] Based on the above, the coordinate transformation relationship between the end effector {E} represented in the electromagnetic tracking and positioning coordinate system {O}, the intraoperative ultrasound three-dimensional image coordinate system {S}, and the preoperative CT image coordinate system {C} can be obtained. At this point, the electromagnetic tracking and positioning coordinate system {O} can be used to represent the preoperative planned path and other coordinates defined in the preoperative CT image coordinate system {C}. Furthermore, due to the fixed geometric relationship of the robotic arm itself, its coordinate system {B} can be unified into the electromagnetic tracking system coordinate system {O}. Thus, all coordinate systems can be expressed through the unified electromagnetic tracking and positioning coordinate system {O}, allowing for the acquisition of preoperative CT-intraoperative ultrasound registration and fusion images. This enables the robot to achieve precise needle insertion path planning based on preoperative images and safe guidance based on intraoperative images, allowing the robot to execute the planned movements. Subsequently, the ablation needle is clamped in the needle holder, and the surgeon manually inserts the needle to complete robot-assisted positioning and ablation. Simultaneously, a second CT scan allows the robot system to adjust in real time during the operation based on puncture errors, ultimately completing robot-assisted ablation.

[0104] In summary, the robot system and method provided in this embodiment are designed for the process of planning the needle insertion for the surgeon before insertion, without directly operating on the surgical patient. Before inserting the needle to ablate the tumor site, the robot's end effector operates an ultrasound probe to scan and image. Based on the preoperative CT images, the insertion position of the ablation needle, the alignment range of the ultrasound scanning head, and the robot's path and posture are planned, automatically completing the needle insertion path planning. Although the patents mentioned in the prior art can accomplish some tasks, they suffer from immature technologies or lack of compatibility with automated hardware, or their solutions fail to demonstrate a precise registration relationship between the system and medical images. Therefore, it is difficult to guarantee whether the planned path of the system conforms to the actual location of the liver tumor and the accuracy of ablation. Therefore, this invention can ensure that the planned path of the system conforms to the actual location of the liver tumor and guarantees the accuracy of ablation.

[0105] Furthermore, all the coordinate systems mentioned in this embodiment can be expressed through a unified electromagnetic tracking system coordinate system. Based on this, the preoperative CT-intraoperative ultrasound registration and fusion images can be obtained, enabling the robot to achieve precise needle insertion path planning based on preoperative images and safe guidance based on intraoperative images according to the location of the patient's liver tumor. The robot can then execute the planned movements, and the ablation needle is clamped at the needle holder. The doctor then manually inserts the needle to complete the robot-assisted positioning and ablation. At the same time, a second CT scan allows the robot system to adjust in time during the operation according to the puncture error, ultimately completing the robot-assisted needle insertion and ablation.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for ultrasound localization based on liver tumor ablation, characterized in that, The method includes the following steps: A first CT scan is performed to acquire initial medical images, which are used as initial registration images. The initial medical images are used to obtain the ablation needle insertion location, the ultrasound scanning head alignment range, and the robot's path and posture. Perform an ultrasound scan to obtain ultrasound medical images; Reconstruction of three-dimensional ultrasound images based on ultrasound medical imaging; The reconstructed ultrasound 3D image coordinate system is registered with the initial medical image coordinate system using a registration algorithm between 3D ultrasound and preoperative CT images, and the end effector coordinate system is registered with the ultrasound 3D image coordinate system. The coordinate transformation relationship between the end effector coordinate system, the ultrasound 3D image coordinate system and the initial medical image coordinate system represented in the electromagnetic tracking and positioning coordinate system is obtained, and a registration and fusion image based on the initial medical image and the ultrasound 3D image is obtained. Based on the registration and fusion of images, a robot trajectory path is generated. The surgical robot realizes precise needle insertion path planning based on the initial medical images and safe guidance based on ultrasound three-dimensional images, so that the surgical robot can execute the planned robot trajectory path. A second CT scan was performed to adjust the robot's trajectory and complete the ultrasound positioning of the surgical robot-assisted puncture needle insertion point. Reconstruction of three-dimensional ultrasound images includes: Three-dimensional reconstruction of ultrasound medical images was performed using an ultrasound reconstruction algorithm based on weighted least squares estimation of voxels in the ultrasound images. That is, to calculate its blocks in three dimensions to reconstruct the three-dimensional ultrasound medical image; The optimal representation of the weighted least squares method in the equation is expressed in matrix form as formulas (1)-(4): （1） （2） （3） （4） Where X represents two-dimensional ultrasound scan image data, and Y represents preoperative CT image data. diag "Indicates that the elements of a diagonal matrix are in China is ; The registration algorithm between the three-dimensional ultrasound and the preoperative CT image determines the registration relationship by calculating the maximum normalized cross-correlation between the blocks of the reconstructed three-dimensional ultrasound image and the blocks of the initial medical image, and obtains the three-dimensional spatial transformation matrix between the first spatial pose and the second spatial pose. An image registration framework was used to acquire a reference image and a floating image. The reference image was obtained from the reconstructed 3D ultrasound image, and the floating image was obtained from the initial medical image of the first CT scan. Using a block matching algorithm, a point correspondence is established between the current transformed floating image and the reference image by dividing the reference image and the transformed floating image into blocks of the same size. Each block in the reference image is compared with the corresponding image block in the neighborhood of the transformed floating image, where the matching block is the block with the maximum normalized cross-correlation with the reference block, and its calculation is expressed as formula (7): （7） in, and These represent voxels in the reference and floating images, respectively; that is, blocks in the reference and floating images. μ and σ Corresponding to the mean and standard deviation values ​​within the block, N It is the number of voxels in the block.

2. The method according to claim 1, characterized in that: Estimating voxels in ultrasound medical images That is, to calculate its block in three dimensions, the voxel estimate is optimized based on weighted least squares. The expression is represented by formula (5): （5） in It is a column vector where all elements except the first column are 0. It is an estimated value; characteristic matrix Written as formula (6): （6）。 3. The method according to claim 1, characterized in that: use LTS The regression algorithm calculates transformation parameters based on the obtained image patches, and iteratively executes the following steps: optimization starts with searching the neighborhood of large patches, and then moves to smaller search neighborhoods in a coarse-to-fine strategy. Obtain the coordinate transformation relationship between the ultrasound three-dimensional image coordinate system and the initial medical image coordinate system.

4. The method according to claim 1, characterized in that: By obtaining the coordinate transformation relationship between the end effector coordinate system, the ultrasound 3D image coordinate system, and the initial medical image coordinate system in the electromagnetic tracking and positioning coordinate system, and by also incorporating the robot's own coordinate system into the electromagnetic tracking and positioning coordinate system, all coordinate systems can be expressed in the electromagnetic tracking and positioning coordinate system, thus obtaining a registered and fused image based on the initial medical image and the ultrasound 3D image.

5. The method according to claim 1, characterized in that: The system receives an indication of movement in the surgical area relative to at least one surgical instrument detected by an electromagnetic position sensor, the electromagnetic position sensor being configured to detect the movement in the surgical area relative to at least one surgical instrument, and to automatically control a six-degree-of-freedom robotic arm in response to the received indication, so as to compensate for the movement of the surgical area by maintaining the position of at least one surgical instrument along a planned robot trajectory path.

6. An ultrasound-guided localization system based on liver tumor ablation, characterized in that, This system is applied to an ultrasound positioning method based on liver tumor ablation as described in any one of claims 1 to 5. The system includes an electromagnetic positioning and tracking system and a surgical robot. The surgical robot includes a robotic arm moving trolley module, on which a series robotic arm module is provided. The series robotic arm module includes a six-degree-of-freedom robotic arm and a robotic arm end effector connected to each other. The end part of the robotic arm end effector is an ultrasound scanning head with a puncture needle. The electromagnetic positioning and tracking system includes an electromagnetic tracking system and an electromagnetic position sensor installed on the ultrasound scanning head. The puncture needle holder with the electromagnetic position sensor is fixed to the ultrasound scanning head. The movement of the six-degree-of-freedom robotic arm drives the ultrasound scanning head that operates the robotic arm end effector to obtain real-time ultrasound scanning images. The surgical robot executes the planned robot trajectory path to complete the ultrasound positioning of the puncture needle intervention position assisted by the surgical robot. The electromagnetic tracking system integrates an electromagnetic tracking system coordinate system.

Images