Puncture mechanical arm control method and system based on three-dimensional medical image

Abstract

The invention discloses a puncture mechanical arm control method and system based on a three-dimensional medical image, and the method comprises the following steps: carrying out the three-dimensional visualization processing of a raw format file of a two-dimensional CT image, and obtaining a three-dimensional CT image; acquiring position coordinates of a target object in the three-dimensional CT image before and after puncture under the coordinate system of the three-dimensional CT image; the position coordinates of the target object before and after puncture are converted into a mechanical arm base coordinate system from a coordinate system of the three-dimensional CT image; the to-be-punctured linear motion track of the puncture mechanical arm is calculated; determining a to-be-punctured posture of the tail end of the puncture mechanical arm; and controlling the puncture mechanical arm to execute a puncture action. According to the method, the problem that the finally obtained three-dimensional medical image is insufficient in resolution or definition and the like due to the fact that the three-dimensional medical image is directly acquired through three-dimensional imaging equipment and the imaging quality is possibly limited by the performance parameters of the equipment and the complexity of three-dimensional imaging in the existing main method for acquiring the three-dimensional medical image is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of puncture robot arms, and in particular to a puncture robot arm control method and system based on three-dimensional medical images. Background Art

[0002] In the current advanced medical field, especially in surgical puncture operations, medical robotic arms are increasingly used to reduce the workload of doctors, shorten operation time and improve the success rate of operations. However, before using medical robotic arms for puncture operations, a crucial step is to obtain three-dimensional medical images of patients and plan accurate puncture trajectories based on these images. Three-dimensional medical images can fully and stereoscopically display the internal structure of patients, provide doctors with detailed anatomical information, and are an important basis for formulating surgical plans and determining puncture paths. At present, the main method of obtaining three-dimensional medical images is direct acquisition through three-dimensional imaging devices, such as three-dimensional ultrasonic imaging systems and three-dimensional optical imaging systems. Although these three-dimensional imaging devices perform well in providing stereoscopic visual information, their imaging quality may be restricted by many factors. For example, the performance parameters of the equipment and the complexity of three-dimensional imaging may lead to deficiencies in the resolution or clarity of the final three-dimensional medical images, thereby affecting the accuracy of the robotic arm puncture trajectory planning. Summary of the invention

[0003] In view of the above-mentioned defects, the present invention proposes a puncture robot arm control method and system based on three-dimensional medical images, aiming to solve the problem that the main method of obtaining three-dimensional medical images is direct acquisition through three-dimensional imaging equipment, but the imaging quality may be limited by the performance parameters of the equipment and the complexity of three-dimensional imaging, resulting in the final three-dimensional medical image being insufficient in resolution or clarity, thereby affecting the accuracy of the robot arm puncture trajectory planning.

[0004] To achieve this object, the present invention adopts the following technical solutions:

[0005] A puncture robot arm control method based on three-dimensional medical images comprises the following steps:

[0006] Step S1: Acquire a two-dimensional CT image and obtain its corresponding raw format file;

[0007] Step S2: performing three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image;

[0008] Step S3: establishing a coordinate system of the three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis;

[0009] Step S4: obtaining the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image;

[0010] Step S5: performing coordinate conversion on the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system;

[0011] Step S6: Calculate the movement direction and linear movement trajectory of the puncture robot arm according to the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system;

[0012] Step S7: adjusting the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured;

[0013] Step S8: According to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured, the puncture robot arm is controlled to perform the puncture action.

[0014] Preferably, in step S2, the following sub-steps are specifically included:

[0015] Step S21: using the VTK tool to read the raw format file of the two-dimensional CT image, and load its content into the data object;

[0016] Step S22: using the data pipeline in the VTK tool to process the data stream in the data object to obtain a processed data stream;

[0017] Step S23: Rendering the processed data stream using a graphics rendering engine in the VTK tool to generate a three-dimensional CT image;

[0018] Step S24: Visually display the three-dimensional CT image.

[0019] Preferably, in step S5, the following sub-steps are specifically included:

[0020] Step S51: Using the PNP method to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system

[0021] Step S52: Obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library And Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system

[0022] Step S53: According to and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows:

[0023]

[0024] Step S54: According to and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows:

[0025]

[0026] Step S55: Obtain the internal parameters of the CT scanner, and obtain the transformation matrix from the CT scanner coordinate system to the image coordinate system based on the internal parameters of the CT scanner

[0027] Step S56: According to and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows:

[0028]

[0029] Step S57: Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system

[0030] Step S58: According to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

[0031] Another aspect of the present application provides a puncture robot arm control system based on three-dimensional medical images, the system comprising:

[0032] An acquisition module, used for acquiring two-dimensional CT images;

[0033] The first acquisition module is used to acquire a raw format file of a two-dimensional CT image;

[0034] A three-dimensional visualization processing module is used to perform three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image;

[0035] An establishing module, used for establishing a coordinate system of a three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis;

[0036] A second acquisition module is used to acquire the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image;

[0037] A coordinate conversion module is used to convert the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image, so as to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base;

[0038] A calculation module, used to calculate the movement direction and linear movement trajectory of the puncture robot arm to be punctured according to the position coordinates of the target object before and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base;

[0039] The adjustment and determination module is used to adjust the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured;

[0040] The robot arm control module is used to control the puncture robot arm to perform the puncture action according to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured.

[0041] Preferably, the three-dimensional visualization processing module includes:

[0042] The reading submodule is used to read the raw format file of two-dimensional CT images using the VTK tool;

[0043] The loading submodule is used to load the content of the raw format file of the two-dimensional CT image into the data object;

[0044] The data processing submodule is used to process the data stream in the data object using the data pipeline in the VTK tool to obtain the processed data stream;

[0045] A rendering submodule is used to render the processed data stream using the graphics rendering engine in the VTK tool to generate a three-dimensional CT image;

[0046] The visualization submodule is used to visualize the three-dimensional CT images.

[0047] Preferably, the coordinate conversion module comprises:

[0048] The first calculation submodule is used to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system using the PNP method

[0049] The first acquisition submodule is used to obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library

[0050] The second calculation submodule is used to Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system

[0051] The third calculation submodule is used to calculate and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows:

[0052]

[0053] The fourth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows:

[0054]

[0055] The second acquisition submodule is used to obtain the internal parameters of the CT scanner;

[0056] The third acquisition submodule is used to obtain the conversion matrix from the CT scanner coordinate system to the image coordinate system according to the internal parameters of the CT scanner.

[0057] The fifth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows:

[0058]

[0059] The sixth calculation submodule is used to Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system

[0060] The conversion submodule is used to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

[0061] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0062] In this scheme, a three-dimensional CT image is obtained, and the movement direction and linear motion trajectory of the puncture robot arm to be punctured are planned based on the three-dimensional CT image, and the robot arm is controlled to perform the puncture action according to the puncture movement direction and linear motion trajectory of the robot arm to be punctured. In terms of obtaining a three-dimensional CT image, specifically, a two-dimensional CT image and its corresponding raw format file are first obtained, and then the raw format file of the two-dimensional CT image is three-dimensionally visualized. The raw format of the two-dimensional CT image can effectively retain the original detail information of the two-dimensional CT image, and these detail information is effectively integrated in the three-dimensional visualization process to form a high-quality three-dimensional CT image. Compared with the traditional three-dimensional medical image directly acquired through a three-dimensional imaging device, the imaging quality of the three-dimensional CT image in this scheme does not need to be limited by factors such as the performance parameters of the three-dimensional imaging device and the complexity of the three-dimensional imaging, thereby preventing the problem of insufficient resolution or clarity of the three-dimensional CT image, thereby improving the accuracy of the puncture trajectory planning of the puncture robot arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The invention is a flowchart of the steps of a puncture robot arm control method based on three-dimensional medical images. DETAILED DESCRIPTION

[0064] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0065] A puncture robot arm control method based on three-dimensional medical images comprises the following steps:

[0066] Step S1: Acquire a two-dimensional CT image and obtain its corresponding raw format file;

[0067] Step S2: performing three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image;

[0068] Step S3: establishing a coordinate system of the three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis;

[0069] Step S4: obtaining the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image;

[0070] Step S5: performing coordinate conversion on the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system;

[0071] Step S6: Calculate the movement direction and linear movement trajectory of the puncture robot arm according to the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system;

[0072] Step S7: adjusting the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured;

[0073] Step S8: According to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured, the puncture robot arm is controlled to perform the puncture action.

[0074] This scheme is a puncture robot arm control method based on three-dimensional medical images, such as Figure 1 As shown, the first step is to collect a two-dimensional CT image and obtain its corresponding raw format file. In this embodiment, a CT scanner can be used to collect a two-dimensional CT image. By obtaining the raw format file of the two-dimensional CT image, the original information and details of the two-dimensional CT image can be effectively retained. The second step is to perform three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image. In this embodiment, by performing three-dimensional visualization processing on the raw format file of the two-dimensional CT image, the detail information in the two-dimensional CT image is more clearly displayed, which is conducive to improving the accuracy of the puncture trajectory planning of the puncture robot arm. The third step is to establish a coordinate system of the three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes the origin O, the x-axis, the y-axis and the z-axis. In this embodiment, by establishing the coordinate system of the CT image, it is conducive to better describing the position and direction in the three-dimensional CT image. The fourth step is to obtain the position coordinates of the target object in the three-dimensional CT image before puncture and the position coordinates after puncture in the three-dimensional CT image coordinate system. Specifically, by obtaining the position coordinates of the target object in the three-dimensional CT image before puncture and the position coordinates after puncture in the three-dimensional CT image coordinate system, it is conducive to the subsequent planning of the puncture trajectory of the puncture robot arm. In one embodiment, if the puncture robot arm needs to obtain a coordinate P from the xOy plane in the coordinate system of the three-dimensional CT image, 1 (x1 ,y 1 , 0) penetrates to a coordinate P in the xOz plane 2 (x 2 , 0, z 2 ), therefore, the position coordinates of the target object before puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image are (x 1 ,y 1 , 0), the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image are (x 2 , 0, z 2 ). The fifth step is to perform coordinate conversion on the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image, and obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the robot base. In this embodiment, since the movement operation of the robot is performed in its base coordinate system, it is necessary to convert the position of the target object in the three-dimensional CT image before and after puncture from the three-dimensional CT image coordinate system to the robot base coordinate system. The sixth step is to calculate the movement direction and straight-line trajectory of the puncture robot arm to be punctured based on the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the robot base. Specifically, by calculating the movement direction and straight-line trajectory of the puncture robot arm to be punctured, it is beneficial for the puncture robot arm to perform the puncture task. In one embodiment, let the position coordinates of the target object in the three-dimensional CT image before puncture in the coordinate system of the robot base be (x 3 ,y 3 , z 3 ), the position coordinates of the target object after puncture in the three-dimensional CT image in the robot base coordinate system are (x 4 ,y 4 , z 4 ), calculate the vector The vector represents the direction of motion and the linear motion trajectory of the puncture robot arm to be punctured. The seventh step is to adjust the rotation angle of the end of the puncture robot arm according to the direction of motion of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the direction of motion of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured. In this embodiment, a puncture needle is installed at the end of the puncture robot arm for puncture surgery. By adjusting the direction of the end of the puncture robot arm to be consistent with the direction of motion of the puncture robot arm to be punctured, it is helpful to ensure the accuracy of the robot arm during the puncture process. The eighth step is to control the puncture robot arm to perform the puncture action according to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured. In this embodiment, by accurately planning the motion trajectory and the end posture of the puncture robot arm, it can be ensured that the puncture needle at the end of the robot arm can accurately reach the target position, effectively reducing the error caused by human operation.

[0075] In this scheme, a three-dimensional CT image is obtained, and the movement direction and linear motion trajectory of the puncture robot arm to be punctured are planned based on the three-dimensional CT image, and the robot arm is controlled to perform the puncture action according to the puncture movement direction and linear motion trajectory of the robot arm to be punctured. In terms of obtaining a three-dimensional CT image, specifically, a two-dimensional CT image and its corresponding raw format file are first obtained, and then the raw format file of the two-dimensional CT image is three-dimensionally visualized. The raw format of the two-dimensional CT image can effectively retain the original detail information of the two-dimensional CT image, and these detail information is effectively integrated in the three-dimensional visualization process to form a high-quality three-dimensional CT image. Compared with the traditional three-dimensional medical image directly acquired through a three-dimensional imaging device, the imaging quality of the three-dimensional CT image in this scheme does not need to be limited by factors such as the performance parameters of the three-dimensional imaging device and the complexity of the three-dimensional imaging, thereby preventing the problem of insufficient resolution or clarity of the three-dimensional CT image, thereby improving the accuracy of the puncture trajectory planning of the puncture robot arm.

[0076] Preferably, in step S2, the following sub-steps are specifically included:

[0077] Step S21: using the VTK tool to read the raw format file of the two-dimensional CT image, and load its content into the data object;

[0078] Step S22: using the data pipeline in the VTK tool to process the data stream in the data object to obtain a processed data stream;

[0079] Step S23: Rendering the processed data stream using a graphics rendering engine in the VTK tool to generate a three-dimensional CT image;

[0080] Step S24: Visually display the three-dimensional CT image.

[0081] In this embodiment, the VTK tool is an open source, cross-platform software system that is specifically used for three-dimensional computer graphics, image processing and visualization. In step S21, when reading a raw format file of a two-dimensional CT image, the VTK tool will load its content into a data object, which is a mesh object, so that subsequent data processing is convenient. In step S22, the data pipeline in the VTK tool is a key component that allows a series of processing of the data, specifically converting the data in the data object into a format that can be used for rendering. By processing the data stream in the data object, it is conducive to better visualization and analysis in the future. In step S23, the graphics rendering engine in the VTK tool is built based on OpenGL, which is responsible for drawing the processed data onto the screen. The image rendering engine provides a variety of rendering techniques and algorithms, such as lighting, shadows, texture mapping, etc., to generate a three-dimensional image. In step S24, the three-dimensional CT image is visualized so that the user can intuitively view and analyze the three-dimensional CT image.

[0082] Preferably, in step S5, the following sub-steps are specifically included:

[0083] Step S51: Using the PNP method to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system

[0084] Step S52: Obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library And Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system

[0085] Step S53: According to and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows:

[0086]

[0087] Step S54: According to and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows:

[0088]

[0089] Step S55: Obtain the internal parameters of the CT scanner, and obtain the transformation matrix from the CT scanner coordinate system to the image coordinate system based on the internal parameters of the CT scanner

[0090] Step S56: According to and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows:

[0091]

[0092] Step S57: Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system

[0093] Step S58: According to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

[0094] In this embodiment, in step S51, the PNP method specifically calculates the external parameters of the CT scanner through the three-dimensional coordinates of n points on the known calibration plate and their two-dimensional projection coordinates on the image plane. In step S52, ROS is an open source operating system that provides hardware abstraction, device driver, function library, visualization tool, message communication, and software package management for robots. In the ROS system, the transformation relationship between different coordinate systems can be viewed and managed through the transform library. Therefore, the transformation matrix from the coordinate system of the end of the manipulator to the coordinate system of the base of the manipulator can be obtained from the transform library. In step S58, the transformation matrix from the image coordinate system to the base coordinate system of the manipulator obtained by calculation is conducive to using it to transform the position coordinates of the target object before and after puncture from the coordinate system of the three-dimensional CT image to the coordinate system of the base of the manipulator.

[0095] Another aspect of the present application provides a puncture robot arm control system based on three-dimensional medical images, the system comprising:

[0096] An acquisition module, used for acquiring two-dimensional CT images;

[0097] The first acquisition module is used to acquire a raw format file of a two-dimensional CT image;

[0098] A three-dimensional visualization processing module is used to perform three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image;

[0099] An establishing module, used for establishing a coordinate system of a three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis;

[0100] A second acquisition module is used to acquire the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image;

[0101] A coordinate conversion module is used to convert the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image, so as to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base;

[0102] A calculation module, used to calculate the movement direction and linear movement trajectory of the puncture robot arm to be punctured according to the position coordinates of the target object before and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base;

[0103] The adjustment and determination module is used to adjust the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured;

[0104] The robot arm control module is used to control the puncture robot arm to perform the puncture action according to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured.

[0105] The present invention provides a puncture robot arm control system based on three-dimensional medical images, which realizes the control of the puncture robot arm through the cooperation of the acquisition module, the first acquisition module, the three-dimensional visualization processing module, the establishment module, the second acquisition module, the coordinate conversion module, the calculation module, the adjustment and determination module and the robot arm control module.

[0106] In this scheme, a three-dimensional CT image is obtained, and the movement direction and linear motion trajectory of the puncture robot arm to be punctured are planned based on the three-dimensional CT image, and the robot arm is controlled to perform the puncture action according to the movement direction and linear motion trajectory of the puncture robot arm to be punctured. In terms of obtaining a three-dimensional CT image, specifically, a two-dimensional CT image and its corresponding raw format file are first obtained, and then the raw format file of the two-dimensional CT image is three-dimensionally visualized. The raw format of the two-dimensional CT image can effectively retain the original detail information of the two-dimensional CT image, and these detail information is effectively integrated in the three-dimensional visualization process to form a high-quality three-dimensional CT image. Compared with the traditional three-dimensional medical image acquisition directly through a three-dimensional imaging device, the imaging quality of the three-dimensional CT image in this scheme does not need to be limited by factors such as the performance parameters of the three-dimensional imaging device and the complexity of the three-dimensional imaging, thereby preventing the problem of insufficient resolution or clarity of the three-dimensional CT image, thereby improving the accuracy of the puncture trajectory planning of the puncture robot arm.

[0107] Preferably, the three-dimensional visualization processing module includes:

[0108] The reading submodule is used to read the raw format file of two-dimensional CT images using the VTK tool;

[0109] The loading submodule is used to load the content of the raw format file of the two-dimensional CT image into the data object;

[0110] The data processing submodule is used to process the data stream in the data object using the data pipeline in the VTK tool to obtain the processed data stream;

[0111] A rendering submodule is used to render the processed data stream using the graphics rendering engine in the VTK tool to generate a three-dimensional CT image;

[0112] The visualization submodule is used to visualize the three-dimensional CT images.

[0113] In this embodiment, by setting the loading submodule, it is convenient for subsequent data processing. By setting the data processing submodule, it is convenient for subsequent better visualization and analysis. By setting the rendering submodule, it is convenient for generating three-dimensional CT images. By setting the visualization display submodule, it is convenient for users to intuitively view and analyze three-dimensional CT images.

[0114] Preferably, the coordinate conversion module comprises:

[0115] The first calculation submodule is used to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system using the PNP method

[0116] The first acquisition submodule is used to obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library

[0117] The second calculation submodule is used to Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system

[0118] The third calculation submodule is used to calculate and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows:

[0119]

[0120] The fourth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows:

[0121]

[0122] The second acquisition submodule is used to obtain the internal parameters of the CT scanner;

[0123] The third acquisition submodule is used to obtain the conversion matrix from the CT scanner coordinate system to the image coordinate system according to the internal parameters of the CT scanner.

[0124] The fifth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows:

[0125]

[0126] The sixth calculation submodule is used to Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system

[0127] The conversion submodule is used to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

[0128] In this embodiment, in the first calculation submodule, the PNP method specifically calculates the external parameters of the CT scanner by the three-dimensional coordinates of n points on the known calibration plate and their two-dimensional projection coordinates on the image plane. In the first acquisition submodule, ROS is an open source operating system that provides hardware abstraction, device driver, function library, visualization tool, message communication and software package management for robots. In the ROS system, the transformation relationship between different coordinate systems can be viewed and managed through the transform library. Therefore, the transformation matrix from the coordinate system of the end of the manipulator to the coordinate system of the base of the manipulator can be obtained from the transform library. Through the mutual cooperation of the first calculation submodule, the first acquisition submodule, the second calculation submodule, the third calculation submodule, the fourth calculation submodule, the second acquisition submodule, the third acquisition submodule, the fifth calculation submodule, the sixth calculation submodule and the conversion submodule, the position coordinates of the target object before and after puncture are converted from the coordinate system of the three-dimensional CT image to the coordinate system of the base of the manipulator.

[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0130] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A puncture robot arm control method based on three-dimensional medical images, characterized in that: The following steps are involved: Step S1: Acquire a two-dimensional CT image and obtain its corresponding raw format file; Step S2: performing three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image; Step S3: establishing a coordinate system of the three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis; Step S4: obtaining the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image; Step S5: performing coordinate conversion on the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system; Step S6: Calculate the movement direction and linear movement trajectory of the puncture robot arm according to the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the robot arm base coordinate system; Step S7: adjusting the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured; Step S8: According to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured, the puncture robot arm is controlled to perform the puncture action.

2. The method for controlling a puncture robot arm based on three-dimensional medical images according to claim 1, characterized in that: In step S2, the following sub-steps are specifically included: Step S21: using the VTK tool to read the raw format file of the two-dimensional CT image, and load its content into the data object; Step S22: using the data pipeline in the VTK tool to process the data stream in the data object to obtain a processed data stream; Step S23: Rendering the processed data stream using a graphics rendering engine in the VTK tool to generate a three-dimensional CT image; Step S24: Visually display the three-dimensional CT image.

3. The puncture robot arm control method based on three-dimensional medical images according to claim 1, characterized in that: In step S5, the following sub-steps are specifically included: Step S51: Using the PNP method to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system Step S52: Obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library And Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system Step S53: According to and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows: Step S54: According to and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows: Step S55: Obtain the internal parameters of the CT scanner, and obtain the transformation matrix from the CT scanner coordinate system to the image coordinate system based on the internal parameters of the CT scanner Step S56: According to and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows: Step S57: Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system Step S58: According to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

4. A puncture robot arm control system based on three-dimensional medical images, using the puncture robot arm control method based on three-dimensional medical images as claimed in any one of claims 1 to 3, characterized in that: The system comprises: An acquisition module, used for acquiring two-dimensional CT images; The first acquisition module is used to acquire a raw format file of a two-dimensional CT image; A three-dimensional visualization processing module is used to perform three-dimensional visualization processing on the raw format file of the two-dimensional CT image to obtain a three-dimensional CT image; An establishing module, used for establishing a coordinate system of a three-dimensional CT image, wherein the coordinate system of the three-dimensional CT image includes an origin O, an x-axis, a y-axis, and a z-axis; A second acquisition module is used to acquire the position coordinates of the target object before puncture and the position coordinates of the target object after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image; A coordinate conversion module is used to convert the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the three-dimensional CT image, so as to obtain the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base; A calculation module, used to calculate the movement direction and linear movement trajectory of the puncture robot arm to be punctured according to the position coordinates of the target object before and after puncture in the three-dimensional CT image in the coordinate system of the robot arm base; The adjustment and determination module is used to adjust the rotation angle of the end of the puncture robot arm according to the movement direction of the puncture robot arm to be punctured, so that the direction of the end of the puncture robot arm is consistent with the movement direction of the puncture robot arm to be punctured, so as to determine the posture of the end of the puncture robot arm to be punctured; The robot arm control module is used to control the puncture robot arm to perform the puncture action according to the linear motion trajectory of the puncture robot arm to be punctured and the posture of the end of the puncture robot arm to be punctured.

5. The puncture robot arm control system based on three-dimensional medical images according to claim 4, characterized in that: The three-dimensional visualization processing module includes: The reading submodule is used to read the raw format file of two-dimensional CT images using the VTK tool; The loading submodule is used to load the content of the raw format file of the two-dimensional CT image into the data object; The data processing submodule is used to process the data stream in the data object using the data pipeline in the VTK tool to obtain the processed data stream; A rendering submodule is used to render the processed data stream using the graphics rendering engine in the VTK tool to generate a three-dimensional CT image; The visualization submodule is used to visualize the three-dimensional CT images.

6. The puncture robot arm control system based on three-dimensional medical images according to claim 4, characterized in that: The coordinate conversion module comprises: The first calculation submodule is used to obtain the conversion matrix from the calibration plate coordinate system to the CT scanner coordinate system using the PNP method The first acquisition submodule is used to obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system from the ROS tranform library The second calculation submodule is used to Take the inverse to get the transformation matrix from the robot base coordinate system to the robot end coordinate system The third calculation submodule is used to calculate and Calculate the transformation matrix from the robot end coordinate system to the calibration plate coordinate system The specific calculation formula is as follows: The fourth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the CT scanner coordinate system The specific calculation formula is as follows: The second acquisition submodule is used to obtain the internal parameters of the CT scanner; The third acquisition submodule is used to obtain the conversion matrix from the CT scanner coordinate system to the image coordinate system according to the internal parameters of the CT scanner. The fifth calculation submodule is used to calculate and Calculate the transformation matrix from the robot base coordinate system to the image coordinate system The specific calculation formula is as follows: The sixth computing submodule is used to Take the inverse to get the transformation matrix from the image coordinate system to the robot base coordinate system The conversion submodule is used to The position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system are converted into the position coordinates of the target object before puncture and after puncture in the three-dimensional CT image coordinate system of the robot base.

Images