2-3D image registration algorithm for target surgical area

Abstract

The invention discloses a 2-3D image registration algorithm for CT and X-Ray of a target area, and the algorithm comprises the following steps: S1, shooting preoperative CT image data, segmenting a rigid structure near the target area, and carrying out the CT preprocessing of an image; s2, X-ray images of c-arm at different angles of the corresponding parts of the patient are shot, and meanwhile, c-arm internal reference and pre-processing images are obtained; s3, registering the rigid structures on the X-ray based on the plurality of rigid structures segmented on the CT, and outputting a plurality of rigid registration parameters; and S4, marking a target area on the CT by a doctor, constructing a model based on the plurality of rigid registration parameters, the X-ray and the CT image, and predicting the position of the target area in the operation under the c-arm coordinate system based on the constructed model. According to the invention, the registration efficiency and the registration precision can be improved; the iteration convergence speed is accelerated, and the algorithm application scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of image-guided surgical navigation of mechanized surgical instruments, and in particular to a 2-3D image registration algorithm for a target surgical area. Background Art

[0002] 2-3D refers to 2D-3D conversion technology, which is a technology used to create three-dimensional (3D) images based on two-dimensional (2D) images.

[0003] In the field of surgical navigation, accurate positioning of the target area during surgery is critical. During surgical treatment, clinicians must refer to intraoperative X-ray multi-angle images, but X-ray images are often of limited help to doctors. Experienced doctors often combine 3D preoperative CT images for intraoperative surgical treatment.

[0004] In image-guided surgical navigation technology, registration is still a necessary process for the accuracy of surgical operations and the accurate positioning of the affected area. The existing surgical navigation spatial registration method is to first perform medical scanning imaging with a device with special metal markers together with the patient, and then manually select the markers in the medical image to obtain the coordinates of these markers in the medical image space; then use a registration probe to obtain the coordinates of the above markers in the navigation space, and finally calculate the conversion relationship between the medical image space and the navigation space through the coordinate data of the above two sets of markers, thereby realizing the spatial registration of surgical navigation.

[0005] However, in actual clinical scenarios, the patient and preoperative target area may deform during surgery, and doctors cannot judge based on experience. Real-time registration of X-ray and CT can help doctors re-calibrate the positional relationship between the target area during surgery and before surgery. The implementation of surgical navigation technology depends on the spatial registration of corresponding points in the navigation space and image space, and depends on the accurate selection of the marker, so the accuracy is low.

[0006] Furthermore, the navigation and positioning of the target area during surgery in clinical practice often uses a human brain alignment model constructed through a large amount of surgical experience of clinicians. This traditional empirical method has the following disadvantages: First, the alignment model constructed based on experience consumes a lot of time and is often obtained by doctors who have undergone a large number of clinical surgical training. Second, the alignment model constructed based on experience is prone to objective deviations, which are difficult to detect during surgery and can easily lead to medical negligence. Third, the doctor's clinical experience determines the quality of the surgery. Fourth, different doctors have different control over the surgical risks of the same patient, resulting in some patients failing to receive timely minimally invasive surgical treatment. Therefore, the 2-3D image registration algorithm of CT and X-Ray can effectively avoid the above problems, while guiding doctors to perform surgical treatment quickly, simply, accurately, and with high consistency.

[0007] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0008] The purpose of the present invention is to provide a 2-3D image registration algorithm for a target surgical area, which can solve the problems existing in the background technology.

[0009] To achieve the above object, the present invention provides a 2-3D image registration algorithm for a target surgical area, which is characterized by comprising:

[0010] Step S1: capture preoperative CT image data, segment rigid structures near the target area, and preprocess the CT image;

[0011] Step S2: taking X-ray images of the c-arm at different angles of the corresponding part of the patient, obtaining the c-arm internal reference at the same time, and preprocessing the image;

[0012] Step S3: registering the rigid structures on the X-ray based on the multiple rigid structures segmented on the CT, and outputting multiple rigid registration parameters;

[0013] Step S4: The doctor marks the target area on the CT, builds a model based on multiple rigid registration parameters, X-ray, and CT images, and predicts the position of the target area in the c-arm coordinate system during the operation based on the built model;

[0014] In one or more embodiments, the step S1 further includes:

[0015] Step S11: the patient lies on the bed in a supine position, the corresponding part is photographed, the noise area is segmented, and the noise area is removed;

[0016] Step S12: Segment corresponding rigid regions by the deep learning model, and the regions must be at least two;

[0017] Step S13: preprocessing the CT image, resampling the CT image to normal clinical layer thickness, and converting from CT value to linear attenuation coefficient value;

[0018] Step S14: According to the rigid region, the CT image after the above conversion is cut into a corresponding size.

[0019] In one or more embodiments, step S2 further includes:

[0020] Step S21: taking X-ray images of the corresponding part of the patient at different angles of the c-arm, at least one image;

[0021] Step S22: Obtaining c-arm internal parameters through camera calibration method;

[0022] Step S23: removing image noise and removing areas such as surgical instruments and tools;

[0023] Step S24: segmenting the overexposed area of ​​the image and cropping the image;

[0024] Step S25: Mapping the X-ray image to the attenuation energy value range.

[0025] In one or more embodiments, step S3 includes:

[0026] Step S31: input the pre-processed CT patient data. Generate virtual X-ray images of different scales from the CT image through the relationship between the c-arm radiation source and the detection plate, and generate corresponding virtual X-ray images of different scales from the delineation result of a local rigid structure;

[0027] Step S32: Calculate the similarity between a virtual X-ray image of a certain scale and a real X-ray;

[0028] Step S33: repeatedly calculating the similarity between the virtual X-ray images and the real X-rays at all scales, returning the gradients of all registration parameters, and optimizing the registration parameters by gradient descent;

[0029] Step S34: repeat the above steps S31-S34 until the registration parameters of each rigid structure are obtained;

[0030] In one or more embodiments, step S4 includes:

[0031] Step S41: Select a fixed coordinate system according to the rigid structure registration parameters of each local area, and construct the rotation and translation parameters of each rigid object;

[0032] Step S42: Based on the rotation and translation parameters, the deformation of each spine is calculated, and a model is constructed, which is not limited to a deep learning model, an interpolation model, etc.;

[0033] Step S43: inputting the deformation of the local rigid tissue, the CT image and the target area, and predicting the deformation of the target area;

[0034] Step S44: after obtaining the deformation field of the target area, deform and interpolate the target area to obtain a deformed CT.

[0035] Compared with the prior art, the multiple technical solutions and embodiments provided by the present invention have at least the following technical effects or advantages:

[0036] 1. A multi-scale gradient similarity calculation function can effectively improve the accuracy and stability of registration, accelerate the iteration process, and improve the efficiency of registration;

[0037] 2. Simply use local rigid structure registration to construct a rigid body deformation field, and then learn the deformation field of the target area, which reduces the difficulty of model construction, accelerates the iterative convergence speed, and improves the algorithm's applicable scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and can be considered as illustrative of various combinations of preferred embodiments and do not constitute an improper limitation on the present invention. In the drawings:

[0039] Figure 1 A flow chart of a 2-3D image registration algorithm for CT and X-Ray of a target area provided in the first embodiment of the present invention;

[0040] Figure 2 A schematic diagram of iterative optimization of registration of the method provided in Embodiment 1 of the present invention;

[0041] Figure 3 A schematic diagram of the overall structure of a control system of a robotic arm for a 2-3D image registration algorithm of CT and X-Ray of a target area provided in the second embodiment of the present invention;

[0042] Explanation of the reference numerals: 1-DCO frame, 2-DCO bracket, 3-SCO, 4-front and rear adjustment slide slot, 5-hand screw, 6-telescopic guide rod, 7-binding strap, 8-left and right adjustment slide slot. DETAILED DESCRIPTION

[0043] Unless explicitly stated otherwise, throughout the specification and claims, the term "comprise" or its variations such as "include" or "comprising", "made of", etc. will be understood to include the stated elements or components but not to exclude other elements or other components.

[0044] The purpose of the present invention is to provide a 2-3D image registration algorithm for the target surgical area, so as to improve the surgical accuracy and enable continuous iterative updates.

[0045] Embodiment 1:

[0046] This embodiment provides a 2-3D image registration algorithm for a target surgical area, including:

[0047] Step S1: capture preoperative CT image data, segment rigid structures near the target area, and preprocess the CT image;

[0048] As a preferred implementation of this embodiment, step S1 includes:

[0049] Step S11: the patient lies on the bed in a supine position, and the corresponding lesion area is photographed, the noise area is segmented, and the noise area is removed;

[0050] Step S12: Segment corresponding rigid regions by the deep learning model, and the regions must be at least two;

[0051] Step S13: preprocessing the CT image, resampling the CT image to normal clinical layer thickness, and converting from CT value to linear attenuation coefficient value;

[0052] Specifically, the calculation method for converting CT value to linear attenuation coefficient value is:

[0053] CT_matrix[CT_matrix <lower]=lower;

[0054] [CT_matrix>higher]=higher;

[0055] ac_ct_matrix=max(1+CT_matrix*w / 1000,0);

[0056] Among them, w is the radiation density value based on water; lower and higher are the minimum and maximum values ​​of all local rigid tissue areas and target areas;

[0057] According to all local rigid tissue regions, the CT image after the above transformation is cut into corresponding sizes;

[0058] Step S2: taking X-ray images of the c-arm at different angles of the corresponding part of the patient, obtaining the c-arm internal reference at the same time, and preprocessing the image;

[0059] As a preferred implementation of this embodiment, step S2 includes:

[0060] Step S21: taking X-ray images of the corresponding part of the patient at different angles of the c-arm, at least one image;

[0061] Step S22: Obtaining c-arm internal parameters through camera calibration method;

[0062] Step S2: construct a deep learning segmentation model for drainage area division;

[0063] Step S2 of the embodiment of the present invention includes the following steps:

[0064] Step S21: taking X-ray images of the c-arm at different angles of the corresponding part of the patient, wherein the number of images is not less than 1;

[0065] Step S22: removing image noise and removing areas such as surgical instruments and tools;

[0066] Step S23: segmenting the overexposed area of ​​the image and cropping the image;

[0067] Step S24: mapping the X-ray image to the attenuation energy value range;

[0068] X-ray image mapping to attenuation energy value range calculation method:

[0069] new_v=65535*exp(-v / max(v));

[0070] Step S3: registering the rigid structures on the X-ray based on the multiple rigid structures segmented on the CT, and outputting multiple rigid registration parameters;

[0071] As a preferred implementation of this embodiment, step S3 includes:

[0072] Step S31: input the pre-processed CT patient data. Generate virtual X-ray images of different scales from the CT image through the relationship between the c-arm radiation source and the detection plate, and generate corresponding virtual X-ray images of different scales from the delineation result of a local rigid structure;

[0073] Step S32: Calculate the similarity between a virtual X-ray image of a certain scale and a real X-ray;

[0074] Step S33: repeatedly calculating the similarity between the virtual X-ray images and the real X-rays at all scales, returning the gradients of all registration parameters, and optimizing the registration parameters by gradient descent;

[0075] The multi-scale similarity calculation method is as follows:

[0076]

[0077] L_gcc=abs(ncc(F(fix),F(moving))

[0078]

[0079] Among them, F is the feature obtained by pre-training of the VGG network model; fix and moving are virtual X-ray images and real X-rays respectively.

[0080] Step S34: repeat the above steps S31-S34 until the registration parameters of each rigid structure are obtained;

[0081] Step S4: The doctor marks the target area on the CT, builds a model based on multiple rigid registration parameters, X-ray, and CT images, and predicts the position of the target area in the c-arm coordinate system during the operation based on the built model;

[0082] As a preferred implementation of this embodiment, step S4 includes:

[0083] Step S41: Select a fixed coordinate system according to the rigid structure registration parameters of each local area, and construct the rotation and translation parameters of each rigid object;

[0084] Step S42: Based on the rotation and translation parameters, the deformation of each spine is calculated, and a model is constructed, which is not limited to a deep learning model, an interpolation model, etc.;

[0085] Step S43: inputting the deformation of the local rigid tissue, the CT image and the target area, and predicting the deformation of the target area;

[0086] Step S44: after obtaining the deformation field of the target area, deform and interpolate the target area to obtain a deformed CT;

[0087] In summary, the embodiment of the present invention provides a 2-3D image registration algorithm for CT and X-Ray of the target area, which can perform registration according to X-ray and CT for different scenes and can efficiently guide doctors to perform minimally invasive surgery.

[0088] Embodiment 2:

[0089] This embodiment provides a control system for a robot arm of a 2-3D image registration algorithm for CT and X-Ray of a target area. Based on the same concept as a 2-3D image registration algorithm for CT and X-Ray of a target area, the control system of the robot arm is used to implement the "shooting preoperative CT image data" in step S1, including:

[0090] DCO frame 1, DCO bracket 2, SCO 3, front and rear adjustment slide 4, hand screw 5, telescopic guide rod 6, strap 7 and left and right adjustment slide 8;

[0091] The DCO frame 1 is set at the bottom and is used to carry the filming equipment. First adjust the SCO3 frame, the front and rear adjustment slots 4, the hand screws 5, the telescopic guide rod 6, and the installation kit composed of the straps 7. By adjusting the left and right adjustment slots 8, change the telescopic range of the telescopic guide rod 6 to just match the shape of the flat side shell of the surgical C-arm; tighten the hand screws 5 on both sides, and then connect the straps 7 to the surgical C-arm; adjust the center of SCO3 to coincide with the center of the flat plate of the surgical C-arm through the front and rear adjustment slots 4 and the left and right adjustment slots 8, and the position can be observed by the laser light of the C-arm; then assemble the DCO frame 1 and the DCO bracket 2 into a whole. The DCO bracket 2 is connected to the DCO frame 1 through an axis, so the DCO frame 1 can still rotate after installation; install the DCO component through the quick-release buckle under SCO3; adjust the DCO frame 1 to make it parallel to the lower surface of SCO3 and take the first shot; flip the DCO frame 1 up and down 180° and take the second shot; after the system completes the alignment through the imaging of the two C-arms and the position information of the passive reflective marker balls on the DCO frame 1, remove the DCO component. SCO3 must be installed on the flat plate of the C-arm throughout the entire operation. The position adjustment during use only needs to be adjusted once. It needs to be adjusted when used for the first time. After adjustment, the position is fixed and no adjustment is required for subsequent use. Through this embodiment, it is possible to achieve shooting at any angle and distance in step S1, solving the problem that 2-3D conversion requires shooting materials at different angles.

[0092] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A 2-3D image registration algorithm for CT and X-Ray of a target area, characterized in that: The following steps are involved: Step S1: capture preoperative CT image data, segment rigid structures near the target area, and preprocess the CT image; Step S2: taking X-ray images of the c-arm at different angles of the corresponding part of the patient, obtaining the c-arm internal reference at the same time, and preprocessing the image; Step S3: registering the rigid structures on the X-ray based on the multiple rigid structures segmented on the CT, and outputting multiple rigid registration parameters; Step S4: The doctor marks the target area on the CT, builds a model based on multiple rigid registration parameters, X-ray, and CT images, and predicts the position of the target area in the c-arm coordinate system during the operation based on the built model.

2. The 2-3D image registration algorithm for CT and X-Ray of a target area as claimed in claim 1, characterized in that: The step S1 comprises: Step S11: the patient lies on the bed in a supine position, the corresponding part is photographed, the noise area is segmented, and the noise area is removed; Step S12: Segment corresponding rigid regions by the deep learning model, and the regions are required to be at least two; Step S13: preprocessing the CT image, resampling the CT image to normal clinical layer thickness, and converting from CT value to linear attenuation coefficient value; Step S14: According to the rigid region, the CT image after the above conversion is cut into a corresponding size.

3. The 2-3D image registration algorithm for CT and X-Ray of a target area as claimed in claim 1, characterized in that: The step S2 comprises: Step S21: taking X-ray images of the corresponding part of the patient at different angles of the c-arm, at least one image; Step S22: Obtaining c-arm internal parameters through camera calibration method; Step S23: removing image noise and removing areas such as surgical instruments and tools; Step S24: segmenting the overexposed area of ​​the image and cropping the image; Step S25: Mapping the X-ray image to the attenuation energy value range.

4. The 2-3D image registration algorithm for CT and X-Ray of a target area as claimed in claim 1, characterized in that: The step S3 comprises: Step S31: input the pre-processed CT patient data. Generate virtual X-ray images of different scales from the CT image through the relationship between the c-arm radiation source and the detection plate, and generate corresponding virtual X-ray images of different scales from the delineation result of a local rigid structure; Step S32: Calculate the similarity between a virtual X-ray image of a certain scale and a real X-ray; Step S33: Repeatedly calculate the similarity between the virtual X-ray images and the real X-ray at all scales, and return the gradients of all registration parameters, and optimize the registration parameters by gradient descent.

5. The 2-3D image registration algorithm for CT and X-Ray of a target area as claimed in claim 1, characterized in that: The step S4 comprises: Step S41: Select a fixed coordinate system according to the rigid structure registration parameters of each local area, and construct the rotation and translation parameters of each rigid object; Step S42: Based on the rotation and translation parameters, the deformation of each spine is calculated, and a model is constructed. The model is not limited to a deep learning model, an interpolation model, etc.; Step S43: inputting the deformation of the local rigid tissue, the CT image and the target area, and predicting the deformation of the target area; Step S44: after obtaining the deformation field of the target area, deform and interpolate the target area to obtain a deformed CT.

Images