A step-by-step 2D and 3D registration and positioning method based on RCWHGD

By using the gradient orientation weighted histogram rank correlation method (RCWHGD) for step-by-step optimization, the problems of numerous iterations and small capture range in 2D/3D image registration in existing technologies are solved, achieving efficient and accurate pose localization in surgical navigation systems and improving surgical quality.

CN114565649BActive Publication Date: 2024-11-29CHANGCHUN UNIV OF SCI & TECH +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210187038.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-28
Publication Date
2024-11-29
Estimated Expiration
2042-02-28

AI Technical Summary

Technical Problem

Existing optimization-based 2D/3D image registration techniques for surgical navigation suffer from problems such as numerous iterations, small capture range, high time overhead, reliance on initial pose, and inaccurate cross-modal image similarity measurement.

Method used

We employ the gradient-directed weighted histogram rank correlation method (RCWHGD) for step-by-step optimization. By using parameter space decoupling techniques, we optimize globally first and then locally. We combine the GPU-accelerated SIDDon algorithm with different optimization algorithms to simplify the problem scale and improve the capture range and accuracy.

Benefits of technology

It enables more accurate and efficient pose positioning in surgical navigation systems, reducing surgical trauma, improving surgical quality, and reducing the number of iterations and time costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114565649B_ABST
    Figure CN114565649B_ABST
Patent Text Reader

Abstract

A step-by-step 2D and 3D registration and positioning method based on RCWHGD belongs to the field of surgical navigation technology. The purpose is to use parameter space decoupling technology to solve the problems of large number of iterations and small capture range caused by too large pose parameter space in the prior art. The gradient direction weighted histogram rank correlation method proposed in the present invention uses the properties of the gradient direction weighted histogram and the rank correlation coefficient to perform step-by-step registration of different parameters, which can theoretically simplify the problem scale to d 2 , and the rank correlation coefficient can also handle the intensity relationship between cross-modal images well. The idea of ​​global optimization followed by local optimization is introduced. In the rough registration, the global optimization algorithm is used to ensure a larger capture range and a higher registration success rate, and in the fine registration, the local optimization algorithm is used to ensure high precision. This solves the problems of large number of iterations, long registration time, and small capture range in traditional optimization methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of surgical navigation, and in particular relates to a step-by-step 2D and 3D registration and positioning method based on RCWHGD. Background Art

[0002] With the continuous development of medical conditions and technology, people have put forward new requirements for surgical operations, such as minimally invasive, precise, and digital. In order to solve the above problems, researchers have developed computer-assisted surgical navigation systems for use in surgical operations. As a new cross-disciplinary research field that combines medicine, computer technology, imaging, and automated control, surgical navigation systems can provide assistance to medical staff through preoperative planning, intraoperative navigation, and postoperative evaluation, thereby reducing surgical trauma, improving surgical quality, and alleviating patient pain.

[0003] Obtaining the precise position of the patient's target part in space through intraoperative information is a key issue in surgical navigation. At present, the methods to solve this problem mainly include external marker-based methods and image registration-based methods.

[0004] Among them, the method based on external markers requires artificially adding external markers to assist system positioning, and there are problems such as positioning errors being affected by soft tissue deformation and causing secondary damage to patients. Therefore, many researchers have turned their attention to methods based on image registration.

[0005] Three-dimensional CT volume data contains the fine structural information of the target, but frequent collection of volume data during surgery is often costly and will cause great radiation damage to the patient. Therefore, CT volume data of patients are often collected before surgery to obtain patient tissue structure information, and two-dimensional data such as X-rays collected during surgery are used to obtain the spatial position information of patient tissues. The image registration-based method uses the respective characteristics of preoperative three-dimensional and intraoperative two-dimensional information to calculate the current position of the patient's target part in space, so it is also called 2D / 3D image registration technology.

[0006] The mainstream 2D / 3D image registration mostly adopts optimization-based methods, see Appendix Figure 1 , which is a 2D / 3D registration algorithm process based on optimization to solve the registration problem. The projection module uses digital reconstruction radiography (DRR) technology to input CT data and current posture to generate a projected DRR image; the measurement module receives the DRR image generated by the projection module and calculates the similarity measure between the DRR and the reference image (intraoperative X-ray image); the optimizer module calls the projection and measurement modules to obtain the similarity measure under the current posture by giving the current posture parameters, and predicts the next posture. The whole process is optimized iteratively until the similarity measure reaches the maximum value, and the posture parameters at this time are the registration results.

[0007] The main problems of the current optimization-based 2D / 3D registration technology are: iterative optimization is required throughout the entire process, DRR image generation and similarity measurement are frequently performed, the convergence speed is slow, the time cost is high, and the capture range is small; the optimization result depends on the given initial pose, and when the actual pose deviates greatly, it is easy to cause registration failure; the similarity measurement function between the same modality images is mostly used in the measurement module, which cannot well reflect the similarity between X-rays and DRR. Summary of the invention

[0008] The purpose of the present invention is to propose a step-by-step 2D and 3D registration and positioning method based on RCWHGD, which uses parameter space decoupling technology to solve the problems of large number of iterations and small capture range caused by excessive posture parameter space in the prior art. In the computer-assisted surgical system, more accurate and efficient positioning is achieved to assist medical staff in performing surgery. This reduces surgical trauma, improves surgical quality, and relieves patient pain.

[0009] To achieve the above object, a step-by-step 2D and 3D registration and positioning method based on RCWHGD of the present invention comprises the following steps:

[0010] Step 1: Quantify the spatial pose of the 3D CT model to obtain a set of parameters, including the rotation r around the x-axis x , rotate around the y axis r y , rotate around the z axis r z , translation along the x-axis t x , translation along the y axis t y and a translation t along the z axis z ; R is sampled at a fixed interval x With r y The parameters are sampled at equal intervals to obtain n groups of r x and r y parameter;

[0011] Step 2: According to each group r x and r y Parameters generate a DRR image and obtain n DRR images;

[0012] Step 3: Calculate the RCWHGD between the n DRR images obtained in step 2 and the reference image respectively, obtain n RCWHGDs, and select the one with the highest RCWHGD value corresponding to r x and r y Parameters and r z ,t x ,t y and t z As the starting pose parameter, r z ,t x ,t y and t zThe four pose parameters adopt fixed values;

[0013] Step 4: Set the measurement method to RCWHGD, the optimization method to the DIRECT global optimization algorithm, and the projection method to the GPU-accelerated Siddon algorithm. Perform a round of calculation on the r obtained in step 3. x , r y Parameter-based optimization of 2D / 3D registration, set the optimization range to ±40°, and the termination condition: the maximum number of iterations is 100. The r after registration is obtained x配 , r y配 , r z配 ;

[0014] Step 5: For the t obtained in step 3 x ,t y The parameters are corrected to obtain the corrected t x修 ,t y修 ;

[0015] Step 6: For the t obtained in step 3 z The parameters are corrected to obtain the corrected t z修 ;

[0016] Step 7: According to the above obtained r x配 , r y配 , r z配 , t x修 , t y修 , t z修 ; Set the metric method to mutual information, the optimization method to the downhill simplex local optimization algorithm, the projection method to the GPU-accelerated Siddon algorithm, and finally perform a round of optimization-based 2D / 3D registration to obtain the final pose parameter r x终 , r y终 , r z终 , t x终 , t y终 , t z终 .

[0017] The fixed sampling interval described in step 1 is 10°.

[0018] Step 5 is described in step 3 obtained t x ,t y The parameters are modified as follows: Use the phase correlation method to calculate the relative translation Δx and Δy between the two images in pixels, and calculate the offset according to formula (1) and formula (2) and

[0019]

[0020]

[0021] Where: spacing is the physical spacing of pixels set in the projection module;

[0022] length is the distance from the light source to the projection screen in the projection module;

[0023] According to the offset and t x , t y Corrected, the corrected t x修 , t y修 .

[0024] Step 6: the t obtained in step 3 z The parameters are modified as follows: z The change of leads to the scaling of the DRR content, and the number of non-zero gradient values ​​n of the reference image and the current pose DRR image are calculated respectively fixed , n DRR , calculate the offset according to formula (III)

[0025]

[0026] According to the offset t z Corrected, the corrected t z修 .

[0027] The search range of the registration process described in step 7 is set within 10° and 10 mm.

[0028] The beneficial effects of the present invention are as follows: the present invention proposes an improved step-by-step optimization method based on the traditional optimization-based 2D / 3D alignment technology, and adopts parameter space decoupling technology to solve the shortcomings of the traditional method such as large number of iterations and small capture range caused by the excessively large pose parameter space.

[0029] At present, most of the optimization-based registration methods register all pose parameters simultaneously, and the scale of the optimization problem is the nth power d of the search interval range d n Where n is the number of parameters, usually a six-parameter pose representation is used, then the problem size is d 6 By using the gradient direction weighted histogram rank correlation method proposed in the present invention, the properties of the gradient direction weighted histogram and the rank correlation coefficient are used to align different parameters step by step, which can theoretically simplify the problem size to d 2, and the rank correlation coefficient can also handle the intensity relationship between cross-modal images well. And the idea of ​​global optimization first and local optimization later is introduced. The global optimization algorithm is used in coarse registration to ensure a larger capture range and a higher registration success rate. The local optimization algorithm is used in fine registration to ensure high precision. Thereby solving the problems of large number of iterations, long registration time, and small capture range in traditional optimization methods. The gradient direction weighted histogram rank correlation method adopted in the present invention effectively reduces the size of the search space, thereby greatly reducing the number of iterations. The global-then-local optimization strategy improves the capture range and registration accuracy of the pose parameters, and the rank correlation coefficient can effectively handle the nonlinear intensity relationship between cross-modal images. Verified by the simulation experimental results, our method can achieve more accurate and efficient positioning in the surgical navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is an optimized 2D / 3D registration algorithm process in the prior art;

[0031] Figure 2 is a schematic diagram of the projection module;

[0032] Figure 3 Simplified process flow for 2D / 3D registration algorithm based on optimization;

[0033] Figure 4 A flow chart of a step-by-step 2D and 3D registration and positioning method based on RCWHGD of the present invention

[0034] Among them: 1. Virtual light source, 2. 3D CT digital floating image, 3. DRR image. DETAILED DESCRIPTION

[0035] The embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0036] Concept explanation:

[0037] Pose parameters: a set of parameters obtained by quantifying the spatial pose of a 3D CT model. After specifying the coordinate system, a six-parameter pose representation is usually used: rotation around the x-axis, rotation around the y-axis, rotation around the z-axis, translation along the x-axis, translation along the y-axis, and translation along the z-axis: (r x , r y , r z , t x , t y , t z ), in this article, the units used are ° and mm.

[0038] Reference image and floating image: The names of the targets in the registration task. In 2D / 3D registration, the reference image is a 2D X-ray image and the floating image is a 3D CT volume data.

[0039] Projection module: After the current posture parameters are given, it is responsible for projecting the CT volume data and obtaining the DRR image. The projection algorithm in the present invention adopts the GPU-accelerated Siddon ray tracing method. The principle is as follows Figure 2 The virtual light source 1 projects the 3D CT digital floating image 2 to obtain the DRR image 3; when we set the coordinate system x and y axes parallel to the receiving screen and the z axis perpendicular to the receiving screen, t x , t y Changes in t will result in a translation of the DRR content; z Changes in will result in scaling of the DRR content; z The change of will cause the rotation of DRR content; and r x , r y Changes will result in nonlinear changes in the DRR content.

[0040] Optimized 2D / 3D registration: Figure 1 As shown in Figure 1, it is represented by three modules: projection, measurement, and optimization. The specific process has been introduced in the background technology. Its input is the floating image, reference image, and initial pose, and its output is the predicted pose parameters. The optimization-based 2D / 3D registration is simplified to Figure 3 The flow chart.

[0041] Weighted histogram of gradient directions (WHGD): A global image feature. In order to obtain the weighted histogram of gradient directions of an image, the gradient of each pixel in the image is first calculated to obtain the gradient direction image and the gradient amplitude image. Then, according to the number of histogram bins set in advance, the bins are divided according to the gradient direction of each pixel to draw the histogram. When each pixel is included in the histogram, a fixed weight of 1 is not used, but the gradient amplitude at that point is used for weighting. The characteristics of WHGD are: when the image content is translated, WHGD does not change; when the image content is scaled, WHGD will scale proportionally; when the image content is rotated, WHGD will translate.

[0042] Rank correlation coefficient (RC): An indicator that describes the correlation between two sets of data. N ) and b=(b1,b2,...b N ), first sort the data a and b into descending order, and then sort the data a in a i with b in b i The value of is replaced by its rank (i.e., serial number) in the respective sorting sequence, and r is obtained. a =(ra1 , r a2 ,...r aN ) and r b =(r b1 , r b2 ,...r bN ), remember d i =r ai -r bi , then the rank correlation coefficient ρ between the two sets of data is:

[0043]

[0044] Rank correlation of weighted histogram of gradient directions (RCWHGD): A method within the metric module. After obtaining the WHGD of each of the two images, the histogram is considered as a set of data and the rank correlation coefficient between the WHGD of the two images is calculated. RCWHGD has the property of being invariant to relative translation and scaling of the two images.

[0045] See attached Figure 4 The present invention provides a step-by-step 2D and 3D registration and positioning method based on RCWHGD, comprising the following steps:

[0046] Step 1: Quantify the spatial pose of the 3D CT model to obtain a set of parameters, including the rotation r around the x-axis x , rotate around the y axis r y , rotate around the z axis r z , translation along the x-axis t x , translation along the y axis t y and a translation t along the z axis z ; R is sampled at a fixed interval x With r y The parameters are sampled at equal intervals to obtain n groups of r x and r y parameter;

[0047] Step 2: According to each group r x and r y The parameters generate a DRR image and obtain n DRR images; the DRR image is generated by the GPU-accelerated Siddon ray tracing method. The Siddon algorithm is a CT forward projection algorithm published in a paper published in 1985;

[0048] Step 3: Calculate the RCWHGD between the n DRR images obtained in step 2 and the reference image respectively, obtain n RCWHGDs, and select the one with the highest RCWHGD value corresponding to r x and ry Parameters and r z ,t x ,t y and t z As the starting pose parameter, r z ,t x ,t y and t z The four pose parameters adopt fixed values;

[0049] Rank Correlation of Weighted Histogram of Gradient Directions, RCWHGD Rank Correlation Coefficient (rc) of Weighted Histogram (wh) of Gradient Directions (gd,) r x r y r z are the rotation angles around the xyz axis, in degrees; t x t y t z are the translation distances along the xyz axis, in millimeters; to calculate the spatial registration relationship between the two images, is to find their rotational relationship and translational relationship in three-dimensional space, the xy axis is set parallel to the projection plane, the z axis is perpendicular to the projection plane, and the origin of the coordinate system is set at the projection light source;

[0050] Step 4: Set the measurement method to RCWHGD, the optimization method to the DIRECT global optimization algorithm, and the projection method to the GPU-accelerated Siddon algorithm. Perform a round of calculation on the r obtained in step 3. x , r y Parameter-based optimization of 2D / 3D registration, set the optimization range to ±40°, and the termination condition: the maximum number of iterations is 100. The r after registration is obtained x配 , r y配 , r z配 ; Since the value of RCWHGD depends on r x , r y , r z Three parameters, so in the measurement module, various possible offsets of the reference image WHGD (corresponding to r z The rotation range) is calculated, so that the registration process actually includes the r z However, since this process does not require the generation of additional DRR (the main time-consuming element in the optimization process), the complexity of the process is still controlled within d 2 ;DIRECT algorithm: Divinding Rectangle segmentation algorithm, a global optimization algorithm;

[0051] Step 5: For the t obtained in step 3x ,t y The parameters are corrected to obtain the corrected t x修 ,t y修 ;

[0052] Step 6: t obtained in step 3 based on the geometric relationship z The parameters are corrected to obtain the corrected t z修 ;

[0053] Step 7: According to the above obtained r x配 , r y配 , r z配 , t x修 , t y修 , t z修 ; Set the metric method to mutual information, the optimization method to the downhill simplex local optimization algorithm, the projection method to the GPU-accelerated Siddon algorithm, and finally perform a round of optimization-based 2D / 3D registration to obtain the final pose parameter r x终 , r y终 , r z终 , t x终 , t y终 , t z终 This step performs fine registration for the six parameters. Since each parameter has undergone a rough registration process, the search range can be set within 10° and 10mm. A smaller search range can reduce the search space. After this step, the entire registration process is completed.

[0054] The fixed sampling interval described in step 1 is 10°.

[0055] Step 5 is described in step 3 obtained t x ,t y The parameters are modified as follows: using the phase correlation method, the relative translation Δx and Δy between the 2D image to be registered and the DRR image generated by 3D CT are calculated in pixels, that is, the relative translation Δx and Δy between the reference image and the DRR image currently undergoing pose parameter optimization, and the offset is calculated according to formula (1) and formula (2) and

[0056]

[0057]

[0058] Where: spacing is the physical spacing of pixels set in the projection module;

[0059] length is the distance from the light source to the projection screen in the projection module;

[0060] According to the offset and t x , t y Corrected, the corrected t x修 , t y修 .

[0061] Step 6: the t obtained in step 3 z The parameters are modified as follows: Due to the scaling of the DRR content caused by the change of tz, the number of non-zero gradient values ​​n of the reference image and the current pose DRR image are calculated respectively fixed , n DRR , calculate the offset according to formula (III) The current pose DRR image is the DRR image that needs to be optimized for tz parameters before executing step 6, which is obtained through the previous parameter optimization;

[0062]

[0063] According to the offset t z Corrected, the corrected t z修 .

Claims

1. A step-by-step 2D and 3D registration and positioning method based on RCWHGD, characterized in that: The following steps are involved: Step 1: Quantify the spatial pose of the 3D CT model to obtain a set of parameters, including the rotation r around the x-axis x , rotate around the y axis r y , rotate around the z axis r z , translation along the x-axis t x , translation along the y axis t y and a translation t along the z axis z ; R is sampled at a fixed interval x With r y The parameters are sampled at equal intervals to obtain n groups of r x and r y parameter; Step 2: According to each group r x and r y Parameters generate a DRR image and obtain n DRR images; Step 3: Calculate the RCWHGD between the n DRR images obtained in step 2 and the reference image respectively, obtain n RCWHGDs, and select the highest one among the n RCWHGD values ​​corresponding to r x and r y Parameters and r z ,t x ,t y and t z As the starting pose parameter, r z ,t x ,t y and t z The four pose parameters adopt fixed values; Step 4: Set the measurement method to RCWHGD, the optimization method to the DIRECT global optimization algorithm, and the projection method to the GPU-accelerated Siddon algorithm. Perform a round of calculation on the r obtained in step 3. x , r y Parameter-based optimization of 2D / 3D registration, set the optimization range to ±40°, and the termination condition: the maximum number of iterations is 100. The r after registration is obtained x配 , r y配 , r z配 ; Step 5: For the t obtained in step 3 x ,t y The parameters are corrected to obtain the corrected t x修 ,t y修 ; Step 6: For the t obtained in step 3 z The parameters are corrected to obtain the corrected t z修 ; Step 7: According to the above obtained r x配 , r y配 , r z配 , t x修 , t y修 , t z修 ; Set the metric method to mutual information, the optimization method to the downhill simplex local optimization algorithm, the projection method to the GPU-accelerated Siddon algorithm, and finally perform a round of optimization-based 2D / 3D registration to obtain the final pose parameter r x终 , r y终 , r z终 , t x终 , t y终 , t z终 .

2. The step-by-step 2D and 3D registration and positioning method based on RCWHGD according to claim 1, characterized in that: The fixed sampling interval described in step 1 is 10°.

3. The step-by-step 2D and 3D registration and positioning method based on RCWHGD according to claim 1, characterized in that: Step 5 is described in step 3 obtained t x ,t y The parameters are modified as follows: using the phase correlation method, the relative translation Δx and Δy between the reference image and the DRR image currently being optimized for posture parameters are calculated in pixels, and the offset is calculated according to formula (1) and formula (2) and Where: spacing is the physical spacing of pixels set in the projection module; length is the distance from the light source to the projection screen in the projection module; According to the offset and t x , t y Corrected, the corrected t x修 , t y修 .

4. The step-by-step 2D and 3D registration and positioning method based on RCWHGD according to claim 1, characterized in that: Step 6: the t obtained in step 3 z The parameters are modified as follows: z The change of leads to the scaling of the DRR content, and the number of non-zero gradient values ​​n of the reference image and the current pose DRR image are calculated respectively fixed , n DRR , calculate the offset according to formula (III) According to the offset t z Corrected, the corrected t z修 .

5. A step-by-step 2D and 3D registration and positioning method based on RCWHGD according to any one of claims 1 to 4, characterized in that: The search range of the registration process described in step 7 is set within 10° and 10 mm.

Citation Information

Patent Citations

  • 3D / 2D medical image registration method based on LoG operator

    CN111080681A

  • Image guided radiosurgery method and apparatus using registration of 2D radiographic images with digitally reconstructed radiographs of 3D scan data

    US20050049478A1