A 3D / 2D intraoperative registration method based on contour feature coding

Abstract

The application provides a 3D / 2D intraoperative registration method based on contour feature coding, comprising the following steps: S1, using a segmentation method to remove the soft tissue of a spine CT and generating an X-ray image; S2, extracting a contour on the generated X-ray image and performing feature coding; S3, inputting the extracted contour and the X-ray image into a network and establishing a regression model for predicting correct projection parameters; and S4, obtaining a clear DRR image under a correct posture and aligning to an X-ray in a unified coordinate system to complete registration. The method has higher precision, is more accurate in parameter prediction, and can adapt to the influence caused by different projection angles; in the registration effect, the average registration error effect brought by the method is higher than that of the current advanced registration method.

Description

Technical Field

[0001] This invention relates to the field of intraoperative imaging equipment technology, and in particular to a 3D / 2D intraoperative registration method based on contour feature encoding. Background Technology

[0002] 3D / 2D medical registration is a key technology in radiological imaging and image-guided surgery. By incorporating three-dimensional and two-dimensional coordinates into the same coordinate system, the transformation from preoperative 3D pose to intraoperative pose is identified. The transformed pose is then used to generate a projected image that aligns with the intraoperative 2D image. Commonly used 3D data include computed tomography (CT), cone-beam computed tomography (CBCT), magnetic resonance imaging (MRI), and computer-aided design (CAD) models of medical devices. The 2D images used during surgery are X-ray images, which penetrate the patient's tissue to aid in diagnosis.

[0003] There are two existing methods for 3D / 2D registration: feature extraction-based and intensity calculation-based. In feature-based methods, feature points and key information are extracted from the image. These extracted feature points are then matched with corresponding key points in the target image to generate a correspondence and calculate the positional offset. However, this method is unreliable in cross-dimensional registration, as the calculated feature points shift during projection, failing to maintain a correspondence with the 2D image. In this case, many-to-many point matching needs to account for invalid point matching relationships, increasing the registration difficulty and reducing accuracy. Medical imaging requires precise imaging, therefore intensity calculation-based methods are the mainstream approach. This method uses a search optimization approach to find the optimal pose for 3D volume projection, generating a digitally reconstructed radiograph (DRR) to reduce the grayscale difference between the DRR image and the X-ray image, achieving the registration effect. This method achieves ideal accuracy, but searching for the correct pose in voxel space requires significant computational cost. An evolutionary algorithm is used with an enhanced search strategy to find the pose most similar to the X-ray image, reducing computational complexity. However, during iteration, as the number of iterations increases, it may get trapped in local extrema of image registration, failing to find the correct pose and leading to registration failure. Meanwhile, DRR images need to be generated during the iteration process when searching for the pose for interpolation calculation. Even generating a single DRR image per iteration is very time-consuming, so reducing the number of iterations cannot complete registration within seconds, making it unsuitable for the real-time nature of intraoperative imaging. Furthermore, intensity-based methods also have computational limitations. CT volume capture has constraint range issues, which some methods can optimize, but in practice, they rely heavily on fixed initial poses and specific conditions. In some methods, DRR images are used to simulate X-rays, but in intraoperative image-guided surgery, the radiation source is a low-dose X-ray source, leading to unstable intraoperative X-ray imaging, resulting in blurring, distortion, scattering, and optical distortion. Simultaneously, the unstable grayscale of X-ray imaging makes it difficult for intensity-based similarity calculation methods to converge grayscale differences, leading to registration failure. Even with a wide image capture range and the ability to extract feature points from multiple viewpoints, feature points extracted from blurry images are inaccurate and few in number.

[0004] To overcome the aforementioned challenges, a registration method based on contour feature extraction was chosen to address the issue of unstable grayscale values. Specifically, a residual regressor based on contour encoding was designed. By encoding the contour, the regressor can extract the shape and position information of the spine, while also corresponding to its spatial pose. This avoids getting trapped in local extrema in terms of accuracy and skips the iterative search process in terms of registration time. There is a significant difference between blurry X-ray images and clear DRR images. To better preserve shape information, preprocessing is required to separate the soft tissue of the skeleton and extract the contour of the spine based on this. During training, the regressor reduces parameter residuals to regress the correct pose. Since the computational difficulty of rotation and translation parameters differs (translation parameters are easier to calculate), a branched regressor structure was adopted for the extracted contour shape and position encoding to improve accuracy. Parameters are more accurate under different feature encodings. Furthermore, two loss functions were designed to reduce the losses in shape and position encoding to assist in the two types of feature encoding. Low registration error was achieved in the experiments. Overall, the experimental results ensured both the accuracy and success rate of registration while reducing registration time, thus meeting the requirements for real-time high-precision registration during surgery. Summary of the Invention

[0005] The technical problem addressed by this invention lies in the difficulty of extracting features from blurred X-rays and the difficulty of aligning DRR images in terms of intensity. To solve the above technical problems, this invention provides a 3D / 2D intraoperative registration method based on contour feature encoding, comprising the following steps:

[0006] S1. Use segmentation methods to remove soft tissue from the spine CT and generate X-ray images;

[0007] S2. Extract the contour from the generated X-ray image and encode its features;

[0008] S3. Input the extracted contours and X-ray images into the network and build a regression model to predict the correct rotation and offset parameters;

[0009] S4. Obtain a clear DRR image with the correct orientation and align it to the unified coordinate system of the X-rays to complete the registration.

[0010] Furthermore, after S1, the process includes S11, extracting the contour of bone tissue from the X-ray image for CT image registration.

[0011] Furthermore, S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: saving the contour mask of the extracted X-ray bone tissue as a binary image, and performing feature encoding on the binary image.

[0012] Furthermore, S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: applying Gaussian blur to the synthesized image to obtain a blurred image, thereby simulating the situation of unstable imaging and thus simulating the situation of missing information in low-dose X-ray imaging during surgery.

[0013] Furthermore, to simulate the attenuation effect of X-ray imaging, the Z-axis distance is used as the position of the simulated detector, and the two are proportional to increase or decrease the blur of the image.

[0014] Furthermore, after acquiring the X-ray image and the saved contour, their sizes are converted to be the same to ensure that the contour remains on the original boundary. After the above operations, both are input into the designed contour encoder for feature encoding, and the correspondence between features and spatial parameters is calculated using regression methods.

[0015] Furthermore, during the network training phase, the pose predicted by the regressor corresponds to the features, and each DRR image and X-ray image uses a labeled dataset; the deviation between the true value and the predicted result is used as the loss, defined by the following formula:

[0016]

[0017] Where N represents the total number of samples, Gth represents the actual label data, Pre represents the predicted result, Θ and ξ represent the rotation and offset parameters respectively, and (||) represents the absolute value.

[0018] Furthermore, invariant point matching of the contours corresponding to the projected DRR image and X-ray image is performed. The entire contour is considered as a point set, with points without contours labeled as 0 and points within the contour area labeled as 1. Due to differences in the projection spatial angle, the points on the contour cannot be completely identical. Only points originally on the contour are considered, ignoring some points that have drifted out. The loss is:

[0019]

[0020] C represents a point on the contour, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of marker points;

[0021] To strengthen the correspondence between shape features and spatial rotation, the entire DRR image is encoded with gradient features, allowing the regressor to learn shape features. The feature direction vector deviation is used as the loss. The network can learn the specific shape by reading the feature direction vector, avoiding the influence of low-level semantic ambiguity. The obtained spatial angles lead to the shape correspondence in the generated image.

[0022]

[0023] S represents the gradient vector, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of gradient vectors;

[0024] The final total loss of the network is as follows:

[0025]

[0026] Where, λ l , λ c , λ g L represents the weighting factors for different losses. total This represents its total loss.

[0027] The X-ray image generated by simulating the CT image after projection segmentation is encoded with gradient features. The feature direction vector deviation is used as the loss. The network learns the shape of the contour by reading the gradient vector and calculates the error of the contour shape tensor between the two images. The gradient magnitude and direction of each pixel are expressed by the following formula:

[0028] G x =I(x+1,y)-I(x-1,y)

[0029] G y =I(x,y+1)-I(x,y-1)

[0030]

[0031] Furthermore, among them, G x G represents the gradient of a pixel in the horizontal direction. y G represents the gradient of a pixel in the vertical direction, (x,y) represents the two-dimensional coordinates, and G and D represent the merged tensors in the feature channels.

[0032] Furthermore, the binary image is feature encoded, wherein the encoder includes a contour encoder for extracting features of the contour and features of the X-ray image, and a hybrid encoder; the contour encoder includes a three-layer convolutional module, a ReLU layer and a batch normalization layer arranged sequentially.

[0033] The 3D / 2D intraoperative registration method based on contour feature encoding of this invention has higher accuracy and more accurate parameter prediction, while also being able to adapt to the effects of different projection angles. In terms of registration effect, the average registration error of this method is more accurate than that of current advanced registration methods. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention. The structures of ConEncoder and CerEncoder in the figure are... Figure 5 As described in the text, Xray represents the target image, Contour represents the contour, Ecablock represents the attention module, and GradEncoder represents the gradient vector encoder.

[0036] Figure 2 This is a schematic diagram of the network error during the training process of the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention;

[0037] Figure 3 This is a schematic diagram illustrating the effect of the finely tuned test set of the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention.

[0038] Figure 4 This is a schematic diagram illustrating the process of gradient encoding calculation of the entire image in the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention;

[0039] Figure 5 This is a flowchart illustrating the feature encoding of a binary image in the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention. Figure 5 (a) represents ConEncoder. Figure 5 (b) represents CerEncoder, where K represents the kernel size, S represents the stride, and C represents the number of feature channels;

[0040] Figure 6 A partial schematic diagram of the evaluation results is provided for this invention;

[0041] In the diagram, Θ represents the rotation parameter, ξ represents the translation parameter, ConEncoder and CerEncoder represent two different encoders, and V... x and V c These represent two different feature vectors: the former representing X-rays and the latter representing contour features. The structures of the two encoders will be shown below. The regressed spatial parameters will be applied to the projection module to generate the registered DRR image. label L cont L gradThese represent three different types of losses: label loss, contour loss, and gradient vector loss. Detailed Implementation

[0042] The above-described solution will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. The implementation conditions used in the embodiments may be further adjusted according to the conditions of specific manufacturers, and the implementation conditions not specified are generally those in routine experiments.

[0043] See Figures 1 to 6 As shown, a 3D / 2D intraoperative registration method based on contour feature encoding includes the following steps:

[0044] S1. Use segmentation methods to remove soft tissue from the spine CT and generate X-ray images;

[0045] S2. Extract the contour from the generated X-ray image and encode its features;

[0046] S3. Input the extracted contours and X-ray images into the network and build a regression model to predict the correct rotation and offset parameters;

[0047] S4. Obtain a clear DRR image with the correct orientation and align it to the unified coordinate system of the X-rays to complete the registration.

[0048] Furthermore, after S1, the process includes S11, extracting the contour of bone tissue from the X-ray image for CT image registration.

[0049] Furthermore, S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: saving the extracted contour of bone tissue from the X-ray image as a binary image, and performing feature encoding on the binary image.

[0050] Furthermore, S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: applying Gaussian blur to the synthesized image to obtain a blurred image, thereby simulating the situation of unstable imaging and thus simulating the situation of missing information in low-dose X-ray imaging during surgery.

[0051] Furthermore, after acquiring the blurred image, a filter is used to retain the grayscale values, and a mask is generated at this location. Based on this, the 2D bone tissue contour is extracted and retained for training the network.

[0052] Furthermore, the attenuation coefficient alpha of the X-rays was set to 0.1, and the Z-axis distance was used as the position of the simulated detector. Both factors proportionally increased or decreased the blur of the image.

[0053] Furthermore, during the network training phase, the pose predicted by the regressor corresponds to the features, and each DRR image and X-ray image uses a labeled dataset; the deviation between the true value and the predicted result is used as the loss, which is defined by the following formula:

[0054]

[0055] Where N represents the total number of samples, L represents the actual label data, P represents the predicted result, Θ and ξ represent the rotation and offset parameters respectively, and (||) represents the absolute value.

[0056] Furthermore, to strengthen the correspondence between spatial translation and displacement features, invariant point matching of the contours corresponding to the projected DRR image and X-ray image is adopted. The overall contour is regarded as a point set, with points without contours labeled as 0 and points within the contour range labeled as 1. Due to the difference in projection spatial angle, the points on the contour cannot be completely consistent. Only the points originally on the contour are considered, ignoring the points that drift out. The loss is:

[0057]

[0058] C represents a point on the contour, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of marker points.

[0059] To strengthen the correspondence between shape features and spatial rotation, the entire DRR image is encoded with gradient features, allowing the regressor to learn shape features. The feature direction vector deviation is used as the loss. The network can learn the specific shape by reading the feature direction vector, avoiding the influence of low-level semantic ambiguity. The obtained spatial angles lead to the shape correspondence in the generated image.

[0060]

[0061] S represents the gradient vector, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of gradient vectors;

[0062] The final total loss of the network is as follows:

[0063]

[0064] Where, λ l , λ c , λ g L represents the weighting factors for different losses. total This represents its total loss.

[0065] The X-ray image generated by simulating the CT image after projection segmentation is encoded with gradient features. The feature direction vector deviation is used as the loss. The network learns the shape of the contour by reading the gradient vector and calculates the error of the contour shape tensor between the two images. The gradient magnitude and direction of each pixel are expressed by the following formula:

[0066] G x =I(x+1,y)-I(x-1,y)

[0067] G y =I(x,y+1)-I(x,y-1)

[0068]

[0069] Furthermore, among them, G x G represents the gradient of a pixel in the horizontal direction. y G represents the gradient of a pixel in the vertical direction, (x,y) represents the two-dimensional coordinates, and G and D represent the merged tensors in the feature channels.

[0070] Furthermore, the binary image is feature encoded, wherein the encoder includes a contour encoder for extracting features of the contour and features of the X-ray image, and a hybrid encoder; the contour encoder includes a three-layer convolutional module, a ReLU layer and a batch normalization layer arranged sequentially.

[0071] The experimental results of the 3D / 2D intraoperative registration method based on contour feature encoding of the present invention for registration of blurred images are as follows:

[0072] Features of blurred images are often not clearly represented due to imaging loss, resulting in low registration accuracy. To verify this, CT scans of 10 patients were used as training data. During training, only the BSD dataset was used, without the BCS dataset. These 10 CT scans were projected into DRR images using the same method as before. For the test set of each set, an additional 300 DRR images were used, from which 100 images were randomly selected as test images. Similarly, the BCS dataset was added to the CT scans of these 10 patients, and our network was deployed and inferred in the same environment. Both datasets were trained simultaneously using the same BSD test set. The training loss and test accuracy are recorded in the table below:

[0073]

[0074] It can be observed that during network training, the label error is higher without BCS dataset assistance than with contour assistance. On the Mtre metric, the error with BCS dataset is 2mm smaller than without. Furthermore, according to the Dice coefficient, the error without BCS dataset is 20% higher than with it. Overall, the evaluation results show a difference between the registration results without contour assistance and the correct registration results, indicating that adding contours is essential for handling blurry data. The overall advantage of the contour encoding regressor is reduced registration error, making registration accuracy a key advantage. The following section compares the network with state-of-the-art registration methods, constructing comparative experiments using different network structures and the same dataset.

[0075] To validate our method, we chose OPT-GO among optimization-based methods. Among recent deep learning methods, we selected ResBlock, UCMAN, and PEHL. We conducted experiments on each method under the same environment and dataset. ResBlock uses residual blocks to capture features and then fine-tunes them using real images. However, during training, we found that the residual regressor, using a fully convolutional structure, is prone to overfitting, so we added DropOut. We encountered problems with the mapping angle; the predicted angle deviated significantly from the actual angle, possibly due to different datasets leading to less than ideal performance. Hierarchical regression, while having a larger overall mTRE on fuzzy datasets, showed a smaller mean deviation range compared to our method, with inference time within 1mm being roughly similar. CNN regressors effectively optimized parameters on fuzzy datasets, with network loss decreasing as the resolution increased. UCMAN improved the network structure using residual blocks, enabling cross-domain adaptive regressors and enhancing network robustness. In terms of performance, using the ResBlock method, we found that this network is prone to overfitting, and its prediction performance is not as good as contour encoding. (See reference...) Figure 2 The designed contour encoding achieved better results in addressing the network's training errors.

[0076] refer to Figure 3 After all networks have been fitted, tests are performed. Here, the regressor is trained. Among various methods, some regressors still need fine-tuning. The actual image and the projected image are calculated and used as the fine-tuning loss. Figure 3 In the test set, the results have been fine-tuned. Comparison of overall results shows that the contour encoding regressor achieved the best registration performance.

[0077] The following are supplementary details regarding the experimental environment and the method for synthesizing the simulated dataset: The deep learning platform used in the CER network is PyTorch, specifically version 1.7.1, CUDA version 11.0, and Python version 3.6.15. The hardware environment uses a Tesla P100-PCIE 16G GPU. Differential learning was employed during training. A very small learning rate was initially determined, and then gradually increased until a final learning rate of 0.0001 was set, with the hyperparameter b at 0.02. Based on the training results for the first patient, three different optimizers were compared: Addam, SGD, and Addamw. During training, the data from the first patient was divided into three groups, using BSD and BCS datasets with 2,000 images each. Each group was trained for 400 epochs. The average error and descent rate of each epoch were used as the basis for comparing the optimizers. From the perspective of convergence speed and loss level, the Addam optimizer achieved the best results. Based on this, different batch sizes were tried. Based on the final convergence performance, the batch size was set to 32. Three sets of losses were included in the training calculation. The total training time for each patient was approximately 8 hours, the retained model parameter size was 17.17 MB, and the total inference time was 5.8 m.

[0078] CT scans of 57 different orthopedic patients were used in the experiment. The CT sequence was imaged during interventional surgery, selecting only the necessary spinal segments to ensure the images were within the CT capture line of sight. The orthopedic vertebral segmentation was performed using a supervised method with manually labeled ground-truths. Simultaneously, X-rays were simulated using DRR images generated from the CT scan. The CT scan volume was uniformly 128*128*128 pixels. 1300 DRR images were generated for each patient using projection, with an additional 400 generated for testing. To simulate more realistic X-rays, Gaussian blurring was used to blur the simulated DRR images. Each DRR image was 128*128 pixels. The correct contours could be easily extracted from the segmentation images. After contour extraction using the Sobel algorithm, zero-padding was applied to the 128*128 pixels, and the extracted contours were saved separately. Regarding the projection angle, to enhance the CT scan of a single patient, a normal distribution was used to randomize six parameters for projection, and the projection parameters were saved as a 1*6 array.

[0079] To verify the structural rationality of the overall design, multiple different regressor structures were used in experiments to select the optimal structure and data combination. Five poses from the same patient's CT scan were selected for demonstration. Since CER and CER-FineTune differ in prediction but are difficult to distinguish, the mTRE results of the experiments are presented below. The first type of experiment is CER-Single (single-branch structure), similar to a regular CNN, which directly regresses 6 parameters in the final fully connected layer and uses these parameters to generate the projection. The second type (CER, CER-Finetune, CER-BCS, CER-BSD) are all dual-branch structures, formed by splicing two types of dual-branch structures through regression, following the hierarchical regression in the PEHL method. In the experiment, angles that are difficult to predict were regressed after the task with easy spatial bias was completed. The experiment assumed that the imaging equipment was similar during surgery, with the equipment fixed in position, so image size scaling was not considered. For FineTune, the latter two losses mentioned above were selected to reduce changes in shape and position while enhancing the generalization ability for practical applications. Observing the table above, we can see that in CER networks using both BCS and BSD as training sets, the mTRE of a single-branch structure is approximately 1 mm higher than that of a dual-branch structure. Some evaluation results are shown below. Figure 6 As shown, the branch structure has excellent performance and achieves the lowest possible effect in terms of appearance error.

[0080] The following table shows the evaluation results based on the average of all test sets:

[0081]

[0082] The branch structure is 1mm lower than the ordinary mTRE, and the CER network after finetune performs better, achieving optimal mTRE and DICE indices. After comparing the results with various advanced methods, the contour encoder has achieved a certain advantage in accuracy. Refer to the data in the table below; the six parameters represent the rotation and translation parameters of the X, Y, and Z axes, respectively. By comparing with the true values, it can be found that the contour encoding is closest to the ground truth, achieving better accuracy.

[0083] Evaluation of accuracy compared to state-of-the-art methods:

[0084]

[0085] The above examples are merely illustrative of the technical concept and features of this invention, intended to enable those skilled in the art to understand and implement the invention, and should not be construed as limiting the scope of protection of this invention. All equivalent transformations or modifications made in accordance with the spirit and essence of this invention should be included within the scope of protection of this invention.

Claims

1. A 3D / 2D intraoperative registration method based on contour feature coding, characterized in that, Includes the following steps: S1. Use segmentation methods to remove soft tissue from the spine CT and generate X-ray images; S2. Extract the contour from the generated X-ray image and encode its features; S3. Input the extracted contours and X-ray images into the network and build a regression model to predict the correct rotation and offset parameters; S4. Obtain a clear DRR image with the correct orientation and align it to the unified coordinate system of X-rays to complete the registration. In S3, the X-ray image and the contour image are processed by ConEncoder, and the intermediate layer of ConEncoder outputs the X-ray feature code Vx and the contour feature code Vc, respectively. The contour feature encoding Vc is divided into two paths: one path is fused with the X-ray image feature encoding Vx via ECABlock and then input into CerEncoder, which generates rotation parameters through a fully connected layer; the other path continues to ConEncoder and finally connects to a fully connected layer to generate offset parameters. ECABlock represents the attention module, and ConEncoder and CerEncoder represent two different encoders.

2. The 3D / 2D intraoperative registration method based on contour feature coding of claim 1, wherein: The step after S1 includes S11, extracting the contour of bone tissue from the X-ray image for CT image registration.

3. The 3D / 2D intraoperative registration method based on contour feature coding of claim 2, wherein: S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: saving the contour mask of the extracted X-ray bone tissue as a binary image, and performing feature encoding on the binary image.

4. The 3D / 2D intraoperative registration method based on contour feature coding of claim 3, wherein: S11, extracting the contour of bone tissue from the X-ray image for CT image registration, includes: applying Gaussian blur to the synthesized image to obtain a blurred image, thereby simulating the situation of unstable imaging and thus simulating the situation of missing information in low-dose X-ray imaging during surgery.

5. The 3D / 2D intraoperative registration method based on contour feature coding of claim 4, wherein: To simulate the attenuation effect of X-rays, the Z-axis distance is used as the position of the simulated detector, and the two are proportional to increase or decrease the blur of the image.

6. The 3D / 2D intraoperative registration method based on contour feature coding of claim 5, wherein: After acquiring X-ray images and separately saved contours, they are input into the designed encoder for feature encoding, and the correspondence between features and spatial parameters is calculated using regression methods.

7. The 3D / 2D intraoperative registration method based on contour feature coding of claim 6, wherein: During the training phase of the network, the pose predicted by the regressor corresponds to the features, and each DRR image and X-ray image uses a labeled dataset; the deviation between the true value and the predicted result is used as the loss, which is defined by the following formula: where N represents the total number of samples, Gth represents the true label data, Pre represents the predicted result, Θ, ξ represent rotation and offset parameters, respectively, "||" represents the absolute value, L represents the true label data, and P represents the predicted result.

8. The 3D / 2D intraoperative registration method based on contour feature coding of claim 7, wherein: Invariant point matching of the contours corresponding to the projected DRR image and X-ray image is performed. The entire contour is considered as a point set, with points without contours labeled as 0 and points within the contour area labeled as 1. Due to differences in projection spatial angles, the points on the contour cannot be completely identical. Only points originally on the contour are considered, ignoring some points that have drifted off. The loss is: ， C represents a point on the contour, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of marker points. To strengthen the correspondence between shape features and spatial rotation, the entire DRR image is encoded with gradient features, allowing the regressor to learn shape features. The feature direction vector deviation is used as the loss. The network can learn the specific shape by reading the feature direction vector, avoiding the influence of low-level semantic ambiguity. The resulting spatial angles lead to the following shape correspondence in the generated image: ， S represents the gradient vector, t represents the target image, m represents the motion image, "||.||" represents the Euclidean distance, and N represents the total number of gradient vectors; The final total loss of the network is as follows: Where, λ l , λ c , λ g These represent weighting factors for different losses. This represents its total loss.

9. The 3D / 2D intraoperative registration method based on contour feature coding of claim 8, wherein: The X-ray image generated by simulating the CT image after projection segmentation is encoded with gradient features. The feature direction vector deviation is used as the loss. The network learns the shape of the contour by reading the gradient vector and calculates the error of the contour shape tensor between the two images. The gradient magnitude and direction of each pixel are expressed by the following formula: wherein, represents the gradient of a pixel point in the horizontal direction, represents the gradient of a pixel point in the vertical direction, (x, y) represents a two-dimensional coordinate, and G and D represent the merged tensors in the feature channel.

10. The 3D / 2D intraoperative registration method based on contour feature encoding of claim 9, wherein: The binary image is feature encoded, wherein the encoder includes a contour encoder for extracting features of the contour and features of the X-ray image, and a hybrid encoder; the contour encoder includes a convolutional module, a ReLU layer and a batch normalization layer arranged in sequence.

Images