Vertebral body registration method and device, equipment, storage medium and program product

Abstract

The invention provides a vertebral body registration method and device, equipment, a storage medium and a program product, and relates to the technical field of image registration. According to the vertebral registration method, a preoperative single-vertebral-body three-dimensional CT image and an intra-operative single-vertebral-body two-dimensional X-ray image corresponding to a fractured target single vertebral body are obtained, rigid registration is carried out on the preoperative single-vertebral-body three-dimensional CT image and the intra-operative single-vertebral-body two-dimensional X-ray image, and a rigid registration single-vertebral-body three-dimensional CT image is obtained; the method comprises the following steps: carrying out rigid registration on a preoperative single-vertebral-body three-dimensional CT image and an intra-operative single-vertebral-body two-dimensional X-ray image to realize coarse registration on the preoperative single-vertebral-body three-dimensional CT image and the intra-operative single-vertebral-body two-dimensional X-ray image, further generating an intra-operative single-vertebral-body three-dimensional simulation CT image according to the intra-operative single-vertebral-body two-dimensional X-ray image, and inputting the rigid registration single-vertebral-body three-dimensional CT image and the intra-operative single-vertebral-body three-dimensional simulation CT image into a three- an elastic registration result output by the three-dimensional elastic registration model is obtained, and accurate registration of the preoperative single-vertebral-body three-dimensional CT image and the intraoperative single-vertebral-body two-dimensional X-ray image of the fractured vertebral body is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of image registration, and in particular to a vertebral registration method, device, equipment, storage medium and program product. Background Art

[0002] Surgical navigation technology, also known as Image Guided Surgery (IGS), is an advanced medical technology. Surgical navigation technology combines computer image processing technology with medical imaging technology to achieve preoperative organ or tissue segmentation of patient images, and perform surgical planning and simulation. In surgical navigation, image registration technology plays a key role. During the operation, by taking two-dimensional X-ray fluoroscopic images and using image registration technology, the preoperative three-dimensional CT image is registered with the intraoperative two-dimensional X-ray image to obtain the registration transformation matrix, that is, the spatial transformation parameters, so as to accurately convert the preoperative plan to the actual position of the patient during the operation, thereby guiding the surgical robot or assisting the doctor to perform high-precision surgical operations. At the same time, doctors can use real-time image-guided surgery to better locate the lesion, protect the surrounding healthy tissue, and ensure the success and safety of the operation.

[0003] In the related art, image registration technology mainly adopts 2D / 3D registration algorithms such as rigid registration algorithms, through which the preoperative 3D CT image corresponding to the complete vertebra and the intraoperative 2D X-ray image are registered. However, in some scenarios, the 2D / 3D registration algorithm cannot realize the registration between the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image. Summary of the invention

[0004] The present application provides a vertebral registration method, device, equipment, storage medium and program product to solve the problem in the related art that it is impossible to achieve the registration of a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image through a 2D / 3D registration algorithm.

[0005] In a first aspect, the present application provides a vertebral registration method, comprising: obtaining a preoperative single vertebra three-dimensional CT image and an intraoperative single vertebra two-dimensional X-ray image corresponding to a target single vertebra, wherein the target single vertebra is a fractured vertebra; rigidly registering the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image to obtain a rigidly registered single vertebra three-dimensional CT image; generating an intraoperative single vertebra three-dimensional simulated CT image based on the intraoperative single vertebra two-dimensional X-ray image; inputting the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image into a three-dimensional elastic registration model to obtain an elastic registration result output by the three-dimensional elastic registration model.

[0006] In one possible implementation, the intraoperative single vertebra two-dimensional X-ray image includes an intraoperative single vertebra two-dimensional X-ray anteroposterior image and an intraoperative single vertebra two-dimensional X-ray lateral image. Based on the intraoperative single vertebra two-dimensional X-ray image, an intraoperative single vertebra three-dimensional simulated CT image is generated, including: inputting the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image into a CT image generation model, and obtaining the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model.

[0007] In a possible implementation, the CT image generation model includes a first image generation network, a second image generation network, and a feature fusion decoder. The intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image are input into the CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model, including: inputting the intraoperative single vertebra two-dimensional X-ray anteroposterior image into the first image generation network, and inputting the intraoperative single vertebra two-dimensional X-ray lateral image into the second image generation network, to obtain a first feature map output by the first image generation network, and a second feature map output by the second image generation network; performing feature fusion on the first feature map and the second feature map through the feature fusion decoder to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the feature fusion decoder.

[0008] In one possible implementation, the three-dimensional elastic registration model includes a third image generation network and a spatial transformation network. The rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image are input into the three-dimensional elastic registration model to obtain the elastic registration result output by the three-dimensional elastic registration model, including: inputting the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image into the third image generation network to obtain the vertebral deformation field output by the third image generation network; inputting the vertebral deformation field and the rigidly registered single vertebra three-dimensional CT image into the spatial transformation network to obtain the elastic registration result output by the spatial transformation network.

[0009] In one possible implementation, a preoperative single vertebra three-dimensional CT image and an intraoperative single vertebra two-dimensional X-ray image are rigidly registered to obtain a rigidly registered single vertebra three-dimensional CT image, including: performing perspective projection on the preoperative single vertebra three-dimensional CT image to obtain a two-dimensional simulated radiographic image corresponding to the preoperative single vertebra three-dimensional CT image; inputting the two-dimensional simulated radiographic image and the intraoperative single vertebra two-dimensional X-ray image into a rigid registration model to obtain spatial transformation parameters output by the rigid registration model; and performing spatial transformation processing on the preoperative single vertebra three-dimensional CT image according to the spatial transformation parameters to obtain a rigidly registered single vertebra three-dimensional CT image.

[0010] In a possible implementation, the two-dimensional simulated radiographic image includes a two-dimensional simulated anteroposterior radiographic image and a two-dimensional simulated lateral radiographic image, the rigid body registration model includes a convolutional neural network layer, a dense network layer and a fully connected layer, the convolutional neural network layer includes different convolutional neural networks, the dense network layer includes a first dense network and a second dense network, the two-dimensional simulated radiographic image and the intraoperative single vertebral two-dimensional X-ray image are input into the rigid body registration model, and the spatial transformation parameters output by the rigid body registration model are obtained, including: performing image feature extraction on the two-dimensional simulated anteroposterior radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebral two-dimensional X-ray anteroposterior image and the intraoperative single vertebral two-dimensional X-ray lateral image through a convolutional neural network in the convolutional neural network layer, respectively, to obtain the convolutional neural network. The network outputs a feature vector; the feature vector corresponding to the two-dimensional simulated anteroposterior radiographic image and the feature vector corresponding to the intraoperative single vertebral two-dimensional X-ray anteroposterior image are spliced ​​to obtain a first feature vector, and the first feature vector is input into the first dense network for spatial position feature extraction to obtain a first position feature output by the first dense network; the feature vector corresponding to the two-dimensional simulated lateral radiographic image and the feature vector corresponding to the intraoperative single vertebral two-dimensional X-ray lateral image are spliced ​​to obtain a second feature vector, and the second feature vector is input into the second dense network for spatial position feature extraction to obtain a second position feature output by the second dense network; after splicing the first position feature and the second position feature, they are input into the fully connected layer to obtain the spatial transformation parameters output by the fully connected layer.

[0011] In one possible implementation, the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated anteroposterior radiographic images and intraoperative single vertebrae two-dimensional X-ray anteroposterior images are the same; the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated lateral radiographic images and intraoperative single vertebrae two-dimensional X-ray lateral images are the same.

[0012] In a second aspect, the present application provides a vertebral body registration device, comprising:

[0013] An acquisition module, used to acquire a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image corresponding to a target single vertebra, where the target single vertebra is a fractured vertebra;

[0014] A registration module is used to perform rigid registration on a preoperative single vertebra 3D CT image and an intraoperative single vertebra 2D X-ray image to obtain a rigidly registered single vertebra 3D CT image;

[0015] A generation module, used for generating a three-dimensional simulated CT image of a single vertebra during surgery according to a two-dimensional X-ray image of a single vertebra during surgery;

[0016] The processing module is used to input the rigid registration single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image into the 3D elastic registration model to obtain the elastic registration result output by the 3D elastic registration model.

[0017] In one possible implementation, the intraoperative single vertebra two-dimensional X-ray image includes an intraoperative single vertebra two-dimensional X-ray anteroposterior image and an intraoperative single vertebra two-dimensional X-ray lateral image, and the generation module is specifically used to: input the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image into a CT image generation model to obtain an intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model.

[0018] In one possible implementation, the CT image generation model includes a first image generation network, a second image generation network and a feature fusion decoder, and the generation module is specifically used to: input the intraoperative single vertebra two-dimensional X-ray anteroposterior image into the first image generation network, and input the intraoperative single vertebra two-dimensional X-ray lateral image into the second image generation network, to obtain a first feature map output by the first image generation network, and a second feature map output by the second image generation network; perform feature fusion on the first feature map and the second feature map through the feature fusion decoder, to obtain an intraoperative single vertebra three-dimensional simulated CT image output by the feature fusion decoder.

[0019] In one possible implementation, the three-dimensional elastic registration model includes a third image generation network and a spatial transformation network, and the processing module is specifically used to: input the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image into the third image generation network to obtain the vertebral deformation field output by the third image generation network; input the vertebral deformation field and the rigidly registered single vertebra three-dimensional CT image into the spatial transformation network to obtain the elastic registration result output by the spatial transformation network.

[0020] In one possible implementation, the registration module is specifically used to: perform perspective projection on the preoperative single vertebra three-dimensional CT image to obtain a two-dimensional simulated radiographic image corresponding to the preoperative single vertebra three-dimensional CT image; input the two-dimensional simulated radiographic image and the intraoperative single vertebra two-dimensional X-ray image into a rigid body registration model to obtain spatial transformation parameters output by the rigid body registration model; and perform spatial transformation processing on the preoperative single vertebra three-dimensional CT image according to the spatial transformation parameters to obtain a rigidly registered single vertebra three-dimensional CT image.

[0021] In a possible implementation, the two-dimensional simulated radiographic image includes a two-dimensional simulated anteroposterior radiographic image and a two-dimensional simulated lateral radiographic image, the rigid body registration model includes a convolutional neural network layer, a dense network layer and a fully connected layer, the convolutional neural network layer includes different convolutional neural networks, the dense network layer includes a first dense network and a second dense network, and the registration module is further used to: respectively extract image features of the two-dimensional simulated anteroposterior radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebral two-dimensional X-ray anteroposterior image and the intraoperative single vertebral two-dimensional X-ray lateral image through a convolutional neural network in the convolutional neural network layer to obtain a feature vector output by the convolutional neural network; extract the two-dimensional simulated anteroposterior radiographic image The corresponding feature vector is spliced ​​with the feature vector corresponding to the intraoperative single vertebra two-dimensional X-ray anteroposterior image to obtain a first feature vector, and the first feature vector is input into the first dense network for spatial position feature extraction to obtain a first position feature output by the first dense network; the feature vector corresponding to the two-dimensional simulated lateral radiographic image and the feature vector corresponding to the intraoperative single vertebra two-dimensional X-ray lateral image are spliced ​​to obtain a second feature vector, and the second feature vector is input into the second dense network for spatial position feature extraction to obtain a second position feature output by the second dense network; after the first position feature and the second position feature are spliced, they are input into the fully connected layer to obtain the spatial transformation parameters output by the fully connected layer.

[0022] In one possible implementation, the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated anteroposterior radiographic images and intraoperative single vertebrae two-dimensional X-ray anteroposterior images are the same; the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated lateral radiographic images and intraoperative single vertebrae two-dimensional X-ray lateral images are the same.

[0023] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0024] The memory stores computer executable instructions;

[0025] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0026] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0027] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.

[0028] The vertebral registration method, device, equipment, storage medium and program product provided in the present application obtain the preoperative single vertebral three-dimensional CT image and the intraoperative single vertebral two-dimensional X-ray image corresponding to the target single vertebra of the fracture, and rigidly align the preoperative single vertebral three-dimensional CT image and the intraoperative single vertebral two-dimensional X-ray image to obtain the rigidly registered single vertebral three-dimensional CT image, thereby achieving coarse alignment of the preoperative single vertebral three-dimensional CT image and the intraoperative single vertebral two-dimensional X-ray image, and further generate the intraoperative single vertebral three-dimensional simulated CT image based on the intraoperative single vertebral two-dimensional X-ray image, input the rigidly registered single vertebral three-dimensional CT image and the intraoperative single vertebral three-dimensional simulated CT image into the three-dimensional elastic registration model, obtain the elastic registration result output by the three-dimensional elastic registration model, and achieve accurate alignment of the preoperative single vertebral three-dimensional CT image and the intraoperative single vertebral two-dimensional X-ray image of the fractured vertebra. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0030] Figure 1 The process of the vertebral registration method provided in the embodiment of the present application Figure 1 ;

[0031] Figure 2 A schematic diagram of the structure of the CT image generation model provided in the embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of a three-dimensional elastic registration model provided in an embodiment of the present application;

[0033] Figure 4 The process of the vertebral registration method provided in the embodiment of the present application Figure 2 ;

[0034] Figure 5 A schematic diagram of the structure of the rigid registration model provided in the embodiment of the present application;

[0035] Figure 6 The process of the vertebral registration method provided in the embodiment of the present application Figure 3 ;

[0036] Figure 7 A schematic diagram of the structure of a vertebral registration device provided in an embodiment of the present application;

[0037] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0038] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] The following first explains the technical terms involved in the embodiments of the present application.

[0041] Term 1, rigid image registration: One image is aligned with another image in terms of image structures such as organs, tissues, and lesions through spatial transformations such as rotation and translation.

[0042] Term 2, Digitally Reconstructed Radiographs (DRR): It is a method of converting a three-dimensional image (three-dimensional volume data) into a simulated radiograph on a two-dimensional image plane by perspective projection. DRR uses a computer algorithm to project the tissue structure and anatomical information in the three-dimensional image onto a two-dimensional image to simulate the image produced by X-rays or other radiological imaging techniques. This technology is widely used in the field of medical imaging and can be used for surgical planning, imaging diagnosis, and treatment simulation, helping doctors better understand the patient's anatomical structure and make relevant medical decisions.

[0043] Term 3, CT image generation model (or 2D to 3D reconstruction, referred to as X2CT model): Generating 3D CT images from 2D X-ray images is a challenging task that aims to infer 3D volume data from 2D projection images. Deep learning-based methods mainly use network models such as Generative Adversarial Networks (GAN), Convolutional Neural Networks (CNN), and U-Net networks to directly learn to generate 3D CT images from 2D X-ray images, which is highly efficient.

[0044] Term 4, 3D / 3D elastic registration: Elastic registration is a technique for accurately aligning two 3D image data, and is particularly suitable for processing image differences caused by deformation (such as organ movement, body position change, etc.). The deep learning-based method automatically learns a vertebral deformation field through an end-to-end training model, and elastically transforms the floating image through the vertebral deformation field to achieve accurate alignment with the fixed image.

[0045] The following describes application scenarios applicable to the vertebral registration method provided in the embodiments of the present application.

[0046] The vertebral registration method provided in the embodiment of the present application is applicable to the scenario of vertebral fracture, and specifically, is applicable to the scenario where there is a difference in vertebral deformation due to vertebral fracture.

[0047] Combined with the above scenario, it can be seen that in related technologies, traditional image registration technology usually uses iterative optimization to perform registration, but it takes a long time. With the development of deep learning technology, learning-based image registration technology makes up for the problem that traditional methods cannot be used in real time. By training registration models such as rigid registration models with a large amount of data, the rigid body transformation relationship between vertebrae can be quickly predicted through this registration model during surgery, greatly improving surgical efficiency. However, the existing registration models are mainly aimed at the rigid registration of intact vertebrae. In the case of vertebral fractures, it is impossible to complete the registration of preoperative three-dimensional CT images and intraoperative two-dimensional X-ray images corresponding to the fractured vertebrae only through rigid registration.

[0048] Based on the problems existing in the related art, the embodiment of the present application first uses the undeformed area of ​​the vertebra to perform rigid alignment through a learning-based method to achieve coarse alignment of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image, and then generates the corresponding intraoperative single vertebra three-dimensional simulated CT image through the intraoperative single vertebra two-dimensional X-ray image. Finally, the rigidly registered single vertebra three-dimensional CT image obtained by the coarse alignment is elastically aligned with the intraoperative single vertebra three-dimensional simulated CT image to find the deformation relationship caused by fracture before and during the operation, thereby completing the 2D / 3D fracture alignment of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image.

[0049] It should be noted that the vertebral registration method provided in the embodiment of the present application can be applied to a spinal surgical robot. The following uses the spinal surgical robot as the executor and combines specific embodiments to explain in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0050] Figure 1 The process of the vertebral registration method provided in the embodiment of the present application Figure 1 .like Figure 1 As shown, the specific implementation of the vertebral registration method includes the following steps:

[0051] S101, obtaining a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image corresponding to a target single vertebra, where the target single vertebra is a fractured vertebra.

[0052] It is understandable that, because the target single vertebra is a fractured vertebra, there is a difference in vertebral deformation between the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image corresponding to the target single vertebra.

[0053] In this step, the preoperative single vertebra three-dimensional CT image is obtained in the following manner: a preoperative three-dimensional CT image containing a target single vertebra is obtained, and based on an artificial neural network such as a 3D UNet network, a VNet network, and a Transformer network, image segmentation processing is performed on the preoperative three-dimensional CT image containing the target single vertebra to obtain a vertebral mask image corresponding to each vertebra in the preoperative three-dimensional CT image and a corresponding name for each vertebra, a target single vertebra is selected from each vertebra, and according to the position information of the target single vertebra in the preoperative three-dimensional CT image represented by the vertebral mask image corresponding to the target single vertebra, the preoperative three-dimensional CT image corresponding to the target single vertebra is segmented from the preoperative three-dimensional CT image containing the target single vertebra to obtain the preoperative single vertebra three-dimensional CT image.

[0054] Exemplarily, the vertebral body may be any one of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae and sacrum Sa. When the vertebral body is the cervical vertebrae, each vertebral body may be named C1 to C7; when the vertebral body is the thoracic vertebrae, each vertebral body may be named T1 to T12; when the vertebral body is the lumbar vertebrae, each vertebral body may be named L1 to L5.

[0055] In this step, the intraoperative single vertebra two-dimensional X-ray image is obtained in the following manner: an intraoperative two-dimensional X-ray image containing a target single vertebra is obtained, and based on an artificial neural network such as YoloV5 to YoloV7, the single vertebra image area contained in the intraoperative two-dimensional X-ray image and the vertebral naming corresponding to the single vertebral image are detected, and the single vertebral image corresponding to the target single vertebra is selected, i.e., the intraoperative single vertebra two-dimensional X-ray image.

[0056] Exemplarily, the vertebral naming corresponding to a single vertebral image is similar to that described above and will not be repeated here.

[0057] S102, rigidly registering the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image to obtain a rigidly registered single vertebra three-dimensional CT image.

[0058] In a possible implementation, a preoperative single vertebra 3D CT image and an intraoperative single vertebra 2D X-ray image are input into a rigid registration model for rigid registration, and a rigid registration result output by the rigid registration model is obtained, namely, a rigidly registered single vertebra 3D CT image.

[0059] Exemplarily, the algorithm corresponding to the rigid registration model may be a 2D / 3D registration algorithm.

[0060] It can be understood that in this step, by rigidly registering the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image, a rough registration of the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image is achieved.

[0061] S103, generating a three-dimensional simulated CT image of a single vertebra during surgery according to the two-dimensional X-ray image of the single vertebra during surgery.

[0062] Exemplarily, the intraoperative single vertebra three-dimensional simulated CT image can be used to reflect the target single vertebra status of the patient during surgery.

[0063] In a possible implementation, a two-dimensional X-ray image of a single vertebra during surgery is input into a CT image generation model to obtain a three-dimensional simulated CT image of a single vertebra during surgery output by the CT image generation model.

[0064] Exemplarily, the CT image generation model is used to generate a corresponding three-dimensional CT image from two-dimensional X-ray image information. The CT image generation model can also be described as an X2CT model.

[0065] It should be noted that there is no difference in order between step S103 and step S102.

[0066] S104, inputting the rigidly registered single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image into the 3D elastic registration model to obtain an elastic registration result output by the 3D elastic registration model.

[0067] In one possible implementation method, based on the three-dimensional elastic registration model, the vertebral deformation field between the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image is first obtained, and then the elastic registration result, that is, the elastically registered single vertebra three-dimensional CT image, is obtained through the vertebral deformation field and the rigidly registered single vertebra three-dimensional CT image.

[0068] In an embodiment of the present application, a preoperative single vertebra three-dimensional CT image and an intraoperative single vertebra two-dimensional X-ray image corresponding to a target single vertebra of a fracture are obtained, and the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image are rigidly registered to obtain a rigidly registered single vertebra three-dimensional CT image, thereby achieving a coarse registration of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image, and further based on the intraoperative single vertebra two-dimensional X-ray image, a three-dimensional simulated CT image of the intraoperative single vertebra is generated, and the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image are input into a three-dimensional elastic registration model to obtain an elastic registration result output by the three-dimensional elastic registration model, thereby achieving a precise registration of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image of the fractured vertebra.

[0069] Optionally, in the vertebra registration method provided in the embodiment of the present application, the intraoperative single vertebra two-dimensional X-ray image includes an intraoperative single vertebra two-dimensional X-ray anteroposterior image and an intraoperative single vertebra two-dimensional X-ray lateral image, that is, an intraoperative single vertebra two-dimensional X-ray image under two views, namely, an anteroposterior view and a lateral view. Among them, the intraoperative single vertebra two-dimensional X-ray anteroposterior image can be expressed as X-ray_AP, and the intraoperative single vertebra two-dimensional X-ray lateral image can be expressed as X-ray_LAT.

[0070] Optionally, a possible implementation method of generating an intraoperative single vertebra three-dimensional simulated CT image based on the intraoperative single vertebra two-dimensional X-ray image in step S103 may be: inputting the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image into a CT image generation model, and obtaining the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model.

[0071] Combine the following Figure 2 The CT image generation model provided in the embodiments of the present application is described in detail.

[0072] Figure 2 This is a schematic diagram of the structure of the CT image generation model provided in the embodiment of the present application. Figure 2 As shown, the CT image generation model includes a first image generation network 21, a second image generation network 22 and a feature fusion decoder 23. The first image generation network and the second image generation network can be networks including an encoder-decoder architecture, such as a U-Net network.

[0073] like Figure 2 As shown, the first image generation network and the second image generation network adopt an encoder-decoder architecture to learn the mapping relationship between the input intraoperative single vertebra two-dimensional X-ray image and the intraoperative single vertebra three-dimensional simulated CT image in the feature space, and further fuse the multi-scale features of each layer of the decoder in the first image generation network and the second image generation network through a feature fusion decoder network to reconstruct the intraoperative single vertebra three-dimensional simulated CT image.

[0074] Specifically, the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image are respectively input into the first image generation network and the second image generation network for feature extraction, and the features extracted by the first image generation network and the second image generation network are further fused through a feature fusion decoder to generate an intraoperative single vertebra three-dimensional simulated CT image.

[0075] Exemplarily, the encoder part in the first image generation network and the second image generation network may be a 2D convolution, the decoder part in the first image generation network and the second image generation network may be a 3D convolution, and the feature fusion decoder may be a 3D convolution. The encoder in the first image generation network and the second image generation network is down-sampling, and the decoder is up-sampling.

[0076] It should be noted that the training data set of the CT image generation model can be a synthetic gold standard X-ray image. The gold standard X-ray image can be obtained by performing frontal and lateral projection on the original three-dimensional CT image to obtain a DRR image, and further performing style transfer processing on the DRR image. In the training process of the CT image generation model, the mean square error between the original three-dimensional CT image and the generated three-dimensional CT image is used as the loss function to optimize the model parameters.

[0077] Based on the above Figure 2 The CT image generation model shown, optionally, inputs the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image into the CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model. A possible implementation method can be: input the intraoperative single vertebra two-dimensional X-ray anteroposterior image into the first image generation network, and input the intraoperative single vertebra two-dimensional X-ray lateral image into the second image generation network, to obtain a first feature map output by the first image generation network, and a second feature map output by the second image generation network; perform feature fusion on the first feature map and the second feature map through a feature fusion decoder to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the feature fusion decoder.

[0078] Combine the following Figure 3 The three-dimensional elastic registration model (3D elastic registration model) provided in the embodiment of the present application is described in detail.

[0079] Figure 3 This is a schematic diagram of the structure of the three-dimensional elastic registration model provided in the embodiment of the present application. Figure 3 As shown, the three-dimensional elastic registration model includes a third image generation network and a spatial transformer network (STN). The third image generation network can be a U-Net network.

[0080] Specifically, the vertebral deformation field between the rigidly registered CT image and the intraoperatively generated CT image is obtained by inputting the rigidly registered CT image and the intraoperatively generated CT image into the third image generation network. The vertebral deformation field and the rigid registration CT image are further input into a space transformation network, and the rigid registration CT image is deformed by the space transformation network to obtain an elastic registration result.

[0081] like Figure 3 As shown in , in the three-dimensional elastic registration model, the data loss between the elastic registration result and the intraoperatively generated CT image can be calculated to determine whether the registration of the rigidly registered CT image and the intraoperatively generated CT image is achieved.

[0082] It should be noted that the training data set used to train the 3D elastic registration model is obtained in the following manner: using the original CT image as a reference image, randomly generating a distorted vertebral deformation field, applying the vertebral deformation field to a fixed image, i.e., the original CT image, to obtain a floating image, i.e., the elastic registration result. Specifically, the vertebral deformation field and the original CT image are used as the training input of the 3D elastic registration model to implement the training of the 3D elastic registration model.

[0083] During the training process of the three-dimensional elastic registration model, the model parameters are optimized by calculating the data loss between the elastically registered CT image and the original three-dimensional CT image.

[0084] Based on the above Figure 3 The three-dimensional elastic registration model shown, optionally, in step S104, the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image are input into the three-dimensional elastic registration model to obtain the elastic registration result output by the three-dimensional elastic registration model. A possible implementation method can be: input the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image into the third image generation network to obtain the vertebral deformation field output by the third image generation network; input the vertebral deformation field and the rigidly registered single vertebra three-dimensional CT image into the spatial transformation network to obtain the elastic registration result output by the spatial transformation network.

[0085] Exemplarily, the elastic registration result may be an elastically registered single vertebral three-dimensional CT image.

[0086] It can be understood that by calculating the data loss between the elastic registration result and the intraoperative single vertebra 3D simulated CT image, it can be determined whether the registration between the rigidly registered single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image is achieved.

[0087] Combine the following Figure 4A specific implementation method of rigidly registering the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image in step S102 to obtain the rigidly registered single vertebra 3D CT image is described in detail.

[0088] Figure 4 The process of the vertebral registration method provided in the embodiment of the present application Figure 2 .like Figure 4 As shown, a specific implementation method of rigidly registering a preoperative single vertebra 3D CT image and an intraoperative single vertebra 2D X-ray image to obtain a rigidly registered single vertebra 3D CT image may include the following steps:

[0089] S401 , performing perspective projection on a preoperative single vertebra 3D CT image to obtain a 2D simulated radiological image corresponding to the preoperative single vertebra 3D CT image.

[0090] Exemplarily, the perspective projection may include a frontal perspective projection and a lateral perspective projection.

[0091] In this step, a two-dimensional simulated frontal radiographic image corresponding to the preoperative single vertebrae three-dimensional CT image is obtained by performing an anteroposterior perspective projection on the preoperative single vertebrae three-dimensional CT image; and a two-dimensional simulated lateral radiographic image corresponding to the preoperative single vertebrae three-dimensional CT image is obtained by performing a lateral perspective projection on the preoperative single vertebrae three-dimensional CT image.

[0092] Optionally, the two-dimensional simulated radiographic image may include a two-dimensional simulated anteroposterior radiographic image and a two-dimensional simulated lateral radiographic image.

[0093] Exemplarily, the two-dimensional simulated anteroposterior radiographic image may be represented as DRR_AP, and the two-dimensional simulated lateral radiographic image may be represented as DRR_LAT.

[0094] S402, inputting the two-dimensional simulated radiological image and the intraoperative single vertebra two-dimensional X-ray image into a rigid body registration model to obtain the spatial transformation parameters output by the rigid body registration model.

[0095] Let's first combine Figure 5 The rigid registration model provided in the embodiments of the present application is described in detail.

[0096] Figure 5 This is a schematic diagram of the structure of the rigid registration model provided in the embodiment of the present application. Figure 5 As shown, the rigid registration model includes a convolutional neural network layer, a dense network (DenseNet) layer and a fully connected layer (Fully Connected Layer, referred to as FC). Among them, the convolutional neural network layer includes different convolutional neural networks (Convolutional Neural Network, referred to as CNN), and the dense network layer includes a first dense network and a second dense network.

[0097] Specifically, different convolutional neural networks are used to extract image features of two-dimensional simulated anteroposterior radiographic images, two-dimensional simulated lateral radiographic images, intraoperative single vertebrae two-dimensional X-ray anteroposterior images, and intraoperative single vertebrae two-dimensional X-ray lateral images, respectively; the first dense network is used to extract the spatial position features of single vertebrae in two-dimensional simulated anteroposterior radiographic images and intraoperative single vertebrae two-dimensional X-ray anteroposterior images; the second dense network is used to extract the spatial position features of single vertebrae in two-dimensional simulated lateral radiographic images and intraoperative single vertebrae two-dimensional X-ray lateral images; and the fully connected layer is used to perform feature regression on the extracted spatial position features.

[0098] It should be noted that the training data set of the rigid registration model may be a synthetic gold standard data set, wherein the synthetic gold standard data set may be obtained by performing frontal and lateral projections on the original three-dimensional CT image to obtain a DRR image, and further performing style transfer processing on the DRR image.

[0099] based on Figure 5 The rigid registration model shown, optionally, a possible implementation method of this step can be: respectively extracting image features of the two-dimensional simulated anteroposterior radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebrae two-dimensional X-ray anteroposterior image, and the intraoperative single vertebrae two-dimensional X-ray lateral image through a convolutional neural network layer to obtain a feature vector output by the convolutional neural network; splicing the feature vector corresponding to the two-dimensional simulated anteroposterior radiographic image and the feature vector corresponding to the intraoperative single vertebrae two-dimensional X-ray anteroposterior image to obtain a first feature vector, and inputting the first feature vector into the first dense network for spatial position feature extraction to obtain a first position feature output by the first dense network; splicing the feature vector corresponding to the two-dimensional simulated lateral radiographic image and the feature vector corresponding to the intraoperative single vertebrae two-dimensional X-ray lateral image to obtain a second feature vector, and inputting the second feature vector into the second dense network for spatial position feature extraction to obtain a second position feature output by the second dense network; after splicing the first position feature and the second position feature, input them into the fully connected layer to obtain the spatial transformation parameters output by the fully connected layer.

[0100] Exemplarily, the concatenation of the feature vectors and the concatenation of the first position feature and the second position feature may be implemented based on a concatenation function such as a Concat function.

[0101] It should be noted that in the vertebral registration method provided in the embodiment of the present application, in the rigid registration model, the principle of binocular vision is utilized to splice the position features output under two views to achieve mutual combination of view position features, thereby improving the accuracy of three-dimensional space position parameter extraction.

[0102] Exemplarily, the spatial transformation parameters may be parameters of rigid body transformation in three-dimensional space. Specifically, the spatial transformation parameters may be six degrees of freedom parameters of rotation and translation in X, Y, and Z dimensions.

[0103] For example, the spatial transformation parameter can be expressed as T delta .

[0104] Optionally, the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated anteroposterior radiographic images and intraoperative single vertebrae two-dimensional X-ray anteroposterior images are the same; the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated lateral radiographic images and intraoperative single vertebrae two-dimensional X-ray lateral images are the same.

[0105] It can be understood that in the rigid registration model, the two-dimensional simulated radiographic image and the intraoperative single vertebral X-ray image under the same view can share the model weight parameters of the same convolutional neural network.

[0106] S403, performing spatial transformation processing on the preoperative single vertebra 3D CT image according to the spatial transformation parameters to obtain a rigidly registered single vertebra 3D CT image.

[0107] Exemplarily, the spatial transformation may be a spatial transformation operation such as rotation and / or translation in the X, Y, and Z dimensions.

[0108] In an embodiment of the present application, a two-dimensional simulated radiographic image corresponding to the preoperative single vertebrae three-dimensional CT image is obtained by perspective projection of the preoperative single vertebrae three-dimensional CT image; the two-dimensional simulated radiographic image and the intraoperative single vertebrae two-dimensional X-ray image are input into a rigid body registration model to obtain spatial transformation parameters output by the rigid body registration model; according to the spatial transformation parameters, the preoperative single vertebrae three-dimensional CT image is spatially transformed to obtain a rigidly registered single vertebrae three-dimensional CT image, thereby achieving coarse registration of the preoperative single vertebrae three-dimensional CT image and the intraoperative single vertebrae two-dimensional X-ray image.

[0109] In summary, the following combination Figure 6 The specific implementation of the vertebral registration method provided in the embodiment of the present application is described in detail.

[0110] Figure 6 The process of the vertebral registration method provided in the embodiment of the present application Figure 3 .like Figure 6As shown, the specific implementation method of the vertebral registration method can be: first, perform image segmentation processing on the preoperative three-dimensional CT image containing the target single vertebra to obtain the preoperative single vertebra three-dimensional CT image corresponding to the target single vertebra, and detect the intraoperative two-dimensional X-ray image containing the target single vertebra to obtain the intraoperative single vertebra two-dimensional X-ray image corresponding to the target single vertebra; secondly, on the one hand, perform perspective projection on the preoperative single vertebra three-dimensional CT image to obtain a two-dimensional simulated radiographic image corresponding to the preoperative single vertebra three-dimensional CT image, and input the two-dimensional simulated radiographic image and the intraoperative single vertebra two-dimensional X-ray image into the rigid registration model to obtain the spatial transformation parameters output by the rigid registration model, and further perform spatial transformation on the preoperative single vertebra three-dimensional CT image based on the spatial transformation parameters to obtain the rigidly registered single vertebra three-dimensional CT image On the other hand, the intraoperative single vertebra three-dimensional CT image is input into the CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model, thereby realizing the generation of the intraoperative single vertebra three-dimensional simulated CT image based on the intraoperative single vertebra two-dimensional X-ray image; finally, the intraoperative single vertebra three-dimensional simulated CT image and the rigidly registered single vertebra three-dimensional CT image are input into the third image generation network of the three-dimensional elastic registration model to obtain the vertebral deformation field, and further based on the vertebral deformation field, the rigidly registered single vertebra three-dimensional CT image is elastically transformed to obtain the elastic registration result output by the three-dimensional elastic registration model, thereby realizing the registration of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image.

[0111] The specific implementation method of the vertebral body registration process is similar to the above and will not be repeated here.

[0112] It can be understood that the vertebral registration method provided in the embodiment of the present application can effectively avoid the influence of vertebral fracture on 2D / 3D registration, and provide strong support for deep learning-based image registration technology.

[0113] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0114] Figure 7 This is a schematic diagram of the structure of the vertebral registration device provided in the embodiment of the present application. Figure 7 As shown, the vertebral registration device 70 includes: an acquisition module 710 , a registration module 720 , a generation module 730 and a processing module 740 .

[0115] The acquisition module 710 is used to acquire a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image corresponding to a target single vertebra, where the target single vertebra is a fractured vertebra;

[0116] A registration module 720 is used to perform rigid registration on the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image to obtain a rigidly registered single vertebra 3D CT image;

[0117] A generating module 730 is used to generate a three-dimensional simulated CT image of a single vertebra during surgery based on the two-dimensional X-ray image of the single vertebra during surgery;

[0118] The processing module 740 is used to input the rigidly registered single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image into the 3D elastic registration model to obtain the elastic registration result output by the 3D elastic registration model.

[0119] In one possible implementation, the intraoperative single vertebra two-dimensional X-ray image includes an intraoperative single vertebra two-dimensional X-ray anteroposterior image and an intraoperative single vertebra two-dimensional X-ray lateral image, and the generation module 730 is specifically used to: input the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image into the CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model.

[0120] In one possible implementation, the CT image generation model includes a first image generation network, a second image generation network and a feature fusion decoder, and the generation module 730 is specifically used to: input the intraoperative single vertebra two-dimensional X-ray anteroposterior image into the first image generation network, and input the intraoperative single vertebra two-dimensional X-ray lateral image into the second image generation network, to obtain a first feature map output by the first image generation network, and a second feature map output by the second image generation network; perform feature fusion on the first feature map and the second feature map through the feature fusion decoder, to obtain an intraoperative single vertebra three-dimensional simulated CT image output by the feature fusion decoder.

[0121] In one possible implementation, the three-dimensional elastic registration model includes a third image generation network and a spatial transformation network, and the processing module 740 is specifically used to: input the rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image into the third image generation network to obtain the vertebral deformation field output by the third image generation network; input the vertebral deformation field and the rigidly registered single vertebra three-dimensional CT image into the spatial transformation network to obtain the elastic registration result output by the spatial transformation network.

[0122] In one possible implementation, the registration module 720 is specifically used to: perform perspective projection on the preoperative single vertebra three-dimensional CT image to obtain a two-dimensional simulated radiographic image corresponding to the preoperative single vertebra three-dimensional CT image; input the two-dimensional simulated radiographic image and the intraoperative single vertebra two-dimensional X-ray image into a rigid body registration model to obtain spatial transformation parameters output by the rigid body registration model; and perform spatial transformation processing on the preoperative single vertebra three-dimensional CT image according to the spatial transformation parameters to obtain a rigidly registered single vertebra three-dimensional CT image.

[0123] In a possible implementation, the two-dimensional simulated radiographic image includes a two-dimensional simulated anteroposterior radiographic image and a two-dimensional simulated lateral radiographic image, the rigid body registration model includes a convolutional neural network layer, a dense network layer and a fully connected layer, the convolutional neural network layer includes different convolutional neural networks, the dense network layer includes a first dense network and a second dense network, and the registration module 720 is further used to: respectively extract image features of the two-dimensional simulated anteroposterior radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebral two-dimensional X-ray anteroposterior image and the intraoperative single vertebral two-dimensional X-ray lateral image through a convolutional neural network in the convolutional neural network layer to obtain a feature vector output by the convolutional neural network; extract the two-dimensional simulated anteroposterior radiographic image from the two-dimensional simulated lateral radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebral two-dimensional X-ray anteroposterior image and the intraoperative single vertebral two-dimensional X-ray lateral image, and obtain a feature vector output by the convolutional neural network; extract the two-dimensional simulated anteroposterior radiographic image from the two-dimensional simulated anteroposterior radiographic image; and extract the feature vector of the two-dimensional simulated anteroposterior radiographic image from the two-dimensional simulated lateral radiographic image. The feature vector corresponding to the image and the feature vector corresponding to the intraoperative single vertebra two-dimensional X-ray anteroposterior image are spliced ​​to obtain a first feature vector, and the first feature vector is input into a first dense network for spatial position feature extraction to obtain a first position feature output by the first dense network; the feature vector corresponding to the two-dimensional simulated lateral radiographic image and the feature vector corresponding to the intraoperative single vertebra two-dimensional X-ray lateral image are spliced ​​to obtain a second feature vector, and the second feature vector is input into a second dense network for spatial position feature extraction to obtain a second position feature output by the second dense network; after splicing the first position feature and the second position feature, they are input into a fully connected layer to obtain the spatial transformation parameters output by the fully connected layer.

[0124] In one possible implementation, the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated anteroposterior radiographic images and intraoperative single vertebrae two-dimensional X-ray anteroposterior images are the same; the model weight parameters of the convolutional neural network used to extract image features from two-dimensional simulated lateral radiographic images and intraoperative single vertebrae two-dimensional X-ray lateral images are the same.

[0125] The vertebral registration device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0126] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 80 includes: at least one processor 801 and a memory 802 .

[0127] Optionally, the electronic device 80 further includes a communication interface 803 . The processor 801 , the memory 802 and the communication interface 803 are connected via a system bus 804 .

[0128] In a specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above method.

[0129] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0130] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0131] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0132] The system bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0133] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0134] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0135] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0137] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0140] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0141] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0142] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A vertebral body registration method, characterized in that: include: Acquire a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image corresponding to a target single vertebra, wherein the target single vertebra is a fractured vertebra; Performing rigid registration on the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image to obtain a rigidly registered single vertebra three-dimensional CT image; generating a three-dimensional simulated CT image of a single vertebra during surgery according to the two-dimensional X-ray image of the single vertebra during surgery; The rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image are input into a three-dimensional elastic registration model to obtain an elastic registration result output by the three-dimensional elastic registration model.

2. The vertebral body registration method according to claim 1, characterized in that: The intraoperative single vertebra two-dimensional X-ray image includes an intraoperative single vertebra two-dimensional X-ray anteroposterior image and an intraoperative single vertebra two-dimensional X-ray lateral image. The generating of the intraoperative single vertebra three-dimensional simulated CT image according to the intraoperative single vertebra two-dimensional X-ray image includes: The intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image are input into a CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model.

3. The vertebral body registration method according to claim 2, characterized in that: The CT image generation model includes a first image generation network, a second image generation network and a feature fusion decoder, and the intraoperative single vertebra two-dimensional X-ray anteroposterior image and the intraoperative single vertebra two-dimensional X-ray lateral image are input into the CT image generation model to obtain the intraoperative single vertebra three-dimensional simulated CT image output by the CT image generation model, including: Inputting the intraoperative single vertebra two-dimensional X-ray anteroposterior image into the first image generation network, and inputting the intraoperative single vertebra two-dimensional X-ray lateral image into the second image generation network, respectively, to obtain a first feature map output by the first image generation network, and a second feature map output by the second image generation network; The feature fusion decoder performs feature fusion on the first feature map and the second feature map to obtain the intraoperative single vertebral three-dimensional simulated CT image output by the feature fusion decoder.

4. The vertebral body registration method according to claim 1, characterized in that: The three-dimensional elastic registration model includes a third image generation network and a space transformation network. The rigidly registered single vertebra three-dimensional CT image and the intraoperative single vertebra three-dimensional simulated CT image are input into the three-dimensional elastic registration model to obtain the elastic registration result output by the three-dimensional elastic registration model, including: Inputting the rigidly registered single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image into the third image generation network to obtain a vertebra deformation field output by the third image generation network; The vertebral deformation field and the rigidly registered single vertebral three-dimensional CT image are input into the space transformation network to obtain the elastic registration result output by the space transformation network.

5. The vertebral body registration method according to claim 1, characterized in that: The rigid registration of the preoperative single vertebra three-dimensional CT image and the intraoperative single vertebra two-dimensional X-ray image to obtain a rigidly registered single vertebra three-dimensional CT image includes: Performing perspective projection on the preoperative single vertebra three-dimensional CT image to obtain a two-dimensional simulated radiological image corresponding to the preoperative single vertebra three-dimensional CT image; Inputting the two-dimensional simulated radiographic image and the intraoperative single vertebra two-dimensional X-ray image into a rigid body registration model to obtain spatial transformation parameters output by the rigid body registration model; According to the spatial transformation parameters, the preoperative single vertebra three-dimensional CT image is spatially transformed to obtain the rigidly registered single vertebra three-dimensional CT image.

6. The vertebral body registration method according to claim 5, characterized in that: The two-dimensional simulated radiographic image includes a two-dimensional simulated anteroposterior radiographic image and a two-dimensional simulated lateral radiographic image, the rigid body registration model includes a convolutional neural network layer, a dense network layer and a fully connected layer, the convolutional neural network layer includes different convolutional neural networks, the dense network layer includes a first dense network and a second dense network, the two-dimensional simulated radiographic image and the intraoperative single vertebral two-dimensional X-ray image are input into the rigid body registration model, and the spatial transformation parameters output by the rigid body registration model are obtained, including: Extracting image features from the two-dimensional simulated anteroposterior radiographic image, the two-dimensional simulated lateral radiographic image, the intraoperative single vertebrae two-dimensional X-ray anteroposterior image, and the intraoperative single vertebrae two-dimensional X-ray lateral image through a convolutional neural network in the convolutional neural network layer, respectively, to obtain a feature vector output by the convolutional neural network; splicing the feature vector corresponding to the two-dimensional simulated anteroposterior radiographic image and the feature vector corresponding to the intraoperative single vertebral two-dimensional X-ray anteroposterior image to obtain a first feature vector, and inputting the first feature vector into the first dense network to extract spatial position features, thereby obtaining a first position feature output by the first dense network; splicing the feature vector corresponding to the two-dimensional simulated lateral radiographic image and the feature vector corresponding to the intraoperative single vertebral two-dimensional X-ray lateral image to obtain a second feature vector, and inputting the second feature vector into the second dense network to extract spatial position features, thereby obtaining a second position feature output by the second dense network; After the first position feature and the second position feature are concatenated, the concatenated features are input into the fully connected layer to obtain the spatial transformation parameters output by the fully connected layer.

7. The vertebral body registration method according to claim 6, characterized in that: The model weight parameters of the convolutional neural network used for extracting image features from the two-dimensional simulated anteroposterior radiographic image and the intraoperative single vertebral two-dimensional X-ray anteroposterior image are the same; The model weight parameters of the convolutional neural network used for extracting image features from the two-dimensional simulated lateral radiographic image and the intraoperative single vertebrae two-dimensional X-ray lateral image are the same.

8. A vertebral body registration device, characterized in that: include: An acquisition module, used to acquire a preoperative single vertebral three-dimensional CT image and an intraoperative single vertebral two-dimensional X-ray image corresponding to a target single vertebra, wherein the target single vertebra is a fractured vertebra; A registration module, used for rigidly registering the preoperative single vertebra 3D CT image and the intraoperative single vertebra 2D X-ray image to obtain a rigidly registered single vertebra 3D CT image; A generating module, used for generating a three-dimensional simulated CT image of a single vertebra during surgery according to the two-dimensional X-ray image of the single vertebra during surgery; The processing module is used to input the rigidly registered single vertebra 3D CT image and the intraoperative single vertebra 3D simulated CT image into a 3D elastic registration model to obtain an elastic registration result output by the 3D elastic registration model.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed.

Images