Vertebral section image registration method and device, computer equipment and storage medium

Abstract

The invention relates to a vertebral segment image registration method and device, computer equipment and a storage medium. The method comprises the steps of obtaining a coarse registration transformation relation between a target vertebral segment segmentation result and a to-be-registered vertebral segment segmentation result according to the to-be-registered sagittal plane contour data of the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result; performing pixel alignment on the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result after coarse registration through a pixel alignment model, and setting a model evaluation standard, the model evaluation standard optimizes the coarse registration transformation relation according to the target vertebral section area loss value and the corresponding target vertebral section area weight of the pixel alignment model and the non-target vertebral section area loss value and the corresponding non-target vertebral section area weight of the pixel alignment model so as to perform fine registration on the preoperative three-dimensional image and the intraoperative three-dimensional image. According to the method, the global optimal solution can be quickly converged in the registration process, and the registration precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the medical imaging field, in particular to a vertebral segment image registration method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of medical imaging and computer technology, 3D image registration technology is more and more applied in the process of doctors making surgical plans and implementing surgeries. For example, in orthopedic surgery, CT images of the target object's surgical site are scanned on the same bed to plan the surgery, and CBCT images of the target object's surgical site are 3D registered in the surgery to complete the robot navigation surgery. Due to the uncertainty of the images and the limitations of the registration algorithm itself and other factors, the traditional registration algorithm has the problem of falling into a local optimal solution, and it is usually difficult to quickly converge to a global optimal solution to achieve the best registration effect. SUMMARY

[0003] Therefore, it is necessary to provide a vertebral segment image registration method, device, computer equipment and storage medium which can solve the problem of falling into a local optimal solution in the 3D vertebral segment image registration process, quickly converge to a global optimal solution and improve the registration accuracy.

[0004] In a first aspect, the present application provides a vertebral segment image registration method, comprising:

[0005] segmenting the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebral segment segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebral segment segmentation result corresponding to the intraoperative three-dimensional image; wherein the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result are segmentation images of the same vertebral segment;

[0006] coarsely registering the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result according to target sagittal plane contour data of the target vertebral segment segmentation result and to-be-registered sagittal plane contour data of the to-be-registered vertebral segment segmentation result, to obtain a coarse registration transformation relationship between the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result;

[0007] aligning the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result through a pixel alignment model, and setting a model evaluation standard, the model evaluation standard optimizing the coarse registration transformation relationship according to a target vertebral segment region loss value and a corresponding target vertebral segment region weight of the pixel alignment model and a non-target vertebral segment region loss value and a corresponding non-target vertebral segment region weight, to obtain a fine registration transformation relationship between the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result;

[0008] registering the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship.

[0009] In one of the embodiments, before the target vertebra segmentation result is coarsely registered with the vertebra segmentation result to be registered according to the target sagittal plane profile data of the target vertebra segmentation result and the to-be-registered sagittal plane profile data of the vertebra segmentation result to be registered, the method further comprises:

[0010] generating a target vertebra bounding box according to the target vertebra segmentation result, and performing clipping on the preoperative three-dimensional image according to the target vertebra bounding box to obtain a target clipped image;

[0011] generating a to-be-registered vertebra bounding box according to the vertebra segmentation result to be registered, and performing clipping on the intraoperative three-dimensional image according to the to-be-registered vertebra bounding box to obtain a to-be-registered clipped image;

[0012] coarsely registering the target clipped image with the to-be-registered clipped image according to the target sagittal plane profile data of the target clipped image and the to-be-registered sagittal plane profile data of the to-be-registered clipped image, to obtain a coarse registration transformation relationship between the target clipped image and the to-be-registered clipped image.

[0013] coarsely registering the target clipped image with the to-be-registered clipped image according to the target sagittal plane profile data of the target clipped image and the to-be-registered sagittal plane profile data of the to-be-registered clipped image, to obtain a coarse registration transformation relationship between the target clipped image and the to-be-registered clipped image.

[0014] In one of the embodiments, coarsely registering the target vertebra segmentation result with the vertebra segmentation result to be registered according to the target sagittal plane profile data of the target vertebra segmentation result and the to-be-registered sagittal plane profile data of the vertebra segmentation result to be registered comprises:

[0015] determining a geometric center of the vertebra segmentation result of the target vertebra segmentation result according to the target vertebra segmentation result, generating a target sagittal plane slice image according to the geometric center of the vertebra segmentation result of the target vertebra segmentation result, and extracting target sagittal plane profile data of the target sagittal plane slice image;

[0016] determining a geometric center of the vertebra segmentation result to be registered according to the vertebra segmentation result to be registered, generating a to-be-registered sagittal plane slice image according to the geometric center of the vertebra segmentation result to be registered, and extracting to-be-registered sagittal plane profile data of the to-be-registered sagittal plane slice image;

[0017] matching the target sagittal plane profile data with the to-be-registered sagittal plane profile data.

[0018] In one of the embodiments, matching the target sagittal plane profile data with the to-be-registered sagittal plane profile data comprises:

[0019] matching the target sagittal plane profile data with the to-be-registered sagittal plane profile data by a shape-based template matching algorithm or a sparse point cloud matching algorithm.

[0020] In one of the embodiments, the target vertebra segmentation result and the to-be-registered vertebra segmentation result are pixel-aligned by a pixel alignment model, and a model evaluation criterion is set. The model evaluation criterion optimizes the coarse registration transformation relationship according to a target vertebra region loss value of the pixel alignment model and a corresponding target vertebra region weight and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, to obtain a fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result, including:

[0021] The target vertebra segmentation result and the to-be-registered vertebra segmentation result after coarse registration are initially pixel-aligned by the pixel alignment model, to obtain the to-be-registered vertebra segmentation result after initial alignment;

[0022] The first target vertebra region and the non-first target vertebra region in the target vertebra segmentation result, and the second target vertebra region and the non-second target vertebra region in the to-be-registered vertebra segmentation result after initial alignment are determined;

[0023] The target vertebra region loss value of the pixel alignment model is determined according to the pixel value deviation of the first target vertebra region and the pixel value deviation of the second target vertebra region by the model evaluation criterion, and the non-target vertebra region loss value of the pixel alignment model is determined according to the pixel value deviation of the non-first target vertebra region and the pixel value deviation of the non-second target vertebra region, and the initial evaluation value of the pixel alignment model is determined according to the target vertebra region loss value and the corresponding target vertebra region weight and the non-target vertebra region loss value and the corresponding non-target vertebra region weight;

[0024] The coarse registration transformation relationship is iteratively optimized according to the initial evaluation value, until the target vertebra region loss value and the non-target vertebra region loss value reach a preset iteration stop condition, to obtain the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0025] In one of the embodiments, the coarse registration transformation relationship is iteratively optimized according to the initial evaluation value, until the target vertebra region loss value and the non-target vertebra region loss value reach a preset iteration stop condition, to obtain the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result, including:

[0026] The coarse registration transformation relationship is first-round optimized according to the initial evaluation value and a preset learning rate, to obtain the first-round optimized registration transformation relationship;

[0027] The target vertebra segmentation result and the to-be-registered vertebra segmentation result after initial alignment are pixel-aligned by the pixel alignment model, to obtain the to-be-registered vertebra segmentation result after this-round alignment;

[0028] determine a first target vertebra region and a non-first target vertebra region in the target vertebra segmentation result, and a second target vertebra region and a non-second target vertebra region in the to-be-registered vertebra segmentation result after the first alignment;

[0029] determine a target vertebra region loss value of the pixel alignment model according to the pixel value deviation of the first target vertebra region and the pixel value deviation of the second target vertebra region, and determine a non-target vertebra region loss value of the pixel alignment model according to the pixel value deviation of the non-first target vertebra region and the pixel value deviation of the non-second target vertebra region, and determine an evaluation value of the pixel alignment model according to the target vertebra region loss value and the corresponding target vertebra region weight and the non-target vertebra region loss value and the corresponding non-target vertebra region weight;

[0030] optimize the registration transformation relationship of the first round of optimization according to the evaluation value and a preset learning rate to obtain the registration transformation relationship of the current round of optimization, update the registration transformation relationship of the first round of optimization to the registration transformation relationship of the current round of optimization, update the to-be-registered vertebra segmentation result after the initial alignment to the to-be-registered vertebra segmentation result after the current round of alignment, and return to the next round of iteration, and perform pixel alignment on the target vertebra segmentation result and the to-be-registered vertebra segmentation result after the alignment of the previous round until the target vertebra region loss value and the non-target vertebra region loss value reach a preset iteration stop condition, and take the registration transformation relationship of the current round of optimization as the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0031] In one of the embodiments, the registration of the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship comprises:

[0032] obtaining a first transformation relationship between the target vertebra segmentation result and the preoperative three-dimensional image, and a second transformation relationship between the to-be-registered vertebra segmentation result and the intraoperative three-dimensional image;

[0033] determining a transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image according to the first transformation relationship, the second transformation relationship and the fine registration transformation relationship, so as to realize the registration of the preoperative three-dimensional image and the intraoperative three-dimensional image.

[0034] In a second aspect, the application further provides a vertebra image registration device, comprising:

[0035] a segmentation module configured to perform vertebra segmentation on the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebra segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebra segmentation result corresponding to the intraoperative three-dimensional image; wherein the target vertebra segmentation result and the to-be-registered vertebra segmentation result are segmentation images of the same vertebra;

[0036] a coarse registration module, configured to perform coarse registration on the target vertebra segmentation result and the vertebra segmentation result to be registered according to target sagittal profile data of the target vertebra segmentation result and to-be-registered sagittal profile data of the vertebra segmentation result to be registered, so as to obtain a coarse registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered;

[0037] a fine registration module, configured to perform pixel alignment on the target vertebra segmentation result and the vertebra segmentation result to be registered by using a pixel alignment model, and set a model evaluation standard, the model evaluation standard being configured to optimize the coarse registration transformation relationship according to a target vertebra region loss value of the pixel alignment model and a corresponding target vertebra region weight and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, so as to obtain a fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered;

[0038] a transformation module, configured to perform registration on the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship.

[0039] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0040] performing vertebra segmentation on the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebra segmentation result corresponding to the preoperative three-dimensional image and a vertebra segmentation result to be registered corresponding to the intraoperative three-dimensional image; wherein the target vertebra segmentation result and the vertebra segmentation result to be registered are segmentation images of the same vertebra;

[0041] performing coarse registration on the target vertebra segmentation result and the vertebra segmentation result to be registered according to target sagittal profile data of the target vertebra segmentation result and to-be-registered sagittal profile data of the vertebra segmentation result to be registered, so as to obtain a coarse registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered;

[0042] performing pixel alignment on the target vertebra segmentation result and the vertebra segmentation result to be registered by using a pixel alignment model, and setting a model evaluation standard, the model evaluation standard being configured to optimize the coarse registration transformation relationship according to a target vertebra region loss value of the pixel alignment model and a corresponding target vertebra region weight and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, so as to obtain a fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered;

[0043] performing registration on the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship.

[0044] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0045] segmenting the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebra segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebra segmentation result corresponding to the intraoperative three-dimensional image; the target vertebra segmentation result and the to-be-registered vertebra segmentation result are segmentation images of the same vertebra;

[0046] performing coarse registration on the target vertebra segmentation result and the to-be-registered vertebra segmentation result according to target sagittal plane profile data of the target vertebra segmentation result and to-be-registered sagittal plane profile data of the to-be-registered vertebra segmentation result, to obtain a coarse registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result;

[0047] performing pixel alignment on the target vertebra segmentation result and the to-be-registered vertebra segmentation result through a pixel alignment model, and setting a model evaluation standard; the model evaluation standard optimizes the coarse registration transformation relationship according to a target vertebra region loss value and a corresponding target vertebra region weight of the pixel alignment model and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, to obtain a fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result;

[0048] performing registration on the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship.

[0049] In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0050] segmenting the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebra segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebra segmentation result corresponding to the intraoperative three-dimensional image; the target vertebra segmentation result and the to-be-registered vertebra segmentation result are segmentation images of the same vertebra;

[0051] performing coarse registration on the target vertebra segmentation result and the to-be-registered vertebra segmentation result according to target sagittal plane profile data of the target vertebra segmentation result and to-be-registered sagittal plane profile data of the to-be-registered vertebra segmentation result, to obtain a coarse registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result;

[0052] performing pixel alignment on the target vertebra segmentation result and the to-be-registered vertebra segmentation result through a pixel alignment model, and setting a model evaluation standard; the model evaluation standard optimizes the coarse registration transformation relationship according to a target vertebra region loss value and a corresponding target vertebra region weight of the pixel alignment model and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, to obtain a fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result;

[0053] The preoperative three-dimensional image and the intraoperative three-dimensional image are registered according to the precise registration transformation relationship.

[0054] The vertebral level image registration method, device, computer equipment, storage medium and computer program product can realize 3D coarse registration of vertebral levels through sagittal plane profile data matching, can quickly converge the solution of the transformation relationship of the preoperative three-dimensional image and the intraoperative three-dimensional image to the vicinity of the global optimal solution, and improves the registration speed. The target vertebral level region weight and the non-target vertebral level region weight are introduced in the optimization process of the coarse registration transformation relationship, can remove the interference of the non-target vertebral level region to the greatest extent, obtain the global optimal solution, and effectively improve the registration accuracy. The method solves the problem of falling into a local optimal solution in the traditional 3D vertebral level image registration process, realizes quick convergence to the global optimal solution, and further improves the registration accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0056] Figure 1 Fig. 1 is a flowchart of a vertebral level image registration method in an embodiment;

[0057] Fig. 2(a) is a sagittal plane schematic diagram in a target vertebral level segmentation result in an embodiment;

[0058] Fig. 2(b) is a coronal plane schematic diagram in a target vertebral level segmentation result in an embodiment;

[0059] Fig. 2(c) is a transverse plane schematic diagram in a target vertebral level segmentation result in an embodiment;

[0060] Fig. 3(a) is a sagittal plane schematic diagram in a to-be-registered vertebral level segmentation result in an embodiment;

[0061] Fig. 3(b) is a coronal plane schematic diagram in a to-be-registered vertebral level segmentation result in an embodiment;

[0062] Fig. 3(c) is a transverse plane schematic diagram in a to-be-registered vertebral level segmentation result in an embodiment;

[0063] Fig. 4(a) is a schematic diagram of a target cropped image in an embodiment;

[0064] Fig. 4(b) is a schematic diagram of a to-be-registered cropped image in an embodiment;

[0065] Fig. 5(a) is a three-dimensional schematic diagram of a target cropped image in an embodiment;

[0066] Fig. 5(b) is a schematic view of the target sagittal profile data extracted from the target sagittal slice image in an embodiment;

[0067] Fig. 6(a) is a schematic view of the three-dimensional image to be registered in an embodiment;

[0068] Fig. 6(b) is a schematic view of the to-be-registered sagittal profile data extracted from the to-be-registered sagittal slice image in an embodiment;

[0069] Fig. 7(a) is a schematic view of the target sagittal slice image in an ideal case in an embodiment;

[0070] Fig. 7(b) is a schematic view of the to-be-registered sagittal slice image in an ideal case in an embodiment;

[0071] Fig. 8(a) is a schematic view of the sagittal registration result of the preoperative three-dimensional image and the intraoperative three-dimensional image in an embodiment;

[0072] Fig. 8(b) is a schematic view of the coronal registration result of the preoperative three-dimensional image and the intraoperative three-dimensional image in an embodiment;

[0073] Fig. 8(c) is a schematic view of the transverse registration result of the preoperative three-dimensional image and the intraoperative three-dimensional image in an embodiment;

[0074] Figure 9 Fig. 9 is a structural block diagram of the vertebral level image registration device in an embodiment;

[0075] Figure 10 Fig. 10 is an internal structural diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0076] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0077] In an exemplary embodiment, as shown in Figure 1 , a vertebral level image registration method is provided, which is described below by taking a terminal in Figure 1 as an example, and includes the following steps 102 to 108. Wherein:

[0078] Step 102, segmenting a target vertebral level in the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebral level segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebral level segmentation result corresponding to the intraoperative three-dimensional image; wherein the target vertebral level segmentation result and the to-be-registered vertebral level segmentation result are segmentation results of the same vertebral level.

[0079] Wherein, the preoperative three-dimensional image refers to a reference vertebral image as a target image in the vertebral image registration process. The intraoperative three-dimensional image refers to a vertebral image that needs to be registered. The preoperative three-dimensional image and the intraoperative three-dimensional image are original medical images, including the same vertebral segments, which can be one or more. The target vertebral segmentation result and the to-be-registered vertebral segmentation result are vertebral voxel point clouds in the images.

[0080] In some embodiments, the same to-be-registered target vertebral segment in the preoperative three-dimensional image and the intraoperative three-dimensional image is extracted using a vertebral segmentation algorithm to obtain a target vertebral segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebral segmentation result corresponding to the intraoperative three-dimensional image. For example, the vertebral segmentation algorithm can include an image processing algorithm, an AI deep learning network, a manual painting method, etc.

[0081] For example, the preoperative three-dimensional image is a CT image, and the intraoperative three-dimensional image is a CBCT image. The color areas in FIG. 2(a), FIG. 2(b) and FIG. 2(c) respectively represent the sagittal, coronal and transverse slice diagrams of the target vertebral segmentation result. The color areas in FIG. 3(a), FIG. 3(b) and FIG. 3(c) respectively represent the sagittal, coronal and transverse slice diagrams of the to-be-registered vertebral segmentation result.

[0082] Step 104, according to the target sagittal profile data of the target vertebral segmentation result and the to-be-registered sagittal profile data of the to-be-registered vertebral segmentation result, the target vertebral segmentation result is coarsely registered with the to-be-registered vertebral segmentation result to obtain a coarse registration transformation relationship between the target vertebral segmentation result and the to-be-registered vertebral segmentation result.

[0083] Wherein, the target sagittal profile data refers to the sagittal skeletal edge profile points of the vertebral segment in the target vertebral segmentation result. The to-be-registered sagittal profile data refers to the sagittal skeletal edge profile points of the vertebral segment in the to-be-registered vertebral segmentation result.

[0084] In one embodiment, before the target vertebral segmentation result is coarsely registered with the to-be-registered vertebral segmentation result according to the target sagittal profile data of the target vertebral segmentation result and the to-be-registered sagittal profile data of the to-be-registered vertebral segmentation result, the above method further comprises: generating a target vertebral bounding volume according to the target vertebral segmentation result, and cutting the preoperative three-dimensional image according to the target vertebral bounding volume to obtain a target cut image; generating a to-be-registered vertebral bounding volume according to the to-be-registered vertebral segmentation result, and cutting the intraoperative three-dimensional image according to the to-be-registered vertebral bounding volume to obtain a to-be-registered cut image.

[0085] Optionally, due to the large size of the preoperative three-dimensional image and the intraoperative three-dimensional image, in addition to the same vertebral segment in the preoperative three-dimensional image and the intraoperative three-dimensional image, i.e. the region of interest, there are many irrelevant regions, which will interfere with the registration process. In order to focus on the region of interest, before the target vertebral segment segmentation result is coarsely registered with the to-be-registered vertebral segment segmentation result, the same vertebral segment in the preoperative three-dimensional image and the intraoperative three-dimensional image is cropped to obtain a target cropped image and a to-be-registered cropped image.

[0086] Further, a target vertebral segment positive bounding cube is generated according to the target vertebral segment segmentation result as a target vertebral segment bounding body, and the preoperative three-dimensional image is cropped according to the target vertebral segment bounding body to obtain a target cropped image corresponding to the preoperative three-dimensional image. A to-be-registered vertebral segment positive bounding cube is generated according to the to-be-registered vertebral segment segmentation result as a to-be-registered vertebral segment bounding body, and the intraoperative three-dimensional image is cropped according to the to-be-registered vertebral segment bounding body to obtain a to-be-registered cropped image corresponding to the intraoperative three-dimensional image. The positive bounding cube is usually referred to as an axis-aligned bounding box (AABB), which refers to a cube that is aligned with the coordinate axis and can completely wrap the target vertebral node cloud. As shown in FIG. 4(a) and FIG. 4(b), the target cropped image and the to-be-registered cropped image are respectively shown.

[0087] In the embodiment, the vertebral segment bounding bodies are respectively generated according to the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result, and the image is cropped according to the vertebral segment bounding body, which is conducive to subsequent focusing on the target vertebral segment for registration and can improve the registration accuracy.

[0088] For example, the preoperative three-dimensional image is a CT image, and the CT image coordinate system is denoted as , the intraoperative three-dimensional image is a CBCT image, and the CBCT image coordinate system is denoted as The transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image is calculated to realize the registration of the preoperative three-dimensional image and the intraoperative three-dimensional image. The transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image is the transformation relationship between the CT image coordinate system and the CBCT image coordinate system , denoted as T, so that

[0089]

[0090] The target cropped image coordinate system is denoted as , and the transformation relationship between the target cropped image coordinate system and the original CT image coordinate system is denoted as , so that

[0091]

[0092] The coordinate system of the to-be-registered cropped image is denoted as The coordinate system of the to-be-registered cropped image The coordinate system of the original CBCT image is denoted as The transformation relationship between the coordinate system of the to-be-registered cropped image and the coordinate system of the original CBCT image is denoted as So that:

[0093]

[0094] In some embodiments, the target cropped image and the to-be-registered cropped image are coarsely registered. Specifically, a target sagittal slice image of the target cropped image is obtained, and target sagittal profile data in the target sagittal slice image is extracted. A to-be-registered sagittal slice image of the to-be-registered cropped image is obtained, and to-be-registered sagittal profile data in the to-be-registered sagittal slice image is extracted. The target sagittal profile data and the to-be-registered sagittal profile data are matched, and a transformation relationship between the target sagittal profile data and the to-be-registered sagittal profile data is calculated, and the to-be-registered cropped image after coarse registration is obtained. The transformation relationship is taken as a coarse registration transformation relationship between the target cropped image and the to-be-registered cropped image, so that the same vertebral level in the target cropped image and the to-be-registered cropped image is approximately aligned. Through coarse registration, the solution of the transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image is quickly optimized to the vicinity of the globally optimal solution.

[0095] Exemplarily, the coarse registration transformation relationship between the target cropped image and the to-be-registered cropped image is denoted as T0, wherein:

[0096]

[0097] In one of the embodiments, coarse registration of the target vertebral segmentation result and the to-be-registered vertebral segmentation result according to the target sagittal profile data of the target vertebral segmentation result and the to-be-registered sagittal profile data of the to-be-registered vertebral segmentation result comprises:

[0098] determining a vertebral geometric center of the target vertebral segmentation result according to the target vertebral segmentation result, generating a target sagittal slice image according to the vertebral geometric center of the target vertebral segmentation result, and extracting target sagittal profile data of the target sagittal slice image; determining a vertebral geometric center of the to-be-registered vertebral segmentation result according to the to-be-registered vertebral segmentation result, generating a to-be-registered sagittal slice image according to the vertebral geometric center of the to-be-registered vertebral segmentation result, and extracting to-be-registered sagittal profile data of the to-be-registered sagittal slice image; and matching the target sagittal profile data and the to-be-registered sagittal profile data.

[0099] Specifically, an average value of the voxel coordinate values of the voxel point cloud of the target vertebra segmentation result is calculated, and the average value is taken as the geometric center of the target vertebra segmentation result. A sagittal plane passing through the geometric center of the target vertebra segmentation result is found, and a target sagittal slice image is generated. An edge extraction algorithm is used to extract the vertebra bone edge in the target sagittal slice image to obtain target sagittal profile data.

[0100] An average value of the voxel coordinate values of the voxel point cloud of the target vertebra segmentation result is calculated, and the average value is taken as the geometric center of the target vertebra segmentation result. A sagittal plane passing through the geometric center of the target vertebra segmentation result is found, and a target sagittal slice image is generated. An edge extraction algorithm is used to extract the vertebra bone edge in the target sagittal slice image to obtain target sagittal profile data.

[0101] The target sagittal profile data and the to-be-registered sagittal profile data are matched to achieve coarse registration of the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0102] Exemplarily, the target cropped image and the to-be-registered cropped image can also be coarsely registered according to the target sagittal profile data of the target cropped image and the to-be-registered sagittal profile data of the to-be-registered cropped image to achieve coarse registration of the preoperative three-dimensional image and the intraoperative three-dimensional image. As shown in FIG. 5(a) and FIG. 6(a), a three-dimensional schematic diagram of the target cropped image and a three-dimensional schematic diagram of the to-be-registered cropped image are respectively shown, wherein the green plane slice is a sagittal plane. FIG. 5(b) and FIG. 6(b) respectively show the target sagittal profile data of the target cropped image and the to-be-registered sagittal profile data of the to-be-registered cropped image, wherein the red box represents the extracted sagittal profile data. The sagittal profile data is specifically described as a series of sparse point clouds.

[0103] In the embodiment, the sagittal slice image is generated according to the geometric center of the vertebra segmentation result, and the vertebra profile data of the sagittal slice image is extracted for coarse registration. Since the sagittal slice image is obtained by slicing the vertebra from the longitudinal axis direction, the arrangement and morphological characteristics of the vertebra in the longitudinal axis direction can be clearly shown, and thus the coarse registration by extracting the sagittal profile data can help to quickly judge the overall position and direction error, and can make the transformation relationship solution of the preoperative three-dimensional image and the intraoperative three-dimensional image converge to the vicinity of the global optimal solution, thereby further improving the registration speed.

[0104] Further, the matching of the target sagittal profile data and the to-be-registered sagittal profile data includes: matching the target sagittal profile data and the to-be-registered sagittal profile data by using a shape-based template matching algorithm or a sparse point cloud matching algorithm.

[0105] The target sagittal plane profile data is matched with the to-be-registered sagittal plane profile data using a shape-based template matching algorithm or a least squares method, a sparse point cloud matching algorithm such as SRP (Slice-wise Registration of Point-clouds), ICP (Iterative Closest Point), or the like. In this embodiment, the profile data matching algorithm can be flexibly selected according to actual needs.

[0106] In step 106, the target vertebra segmentation result and the to-be-registered vertebra segmentation result are pixel-aligned by a pixel alignment model, and a model evaluation standard is set. The model evaluation standard optimizes the coarse registration transformation relationship according to a target vertebra region loss value and a corresponding target vertebra region weight and a non-target vertebra region loss value and a corresponding non-target vertebra region weight of the pixel alignment model, to obtain a fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0107] The target vertebra region loss value refers to a loss function value of the target vertebra region, and the non-target vertebra region loss value refers to a loss function value of the non-target vertebra region. The target vertebra region refers to the same vertebra part corresponding to the optimization registration result of the current round, and the same vertebra can be referred to as a target vertebra. The non-target vertebra region refers to a part such as the previous vertebra, the next vertebra, and the surrounding region outside the target vertebra region, which interferes with the registration of the target vertebra.

[0108] Since the above steps have rapidly optimized the solution of the transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image to the vicinity of the global optimal solution, at this time, the vertebra fine registration of the target vertebra segmentation result and the to-be-registered vertebra segmentation result needs to be completed, that is, the global optimal solution is obtained.

[0109] Specifically, the process of vertebra fine registration can be regarded as a process of vertebra pixel alignment of the target vertebra segmentation result and the to-be-registered vertebra segmentation result, and can be calculated by a pixel alignment model. For example, the pixel alignment model can be a voxel-based gradient descent optimization algorithm. The optimization accuracy of the pixel alignment model depends on the model evaluation standard K of the pixel alignment model.

[0110] Considering that, in addition to the target vertebra region, there is also a non-target vertebra region in the target vertebra segmentation result and the to-be-registered vertebra segmentation result, when defining the model evaluation standard K of the pixel alignment model of vertebra fine registration, the target vertebra region and the non-target vertebra region corresponding to the optimization registration result of the current round are calculated respectively, that is, the target vertebra region weight and the non-target vertebra region weight After defining the model evaluation standard K, the pixel alignment model and the model evaluation standard K are used to iteratively optimize the coarse registration transformation relationship, so as to obtain the fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered, that is, the global optimal solution .

[0111] In one of the embodiments, the target vertebra segmentation result and the vertebra segmentation result to be registered are pixel-aligned by the pixel alignment model, and a model evaluation standard is set. The model evaluation standard optimizes the coarse registration transformation relationship according to the target vertebra region loss value of the pixel alignment model and the corresponding target vertebra region weight, and the non-target vertebra region loss value and the corresponding non-target vertebra region weight, to obtain the fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered, including:

[0112] The target vertebra segmentation result and the vertebra segmentation result to be registered are initially pixel-aligned by the pixel alignment model to obtain the initial aligned vertebra segmentation result to be registered; a first target vertebra region and a non-first target vertebra region in the target vertebra segmentation result, and a second target vertebra region and a non-second target vertebra region in the initial aligned vertebra segmentation result to be registered are determined; the pixel value deviation of the first target vertebra region and the pixel value deviation of the second target vertebra region are used to determine the target vertebra region loss value of the pixel alignment model by the model evaluation standard, and the pixel value deviation of the non-first target vertebra region and the pixel value deviation of the non-second target vertebra region are used to determine the non-target vertebra region loss value of the pixel alignment model, and the initial evaluation value of the pixel alignment model is determined according to the target vertebra region loss value and the corresponding target vertebra region weight, and the non-target vertebra region loss value and the corresponding non-target vertebra region weight; the coarse registration transformation relationship is iteratively optimized according to the initial evaluation value until the target vertebra region loss value and the non-target vertebra region loss value reach the preset iteration stopping condition, to obtain the fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered.

[0113] Further, the coarse registration transformation relationship is iteratively optimized according to the initial evaluation value until the target vertebra region loss value and the non-target vertebra region loss value reach the preset iteration stopping condition, to obtain the fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered, including:

[0114] According to the initial evaluation value and the preset learning rate, the coarse registration transformation relationship is optimized for the first time to obtain a first round of optimized registration transformation relationship; the pixel alignment model is used to perform pixel alignment on the target vertebra segmentation result and the initial aligned to-be-registered vertebra segmentation result to obtain a round of aligned to-be-registered vertebra segmentation result; the first target vertebra region and the non-first target vertebra region in the target vertebra segmentation result and the second target vertebra region and the non-second target vertebra region in the round of aligned to-be-registered vertebra segmentation result are determined; according to the pixel value deviation of the first target vertebra region and the pixel value deviation of the second target vertebra region, the target vertebra region loss value of the pixel alignment model is determined according to the model evaluation standard, and according to the pixel value deviation of the non-first target vertebra region and the pixel value deviation of the non-second target vertebra region, the non-target vertebra region loss value of the pixel alignment model is determined, and the evaluation value of the pixel alignment model is determined according to the target vertebra region loss value and the corresponding target vertebra region weight and the non-target vertebra region loss value and the corresponding non-target vertebra region weight; according to the evaluation value and the preset learning rate, the initial optimized registration transformation relationship is optimized to obtain a round of optimized registration transformation relationship, the first round of optimized registration transformation relationship is updated to the round of optimized registration transformation relationship, the initial aligned to-be-registered vertebra segmentation result is updated to the round of aligned to-be-registered vertebra segmentation result, and the next round of iteration is returned, the pixel alignment is performed on the target vertebra segmentation result and the to-be-registered vertebra segmentation result aligned in the last round until the target vertebra region loss value and the non-target vertebra region loss value reach the preset iteration stop condition, and the current round of optimized registration transformation relationship is taken as the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0115] In some embodiments, the coarse registration transformation relationship between the target cropped image and the to-be-registered cropped image can also be optimized to obtain a fine registration transformation relationship between the target cropped image and the to-be-registered cropped image, so as to realize fine registration between the preoperative three-dimensional image and the intraoperative three-dimensional image. The process of optimizing the coarse registration transformation relationship between the target cropped image and the to-be-registered cropped image is the same as the process of optimizing the coarse registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0116] The following will be described taking the optimization of the coarse registration transformation relationship between the target cropped image and the to-be-registered cropped image as an example. As shown in FIGS. 5(a), 5(b), 6(a) and 6(b), in the cropped image, in addition to the target vertebra region, there are also the previous vertebra and the next vertebra and the surrounding region, i.e., the non-target vertebra region, the target vertebra region is the sagittal plane profile data in the figure, and the non-target vertebra region is the white region other than the sagittal plane profile data.

[0117] The ideal target sagittal plane slice image and the to-be-registered sagittal plane slice image are shown in FIG. 7(a) and FIG. 7(b) respectively. Based on the above considerations, the model evaluation standard K of the pixel alignment model is defined as follows:

[0118]

[0119] wherein, and are the loss function of the target vertebral region and the loss function of the non-target vertebral region respectively, and are the target vertebral region weight and the non-target vertebral region weight respectively.

[0120] The pixel value deviation E is defined as the difference between the pixel value and the pixel average value, and the loss function L is calculated as follows:

[0121]

[0122] wherein, represents the pixel value deviation of the first target vertebral region, represents the pixel value deviation of the second target vertebral region.

[0123] The target cropped image and the to-be-registered cropped image are initially pixel-aligned by the pixel alignment model to obtain the initial alignment registration parameter. The first target vertebral region and the non-first target vertebral region in the target cropped image, and the second target vertebral region and the non-second target vertebral region in the initial alignment registration parameter are determined. The pixel value deviation of the first target vertebral region and the pixel value deviation of the second target vertebral region are substituted into the calculation formula of the loss function L to obtain the target vertebral region loss value of the pixel alignment model. The pixel value deviation of the non-first target vertebral region and the pixel value deviation of the non-second target vertebral region are substituted into the calculation formula of the loss function L to obtain the non-target vertebral region loss value of the pixel alignment model.

[0124] The target vertebral region loss value and the non-target vertebral region loss value are substituted into the formula of the model evaluation standard K to calculate the value of the model evaluation standard K of the pixel alignment model, i.e. the initial evaluation value. According to the initial evaluation value and the preset learning rate, the coarse registration transformation relationship is optimized for the first round to obtain the first round optimized registration transformation relationship.

[0125] The target vertebra segmentation result and the registration parameter after initial alignment are pixel-aligned by a pixel alignment model to obtain a to-be-registered vertebra segmentation result after this round of alignment. A first target vertebra region and a non-first target vertebra region in the target vertebra segmentation result and a second target vertebra region and a non-second target vertebra region in the to-be-registered vertebra segmentation result after alignment are determined. A target vertebra region loss value of the pixel alignment model is determined according to a pixel value deviation of the first target vertebra region and a pixel value deviation of the second target vertebra region according to a model evaluation standard, and a non-target vertebra region loss value of the pixel alignment model is determined according to a pixel value deviation of the non-first target vertebra region and a pixel value deviation of the non-second target vertebra region, and an evaluation value of the pixel alignment model is determined according to the target vertebra region loss value and a corresponding target vertebra region weight and the non-target vertebra region loss value and a corresponding non-target vertebra region weight.

[0126] The registration transformation relationship of the first round of optimization is optimized according to the evaluation value and a preset learning rate to obtain a registration transformation relationship of the current round of optimization, the registration transformation relationship of the first round of optimization is updated to the registration transformation relationship of the current round of optimization, the registration parameter after initial alignment is updated to the to-be-registered vertebra segmentation result after this round of alignment, and the target vertebra segmentation result and the to-be-registered vertebra segmentation result after alignment of the last round are returned to the next round of iteration, and the pixel alignment is performed until the target vertebra region loss value and the non-target vertebra region loss value reach a preset iteration stop condition, and the registration transformation relationship of the current round of optimization is taken as the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0127] The formula for iterative optimization of the coarse registration transformation relationship is as follows:

[0128]

[0129] wherein, The preset learning rate is preferably 0.001; The registration transformation relationship calculated in the i th iteration, The registration transformation relationship calculated in the i+1 th iteration. In the first round of iterative optimization, T0, that is, the coarse registration transformation relationship. When the target vertebra region loss value and the non-target vertebra region loss value decrease to the preset iteration stop condition, is taken as That is, the fine registration transformation relationship between the target cropped image and the to-be-registered cropped image. The preset iteration stop condition can be that the target vertebra region loss value and the non-target vertebra region loss value converge or reach a preset threshold.

[0130] In the embodiment, by setting the model evaluation criterion, adding the target vertebra region weight and the non-target vertebra region weight in the model evaluation criterion, and iteratively optimizing the coarse registration transformation relationship according to the evaluation value of the model evaluation criterion, the interference of the non-target vertebra region can be effectively reduced, and the registration accuracy can be improved.

[0131] In step 108, the preoperative three-dimensional image and the intraoperative three-dimensional image are registered according to the fine registration transformation relationship.

[0132] In one of the embodiments, the registration of the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship comprises: obtaining a first transformation relationship between the target vertebra segmentation result and the preoperative three-dimensional image, and a second transformation relationship between the to-be-registered vertebra segmentation result and the intraoperative three-dimensional image; determining a transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image according to the first transformation relationship, the second transformation relationship, and the fine registration transformation relationship, so as to realize the registration of the preoperative three-dimensional image and the intraoperative three-dimensional image.

[0133] Further, the transformation relationship T between the preoperative three-dimensional image and the intraoperative three-dimensional image can also be determined according to the first transformation relationship between the target cropped image and the preoperative three-dimensional image , the second transformation relationship between the to-be-registered cropped image and the intraoperative three-dimensional image , and the fine registration transformation relationship between the target cropped image and the to-be-registered cropped image The calculation formula of the transformation relationship T between the preoperative three-dimensional image and the intraoperative three-dimensional image is as follows:

[0134]

[0135] The alignment of the same vertebra in the preoperative three-dimensional image and the intraoperative three-dimensional image can be realized through the transformation relationship T between the preoperative three-dimensional image and the intraoperative three-dimensional image, as shown in FIGS. 8(a), 8(b), and 8(c), which respectively represent the sagittal plane, coronal plane, and transverse plane registration result schematic diagrams of the preoperative three-dimensional image and the intraoperative three-dimensional image.

[0136] In the embodiment, after the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result is calculated, the transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image can be quickly determined according to the first transformation relationship between the target vertebra segmentation result and the preoperative three-dimensional image, and the second transformation relationship between the to-be-registered vertebra segmentation result and the intraoperative three-dimensional image.

[0137] In the above-mentioned vertebral level image registration method, the 3D coarse registration of the vertebral level is achieved by matching the sagittal plane profile data, which can quickly converge the solution of the transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image to the vicinity of the global optimal solution, and improve the registration speed. In the optimization process of the coarse registration transformation relationship, the target vertebral level area weight and the non-target vertebral level area weight are introduced, which can remove the interference of the non-target vertebral level area to the greatest extent, obtain the global optimal solution, and effectively improve the registration accuracy. The method solves the problem of falling into a local optimal solution in the traditional 3D vertebral level image registration process, quickly converges to the global optimal solution, and further improves the registration accuracy on the basis of ensuring the registration robustness.

[0138] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the order of the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or stages or steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiments of the present application also provide a vertebral level image registration device for implementing the above-mentioned vertebral level image registration method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more vertebral level image registration device embodiments provided below can refer to the limitations of the vertebral level image registration method described above, which will not be repeated here.

[0140] In one exemplary embodiment, as shown in Figure 9 A vertebral level image registration device is provided, comprising: a segmentation module 902, a coarse registration module 904, a fine registration module 906, and a transformation module 908, wherein:

[0141] The segmentation module 902 is configured to segment the preoperative three-dimensional image and the intraoperative three-dimensional image to obtain a target vertebral level segmentation result corresponding to the preoperative three-dimensional image and a to-be-registered vertebral level segmentation result corresponding to the intraoperative three-dimensional image; wherein the target vertebral level segmentation result and the to-be-registered vertebral level segmentation result are segmentation images of the same vertebral level.

[0142] The coarse registration module 904 is configured to perform coarse registration on the target vertebra segmentation result and the vertebra segmentation result to be registered according to the target sagittal profile data of the target vertebra segmentation result and the sagittal profile data to be registered of the vertebra segmentation result to be registered, so as to obtain a coarse registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered.

[0143] The fine registration module 906 is configured to perform pixel alignment on the target vertebra segmentation result and the vertebra segmentation result to be registered by using a pixel alignment model, and set a model evaluation standard. The model evaluation standard is used to optimize the coarse registration transformation relationship according to a target vertebra region loss value of the pixel alignment model and a corresponding target vertebra region weight and a non-target vertebra region loss value and a corresponding non-target vertebra region weight, so as to obtain a fine registration transformation relationship between the target vertebra segmentation result and the vertebra segmentation result to be registered.

[0144] The transformation module 908 is configured to perform registration on the preoperative three-dimensional image and the intraoperative three-dimensional image according to the fine registration transformation relationship.

[0145] In an exemplary embodiment, the device further comprises:

[0146] The cropping module is configured to generate a target vertebra bounding volume according to the target vertebra segmentation result, crop the preoperative three-dimensional image according to the target vertebra bounding volume to obtain a target cropped image, generate a vertebra segmentation result to be registered bounding volume according to the vertebra segmentation result to be registered, and crop the intraoperative three-dimensional image according to the vertebra segmentation result to be registered bounding volume to obtain a vertebra segmentation result to be registered cropped image.

[0147] The cropping module is further configured to perform coarse registration on the target cropped image and the vertebra segmentation result to be registered cropped image according to the target sagittal profile data of the target cropped image and the sagittal profile data to be registered of the vertebra segmentation result to be registered cropped image, so as to obtain a coarse registration transformation relationship between the target cropped image and the vertebra segmentation result to be registered cropped image.

[0148] In an exemplary embodiment, the coarse registration module 904 is further configured to determine a vertebra geometric center of the target vertebra segmentation result according to the target vertebra segmentation result, generate a target sagittal slice image according to the vertebra geometric center of the target vertebra segmentation result, and extract target sagittal profile data of the target sagittal slice image; determine a vertebra geometric center of the vertebra segmentation result to be registered according to the vertebra segmentation result to be registered, generate a sagittal slice image to be registered according to the vertebra geometric center of the vertebra segmentation result to be registered, and extract sagittal profile data to be registered of the sagittal slice image to be registered; and match the target sagittal profile data and the sagittal profile data to be registered.

[0149] In an example embodiment, the coarse registration module 904 is further configured to match the target sagittal profile data with the to-be-registered sagittal profile data by a shape-based template matching algorithm or a sparse point cloud matching algorithm.

[0150] In an example embodiment, the fine registration module 906 is further configured to: perform initial pixel alignment of the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result by a pixel alignment model to obtain an initial aligned to-be-registered vertebral segment segmentation image; determine a first target vertebral segment region and a non-first target vertebral segment region in the target vertebral segment segmentation result, and a second target vertebral segment region and a non-second target vertebral segment region in the initial aligned to-be-registered vertebral segment segmentation result; determine a target vertebral segment region loss value of the pixel alignment model according to a pixel value deviation of the first target vertebral segment region and a pixel value deviation of the second target vertebral segment region, and determine a non-target vertebral segment region loss value of the pixel alignment model according to a pixel value deviation of the non-first target vertebral segment region and a pixel value deviation of the non-second target vertebral segment region according to a model evaluation standard, and determine an initial evaluation value of the pixel alignment model according to the target vertebral segment region loss value and a corresponding target vertebral segment region weight and the non-target vertebral segment region loss value and a corresponding non-target vertebral segment region weight; and iteratively optimize the coarse registration transformation relationship according to the initial evaluation value until the target vertebral segment region loss value and the non-target vertebral segment region loss value reach a preset iteration stopping condition to obtain a fine registration transformation relationship between the target vertebral segment segmentation result and the to-be-registered vertebral segment segmentation result.

[0151] In an example embodiment, the fine registration module 906 is further configured to perform first round optimization on the coarse registration transformation relationship according to the initial evaluation value and a preset learning rate to obtain a first round optimized registration transformation relationship; perform pixel alignment on the target vertebra segmentation result and the initial aligned to-be-registered vertebra segmentation result by using the pixel alignment model to obtain a current round aligned to-be-registered vertebra segmentation result; determine a first target vertebra region and a non-first target vertebra region in the target vertebra segmentation result and a second target vertebra region and a non-second target vertebra region in the current round aligned to-be-registered vertebra segmentation result; determine a target vertebra region loss value of the pixel alignment model according to the pixel value deviation of the first target vertebra region and the pixel value deviation of the second target vertebra region and determine a non-target vertebra region loss value of the pixel alignment model according to the pixel value deviation of the non-first target vertebra region and the pixel value deviation of the non-second target vertebra region according to the model evaluation standard, and determine an evaluation value of the pixel alignment model according to the target vertebra region loss value and a corresponding target vertebra region weight and the non-target vertebra region loss value and a corresponding non-target vertebra region weight; perform optimization on the first round optimized registration transformation relationship according to the evaluation value and the preset learning rate to obtain a current round optimized registration transformation relationship, update the first round optimized registration transformation relationship to the current round optimized registration transformation relationship, update the initial aligned to-be-registered vertebra segmentation result to the current round aligned to-be-registered vertebra segmentation result, and return to the next round of iteration to perform pixel alignment on the target vertebra segmentation result and the to-be-registered vertebra segmentation result aligned in the previous round until the target vertebra region loss value and the non-target vertebra region loss value reach a preset iteration stop condition, and take the current round optimized registration transformation relationship as the fine registration transformation relationship between the target vertebra segmentation result and the to-be-registered vertebra segmentation result.

[0152] In an example embodiment, the transformation module 908 is further configured to obtain a first transformation relationship between the target cropped image and the preoperative three-dimensional image and a second transformation relationship between the to-be-registered cropped image and the intraoperative three-dimensional image; determine a transformation relationship between the preoperative three-dimensional image and the intraoperative three-dimensional image according to the first transformation relationship, the second transformation relationship and the fine registration transformation relationship to realize registration of the preoperative three-dimensional image and the intraoperative three-dimensional image.

[0153] The above-mentioned modules in the vertebra image registration device can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned modules.

[0154] In an example embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 10The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a vertebral image registration method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0155] Those skilled in the art can understand that, Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0157] In one embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0158] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0159] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0160] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0161] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0162] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A vertebral segment image registration method, characterized in that: The method comprises: Performing vertebral segmentation on the preoperative 3D image and the intraoperative 3D image to obtain a target vertebral segmentation result corresponding to the preoperative 3D image and a vertebral segmentation result to be registered corresponding to the intraoperative 3D image; wherein the target vertebral segmentation result and the vertebral segmentation result to be registered are segmented images of the same vertebral segment; Based on the target sagittal contour data of the target vertebral segmentation result and the to-be-registered sagittal contour data of the to-be-registered vertebral segmentation result, the target vertebral segmentation result and the to-be-registered vertebral segmentation result are coarsely registered to obtain a coarse registration transformation relationship between the target vertebral segmentation result and the to-be-registered vertebral segmentation result; The target vertebral segmentation result and the vertebral segmentation result to be registered are pixel-aligned by a pixel alignment model, and a model evaluation standard is set. The model evaluation standard optimizes the coarse registration transformation relationship according to the target vertebral segment area loss value and the corresponding target vertebral segment area weight and the non-target vertebral segment area loss value and the corresponding non-target vertebral segment area weight of the pixel alignment model to obtain a fine registration transformation relationship between the target vertebral segmentation result and the vertebral segment segmentation result to be registered; The preoperative three-dimensional image and the intraoperative three-dimensional image are registered according to the precise registration transformation relationship.

2. The method according to claim 1, characterized in that Before roughly registering the target vertebral segmentation result with the vertebral segmentation result to be registered based on the target sagittal contour data of the target vertebral segmentation result and the sagittal contour data to be registered of the vertebral segmentation result to be registered, the method further includes: generating a target vertebral segment bounding volume according to the target vertebral segment segmentation result, and cropping the preoperative three-dimensional image according to the target vertebral segment bounding volume to obtain a target cropped image; generating a bounding volume of the vertebral segment to be registered according to the segmentation result of the vertebral segment to be registered, and cropping the intraoperative three-dimensional image according to the bounding volume of the vertebral segment to be registered to obtain a cropped image to be registered; The step of coarsely registering the target vertebral segmentation result with the vertebral segmentation result to be registered based on the target sagittal contour data of the target vertebral segmentation result and the sagittal contour data to be registered of the vertebral segmentation result to be registered, and obtaining a coarse registration transformation relationship between the target vertebral segmentation result and the vertebral segmentation result to be registered comprises: The target cropped image and the cropped image to be registered are coarsely registered according to the target sagittal contour data of the target cropped image and the sagittal contour data to be registered of the cropped image to be registered, so as to obtain a coarse registration transformation relationship between the target cropped image and the cropped image to be registered.

3. The method according to claim 1, characterized in that The coarsely registering the target vertebral segmentation result with the vertebral segmentation result to be registered based on the target sagittal contour data of the target vertebral segmentation result and the sagittal contour data to be registered of the vertebral segmentation result to be registered comprises: Determining a vertebral segment geometric center of the target vertebral segment segmentation result according to the target vertebral segment segmentation result, generating a target sagittal slice image according to the vertebral segment geometric center of the target vertebral segment segmentation result, and extracting target sagittal contour data of the target sagittal slice image; Determining the geometric center of the vertebral segmentation result to be registered according to the vertebral segment segmentation result to be registered, generating a sagittal slice image to be registered according to the geometric center of the vertebral segment segmentation result to be registered, and extracting the sagittal contour data to be registered of the sagittal slice image to be registered; The target sagittal plane contour data is matched with the sagittal plane contour data to be registered.

4. The method according to claim 3, characterized in that The matching of the target sagittal profile data with the sagittal profile data to be registered comprises: The target sagittal plane contour data is matched with the sagittal plane contour data to be registered by using a shape-based template matching algorithm or a sparse point cloud matching algorithm.

5. The method according to claim 1, wherein The target vertebral segmentation result and the to-be-registered vertebral segmentation result are pixel-aligned by a pixel alignment model, and a model evaluation standard is set. The model evaluation standard optimizes the coarse registration transformation relationship according to the target vertebral segment region loss value and the corresponding target vertebral segment region weight and the non-target vertebral segment region loss value and the corresponding non-target vertebral segment region weight of the pixel alignment model, and obtains a fine registration transformation relationship between the target vertebral segmentation result and the to-be-registered vertebral segment segmentation result, including: Performing initial pixel alignment on the target vertebral segmentation result and the vertebral segmentation result to be registered using a pixel alignment model to obtain the vertebral segmentation result to be registered after initial alignment; Determining a first target vertebral segment region and a non-first target vertebral segment region in the target vertebral segment segmentation result, and a second target vertebral segment region and a non-second target vertebral segment region in the vertebral segmentation result to be registered after the initial alignment; Determining the target vertebral segment region loss value of the pixel alignment model according to the pixel value deviation of the first target vertebral segment region and the pixel value deviation of the second target vertebral segment region through a model evaluation standard, and determining the non-target vertebral segment region loss value of the pixel alignment model according to the pixel value deviation of the non-first target vertebral segment region and the pixel value deviation of the non-second target vertebral segment region, and determining the initial evaluation value of the pixel alignment model according to the target vertebral segment region loss value and the corresponding target vertebral segment region weight and the non-target vertebral segment region loss value and the corresponding non-target vertebral segment region weight; The coarse registration transformation relationship is iteratively optimized according to the initial evaluation value until the target vertebral segment region loss value and the non-target vertebral segment region loss value reach a preset iterative stop condition, thereby obtaining a fine registration transformation relationship between the target vertebral segmentation result and the vertebral segmentation result to be registered.

6. The method according to claim 5, characterized in that The iterative optimization of the coarse registration transformation relationship according to the initial evaluation value until the target vertebral segment region loss value and the non-target vertebral segment region loss value reach a preset iterative stop condition, and obtaining a fine registration transformation relationship between the target vertebral segmentation result and the vertebral segmentation result to be registered includes: Performing a first-round optimization on the coarse registration transformation relationship according to the initial evaluation value and a preset learning rate to obtain a first-round optimized registration transformation relationship; Perform pixel alignment on the target vertebral segmentation result and the vertebral segmentation result to be registered after the initial alignment using the pixel alignment model to obtain the vertebral segmentation result to be registered after this round of alignment; Determining a first target vertebral segment region and a non-first target vertebral segment region in the target vertebral segment segmentation result, and a second target vertebral segment region and a non-second target vertebral segment region in the vertebral segmentation result to be registered after the current round of alignment; Determine the target vertebral segment region loss value of the pixel alignment model according to the pixel value deviation of the first target vertebral segment region and the pixel value deviation of the second target vertebral segment region through the model evaluation standard, and determine the non-target vertebral segment region loss value of the pixel alignment model according to the pixel value deviation of the non-first target vertebral segment region and the pixel value deviation of the non-second target vertebral segment region, and determine the evaluation value of the pixel alignment model according to the target vertebral segment region loss value and the corresponding target vertebral segment region weight and the non-target vertebral segment region loss value and the corresponding non-target vertebral segment region weight; The registration transformation relationship of the first round of optimization is optimized according to the evaluation value and the preset learning rate to obtain the registration transformation relationship of this round of optimization, the registration transformation relationship of the first round of optimization is updated to the registration transformation relationship of this round of optimization, the vertebral segmentation result to be registered after the initial alignment is updated to the vertebral segmentation result to be registered after the alignment of this round, and the next round of iteration is returned to perform pixel alignment on the target vertebral segmentation result and the vertebral segmentation result to be registered after the previous round of alignment until the target vertebral segment area loss value and the non-target vertebral segment area loss value reach the preset iteration stop condition, and the registration transformation relationship of the current round of optimization is used as the precise registration transformation relationship between the target vertebral segmentation result and the vertebral segmentation result to be registered.

7. The method according to claim 1, characterized in that The registering the preoperative 3D image and the intraoperative 3D image according to the precise registration transformation relationship comprises: Acquiring a first transformation relationship between the target vertebral segmentation result and the preoperative three-dimensional image, and a second transformation relationship between the to-be-registered vertebral segmentation result and the intraoperative three-dimensional image; The transformation relationship between the preoperative 3D image and the intraoperative 3D image is determined according to the first transformation relationship, the second transformation relationship and the fine registration transformation relationship to achieve registration of the preoperative 3D image and the intraoperative 3D image.

8. A vertebral segment image registration device, characterized in that: The device comprises: a segmentation module for performing vertebral segmentation on the preoperative 3D image and the intraoperative 3D image to obtain a target vertebral segmentation result corresponding to the preoperative 3D image and a vertebral segmentation result to be registered corresponding to the intraoperative 3D image; wherein the target vertebral segmentation result and the vertebral segmentation result to be registered are segmented images of the same vertebral segment; a coarse registration module, configured to perform coarse registration on the target vertebral segmentation result and the to-be-registered vertebral segmentation result based on the target sagittal contour data of the target vertebral segmentation result and the to-be-registered sagittal contour data of the to-be-registered vertebral segmentation result, and obtain a coarse registration transformation relationship between the target vertebral segmentation result and the to-be-registered vertebral segmentation result; a fine registration module for performing pixel alignment on the target vertebral segmentation result and the vertebral segmentation result to be registered using a pixel alignment model, and setting a model evaluation standard. The model evaluation standard optimizes the coarse registration transformation relationship based on the target vertebral segment area loss value and the corresponding target vertebral segment area weight, as well as the non-target vertebral segment area loss value and the corresponding non-target vertebral segment area weight of the pixel alignment model, to obtain a fine registration transformation relationship between the target vertebral segmentation result and the vertebral segment segmentation result to be registered; A transformation module is used to register the pre-operative 3D image and the intra-operative 3D image according to the precise registration transformation relationship.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images