Registration method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a registration method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: segmenting a first modal image and a second modal image of a spine based on a centrum segmentation model to obtain a first mask region of each centrum in the first modal image and a second mask region of each centrum in the second modal image, and for each centrum, registering the first mask region and the second mask region of each centrum to obtain a first mask region and a second mask region of each centrum; and obtaining a target transformation matrix, and determining a fusion deformation field between the first modal image and the second modal image based on the target transformation matrix of each cone. Wherein the fusion deformation field is used for registering the first modal image and the second modal image, and the fusion deformation field comprises a rigid deformation field and a non-rigid deformation field. By adopting the method, the registration efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to a registration method, apparatus, computer equipment, readable storage medium, and program product. Background Technology

[0002] Different modalities of imaging can provide complementary diagnostic or therapeutic information. For example, magnetic resonance (MR) imaging shows better soft tissue contrast, while computed tomography (CT) imaging more clearly distinguishes between skeletal and non-skeletal structures. Therefore, in the diagnosis and analysis of many spine-related diseases, registration techniques are used to fuse multimodal images to fully utilize the advantages of each modality, thereby improving the diagnosis and surgical planning of spinal diseases.

[0003] Each vertebra in the spine possesses a rigid deformation field and a non-rigid deformation field from soft tissues (intervertebral discs, spinal cord, nerves). Related techniques utilize neural networks to learn the relationship between the rigid and non-rigid deformation fields in the spine to achieve registration between images of different modalities. However, these registration methods are inefficient. Summary of the Invention

[0004] Therefore, it is necessary to provide a registration method, apparatus, computer equipment, readable storage medium, and program product that can improve efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a registration method, including:

[0006] Based on the vertebral body segmentation model, the first and second modal images of the spine are segmented to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image.

[0007] For each vertebra, the first and second mask regions of the vertebra are registered to obtain the target transformation matrix;

[0008] Based on the target transformation matrix of each vertebra, the fusion deformation field between the first modal image and the second modal image is determined; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

[0009] In one embodiment, for each vertebra, the first mask region and the second mask region of the vertebra are registered to obtain a target transformation matrix, including:

[0010] For each vertebra, determine the prior transformation matrix between the first mask region and the second mask region of the vertebra;

[0011] Based on the prior transformation matrix corresponding to the vertebra, the first mask region and the second mask region, the first mask region and the second mask region of the vertebra are registered to obtain the initial transformation matrix corresponding to the vertebra.

[0012] Based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image corresponding to the first mask region of the vertebra in the first modal image, and the second sub-modal image corresponding to the second mask region of the vertebra in the second modal image, the first sub-modal image and the second sub-modal image of the vertebra are registered to obtain the target transformation matrix of the vertebra.

[0013] In one embodiment, based on the prior transformation matrix corresponding to the vertebra, the first mask region and the second mask region of the vertebra are registered to obtain the initial transformation matrix corresponding to the vertebra, including:

[0014] Based on the prior transformation matrix corresponding to the vertebra, the first transformation region is obtained by performing a geometric transformation on the first mask region of the vertebra.

[0015] Determine the similarity between the first transformation region and the second mask region of the cone;

[0016] Based on similarity, the first and second mask regions of the vertebra are registered to obtain the initial transformation matrix corresponding to the vertebra.

[0017] In one embodiment, based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image of the vertebra corresponding to the first mask region of the vertebra in the first modal image, and the second sub-modal image of the vertebra corresponding to the second mask region of the vertebra in the second modal image, the first sub-modal image and the second sub-modal image of the vertebra are registered to obtain the target transformation matrix of the vertebra, including:

[0018] Based on the initial transformation matrix corresponding to the vertebral body, a geometric transformation is performed on the first submodal image of the vertebral body to obtain the second transformation region;

[0019] Determine the similarity between the second transformed region and the second submodal image of the vertebral body;

[0020] Registration is performed based on the similarity between the second transformation region and the second submodal image of the vertebral body to obtain the target transformation matrix corresponding to the vertebral body.

[0021] In one embodiment, determining the prior transformation matrix between the first and second mask regions of the vertebral body includes:

[0022] Determine a first preset position in the first mask region of the vertebral body and a second preset position in the second mask region corresponding to the first preset position;

[0023] The prior transformation matrix is ​​determined based on the first and second preset positions.

[0024] In one embodiment, based on the target transformation matrix of each vertebra, the fusion deformation field between the first modal image and the second modal image is determined, including:

[0025] Generate Gaussian distribution maps corresponding to the second submodal images of each vertebra; the Gaussian distribution maps are used to characterize the probability distribution of the rigidity of each pixel in the second submodal image; the second submodal image of the vertebra is the image corresponding to the second mask region of the vertebra in the second modal image of the vertebra;

[0026] Based on the target transformation matrix of each vertebra, the rigid deformation field of each vertebra is determined.

[0027] Based on the rigid deformation field and Gaussian distribution map of each vertebra, the fused deformation field between the first modal image and the second modal image is determined.

[0028] In one embodiment, the rigid deformation field of each vertebra is determined based on the target transformation matrix of each vertebra, including:

[0029] For each vertebra, determine the first coordinates of each pixel in the first submodal image of the vertebra; the second submodal image of the vertebra is the image in the second modal image of the vertebra that corresponds to the second mask region of the vertebra.

[0030] For each pixel, the second coordinate of the pixel is determined by the product of the pixel's first coordinate and the corresponding element in the target transformation matrix.

[0031] The rigid deformation field of the cone is determined based on the first and second coordinates of each pixel.

[0032] In one embodiment, the fused deformation field between the first modal image and the second modal image is determined based on the rigid deformation field and Gaussian distribution map of each vertebra, including:

[0033] For each vertebra, the aggregation diagram of the vertebra is determined based on the rigid deformation field and Gaussian distribution of the vertebra;

[0034] Gaussian distribution map of the second modality image is obtained based on Gaussian distribution map of each vertebra, and aggregation map of the second modality image is obtained based on aggregation map of each vertebra;

[0035] The fused deformation field is determined based on the Gaussian distribution map and the aggregate map of the second modality image.

[0036] In one embodiment, the aggregation diagram of the vertebral body is determined based on the rigid deformation field and Gaussian distribution map of the vertebral body, including:

[0037] The aggregate map of the vertebra is determined by multiplying the pixel value of the first pixel in the rigid deformation field of the vertebra with the pixel value of the second pixel in the Gaussian distribution map of the vertebra; the first pixel corresponds to the second pixel.

[0038] In one embodiment, obtaining a Gaussian distribution map of the second modality image based on the Gaussian distribution map of each vertebra, and obtaining an aggregated map of the second modality image based on the aggregated map of each vertebra, includes:

[0039] Based on the position of each vertebra in the second modality image, the Gaussian distribution map of each vertebra is arranged to obtain the Gaussian distribution map of the second modality image, and the aggregation map of each vertebra is arranged to obtain the aggregation map of the second modality image.

[0040] In one embodiment, determining the fused deformation field based on the Gaussian distribution map of the second modal image and the aggregate map of the second modal image includes:

[0041] The fused deformation field is determined by multiplying the pixel value of the third pixel in the Gaussian distribution map of the second modality image with the pixel value of the fourth pixel in the aggregate map of the second modality image; the third pixel corresponds to the fourth pixel.

[0042] In one embodiment, a first modal image and a second modal image of the spine are segmented based on a vertebral segmentation model to obtain a first mask region for each vertebra in the first modal image and a second mask region for each vertebra in the second modal image, including:

[0043] The third modal image is preprocessed to obtain the first modal image, and the fourth modal image is preprocessed to obtain the second modal image;

[0044] The first modality image is input into the vertebral segmentation model to obtain each first mask region, and the second modality image is input into the vertebral segmentation model to obtain each second mask region.

[0045] Secondly, this application also provides a registration apparatus, comprising:

[0046] The segmentation module is used to segment the first and second modal images of the spine based on the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image.

[0047] The registration module is used to register the first and second mask regions of each vertebra to obtain the target transformation matrix.

[0048] The determination module is used to determine the fusion deformation field between the first modal image and the second modal image based on the target transformation matrix of each vertebra; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

[0049] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0050] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0051] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0052] The aforementioned registration method, apparatus, computer equipment, readable storage medium, and program product, because they can segment the first and second modal images of the spine based on a vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image, can automatically segment the spine using the vertebral segmentation model. Furthermore, for each vertebra, registering the first and second mask regions allows for registration based on the segmented individual vertebrae, improving the accuracy and efficiency of the obtained target transformation matrix. Furthermore, based on the target transformation matrix of each vertebra, the fusion deformation field between the first modal image and the second modal image can be determined to register the first modal image and the second modal image. On the one hand, the fusion deformation field includes rigid deformation field and non-rigid deformation field. Therefore, both rigid deformation field registration and non-rigid deformation field registration are realized, improving the registration accuracy. On the other hand, registration between multimodal images can be completed automatically without any manual operation, and it is not necessary for each modal image to have the same scanning field of view, thus improving the registration efficiency. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a diagram illustrating the application environment of the registration method in one embodiment;

[0055] Figure 2 This is a flowchart illustrating the registration method in one embodiment;

[0056] Figure 3 This is a schematic diagram of a first modal image and a second modal image in one embodiment;

[0057] Figure 4 This is a schematic diagram of a process for determining a target transformation matrix in one embodiment;

[0058] Figure 5 This is a flowchart illustrating the process of determining the initial transformation matrix in one embodiment;

[0059] Figure 6 This is a schematic diagram of another process for determining the target transformation matrix in one embodiment;

[0060] Figure 7 This is a flowchart illustrating the process of determining the prior transformation matrix in one embodiment;

[0061] Figure 8 This is a schematic diagram illustrating the process of determining the target transformation matrix in one embodiment;

[0062] Figure 9 This is a schematic diagram of a process for determining the fusion deformation field in one embodiment;

[0063] Figure 10 This is a flowchart illustrating the process of determining a rigid deformation field in one embodiment;

[0064] Figure 11 This is a schematic diagram of another process for determining the fusion deformation field in one embodiment;

[0065] Figure 12 This is a flowchart illustrating the process of determining the first mask region and the second mask region in one embodiment;

[0066] Figure 13 Here is a flowchart of the registration method in one embodiment;

[0067] Figure 14 This is a schematic diagram of the registration method in one embodiment;

[0068] Figure 15 This is a registration result image;

[0069] Figure 16 This is another registration result diagram;

[0070] Figure 17 This is a structural block diagram of the registration device in one embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] Multimodal image-based spinal registration plays a crucial role in many orthopedic applications, including assessment, diagnosis, surgical planning, and image-guided interventional procedures for various spinal diseases. The simultaneous existence of rigid variations in each vertebra and non-rigid variations in soft tissues (intervertebral discs, spinal cord, nerves) makes multimodal spinal registration a transformation of both global elasticity and local rigidity. This is why multimodal spinal registration is more complex and challenging than other registration tasks.

[0073] Two registration methods exist in related technologies: (1) Global elastic registration with added local rigidity loss constraints. By constraining the Jacobian matrix of the deformation field in the rigid region of the image to be an orthogonal matrix, the deformation of the rigid region is guaranteed to be rigid. However, this method is difficult to implement, computationally intensive, and has poor practicality. (2) Using deep learning to learn the relationship between the rigid and non-rigid deformation fields in the spine through neural networks. However, this method requires a large number of pairs of multimodal images with the same scanning field of view, and learning the relationship between the rigid and non-rigid deformation fields in the spine is also difficult.

[0074] It is evident that registration between multimodal images of the spine is challenging due to significant differences in scanning fields of view and the overlapping of rigidity and elastic deformation. Consequently, most related techniques require manual registration by physicians or registration only for a single segment of the spine, such as registration of the lumbar spine alone, making them difficult to apply to complex and varied clinical scenarios. Therefore, it is necessary to provide a registration method, which will be described below.

[0075] Figure 1 This is an application environment diagram of the registration method in one embodiment. In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 1As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a registration method.

[0076] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] Figure 2 This is a flowchart illustrating the registration method in one embodiment. In an exemplary embodiment, such as... Figure 2 As shown, a registration method is provided, which can be applied to... Figure 1 The following explanation uses computer equipment as an example, including the following steps S201 to S203.

[0078] S201, based on the vertebral body segmentation model, the first modal image and the second modal image of the spine are segmented to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image.

[0079] In this embodiment, the computer device is capable of acquiring a first modal image and a second modal image of the spine. Optionally, the computer device may acquire the first modal image or the second modal image sent by a medical scanning device, or it may acquire the first modal image or the second modal image from other devices or storage. Further optionally, the first modal image and the second modal image may be the same size.

[0080] The first modality image and the second modality image are any two different modalities of images within the overall image set. These modalities may include, but are not limited to, computed tomography (CT) images, cone-beam computed tomography (CBCT) images, positron emission tomography (PET) images, magnetic resonance imaging (MR) images, PET-CT images, and PET-MR images. The following explanation will use the first modality image as an MR image and the second modality image as a CT image as an example, illustrating the registration of the MR image to the CT image.

[0081] Furthermore, both the first and second modal images include at least one vertebral body. Optionally, the vertebral bodies in the first and second modal images have a corresponding relationship, for example, a one-to-one correspondence. Figure 3 This is a schematic diagram of the first and second modal images in one embodiment, as shown by the rectangular dashed lines. Figure 3 (a) shows a CT image of the spine. Figure 3 (b) shows an MR image of the spine. It can be seen that both the CT and MR images include vertebral bodies 1 to 6. That is, vertebral body 1 in the CT image and vertebral body 1 in the MR image are the same vertebral body, and the same applies to the other vertebral bodies.

[0082] Then, the computer device can segment the first modal image and the second modal image based on the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image.

[0083] The cone segmentation model may include, but is not limited to, at least one of the following: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Fully Convolutional Neural Networks (FCN), Generative Adversarial Networks (GAN), Back-propagation (BP) machine learning models, Radial Basis Functions (RBF), Deep Belief Networks (DBN), Elman models, or combinations thereof.

[0084] Optionally, to improve segmentation efficiency, a lightweight model can be used for cone segmentation.

[0085] In one embodiment, different modal images can correspond to the same vertebral segmentation model. For example, the vertebral segmentation model can be trained based on different modal image samples and the mask regions of each vertebra in each modal image sample. Furthermore, a computer device can input the first modal image and the second modal image into the vertebral segmentation model to obtain the first mask region and the second mask region output by the vertebral segmentation model, respectively.

[0086] In some embodiments, the vertebral segmentation model may also include segmentation sub-models corresponding to different modal images. Each segmentation sub-model can be trained from a modal image sample and the mask regions of each vertebra in each modal image sample. Furthermore, the computer device can input the first modal image into the segmentation sub-model corresponding to the first modal image to obtain the first mask region, and input the second modal image into the segmentation sub-model corresponding to the second modal image to obtain the second mask region. This embodiment is not limited thereto.

[0087] The mask region is the mask region corresponding to each vertebra in the modal image. That is, the first mask region is used to indicate the location of the vertebra in the first modal image, and the second mask region is used to indicate the location of the vertebra in the second modal image. The mask region can be rectangular, circular, or other irregular shapes. It should be noted that the mask region at least covers the vertebra in the modal image. In some embodiments, the mask region can also cover the area near the appendage; this embodiment does not impose such limitations.

[0088] The mask region contains only two pixel values, and the pixel values ​​of pixels with cones in the mask region are different from those of pixels without cones. In other words, the pixel value of each pixel in the mask region either indicates that the pixel has a cone or that the pixel does not have a cone.

[0089] Please continue to refer to this. Figure 3 The computer device segments the first modal image based on the cone segmentation model, thus obtaining first mask regions 1 to 6. First mask region 1 is the mask region corresponding to cone 1 in the first modal image, first mask region 2 is the mask region corresponding to cone 2 in the first modal image, and so on. Similarly, the computer device segments the second modal image based on the cone segmentation model, thus obtaining second mask regions 1 to 6 corresponding to cones 1 to 2 in the second modal image.

[0090] Optionally, the pixel values ​​corresponding to the same cone can be the same in each mask region. For example, a pixel value of 1 in both the first mask region 1 and the second mask region 1 can indicate the presence of cone 1, while a pixel value of 0 indicates the absence of cone 1.

[0091] Optionally, the pixel values ​​corresponding to different cones in each mask region are different. For example, in the first mask region 1 or the second mask region 1, a pixel value of 1 indicates the presence of cone 2, and a pixel value of 0 indicates the absence of cone 2; in the first mask region 2 or the second mask region 2, a pixel value of 2 indicates the presence of cone 2, and a pixel value of 0 indicates the absence of cone 2.

[0092] Because different cones in each mask region correspond to different pixel values, computer equipment can also distinguish between the cones in the first mask region and the second mask region. Please continue to refer to... Figure 3 , Figure 3 In (a), different vertebrae are distinguished by different colors. Figure 3 (b) Different vertebrae are distinguished by different colors, while Figure 3 (a) and Figure 3 (b) The same vertebra can be marked with the same color.

[0093] Further optionally, in some embodiments, the mask regions of the same modal image may overlap. For example, the first mask region 1 and the first mask region 6 may overlap.

[0094] S202, for each vertebra, the first mask region and the second mask region of the vertebra are registered to obtain the target transformation matrix.

[0095] In this embodiment, the target transformation matrix is ​​used to indicate the spatial transformation relationship between the first mask region and the second mask region. The target transformation matrix can represent the translation and / or rotation between the first mask region and the second mask region of the same cone. For example, the target transformation matrix may include a translation vector and / or a rotation matrix. Each cone corresponds to a target transformation matrix. For example, cone 1 corresponds to target transformation matrix 1, cone 2 corresponds to target transformation matrix 2, and so on.

[0096] Continue with Figure 3 For example, for vertebra 1, the computer registers the first mask region 1 and the second mask region 1 to obtain the target transformation matrix 1 for that vertebra. For vertebra 2, i.e., vertebra b2, the computer registers the first mask region 2 and the second mask region 2 to obtain the target transformation matrix 2 for that vertebra. The other target transformation matrices 3 through 6 are obtained similarly and will not be elaborated further here. In other words, for the same vertebra, registering the first and second mask regions of that vertebra yields the corresponding target transformation matrix.

[0097] Further optionally, for each vertebra, the computer device can register the first mask region and the second mask region of the vertebra using methods including but not limited to rigid registration, affine registration, and registration models to obtain the corresponding target transformation matrix.

[0098] S203, based on the target transformation matrix of each vertebra, determine the fusion deformation field between the first modal image and the second modal image; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

[0099] In this embodiment, continuing the example above, after the computer device obtains the target transformation matrix 1 to the target transformation matrix 6, it can determine the fused deformation field between the first modal image and the second modal image based on the target transformation matrix 1 to the target transformation matrix 6.

[0100] The fused deformation field includes both rigid and non-rigid deformation fields between the first and second modal images. The rigid deformation field indicates rigid changes between the first and second modal images, and may include changes in the position and / or orientation of the spine, such as translation or rotation. It is understood that rigid deformation fields are typically used to indicate changes in bony regions of the spine.

[0101] Non-rigid deformation fields are typically used to indicate changes in the deformation or distortion of tissues, spinal cord, intervertebral discs, blood vessels, etc., in the first and second modal images. That is, rigid deformation fields are used to indicate non-rigid changes between the first and second modal images, and can include the displacement field and deformation field of the spine.

[0102] Optionally, the computer device can determine the rigid deformation field between the first modal image and the second modal image based on the target transformation matrix of each vertebra, and determine the non-rigid deformation field between the first modal image and the second modal image. Then, the computer device can obtain the fused deformation field based on the rigid and non-rigid deformation fields. For example, the computer device can perform weighted calculations on the target transformation matrices of each vertebra to determine the rigid deformation field between the first modal image and the second modal image, and determine the non-rigid deformation field between the first modal image and the second modal image using a preset algorithm or deep learning model. Then, the computer device fuses the rigid and non-rigid deformation fields to obtain the fused deformation field. The fusion method can be, but is not limited to, weighted calculation or summation.

[0103] In the above registration method, since the first and second modal images of the spine can be segmented based on the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image, the spine can be automatically segmented using the vertebral segmentation model. Furthermore, for each vertebra, registration of the first and second mask regions allows for registration based on the segmented individual vertebrae, improving the accuracy and efficiency of the obtained target transformation matrix. Furthermore, based on the target transformation matrix of each vertebra, the fusion deformation field between the first and second modal images can be determined for registration. On the one hand, the fusion deformation field includes both rigid and non-rigid deformation fields; therefore, registration of both rigid and non-rigid deformation fields is achieved, improving registration accuracy. On the other hand, registration between multimodal images can be automatically completed without any manual operation, and the images do not need to have the same scanning field of view, improving registration efficiency.

[0104] Figure 4 This is a schematic diagram of a process for determining a target transformation matrix in one embodiment. In an exemplary embodiment, such as... Figure 4 As shown, S202 includes S401 to S403.

[0105] S401, for each vertebra, determine the prior transformation matrix between the first mask region and the second mask region of the vertebra.

[0106] In this embodiment, the computer device determines a corresponding prior transformation matrix for each vertebra. Continuing the example above, for vertebra 1, the computer device determines a prior transformation matrix 1 between the first mask region 1 and the second mask region 2. The other prior transformation matrices are determined similarly to determine prior transformation matrices 1 to 6 for vertebrae 1 to 6 respectively.

[0107] The prior transformation matrix and the target transformation matrix operate on the same principle, both indicating the spatial transformation relationship between the first and second mask regions. Typically, the precision of the prior transformation matrix is ​​less than or equal to the precision of the target transformation matrix.

[0108] Optionally, for each vertebra, the computer device can register the first mask region and the second mask region of the vertebra using methods including but not limited to rigid registration, affine registration, and registration models to obtain a priori transformation matrix.

[0109] S402, based on the prior transformation matrix corresponding to the vertebra, the first mask region and the second mask region of the vertebra are registered to obtain the initial transformation matrix corresponding to the vertebra.

[0110] Continuing with vertebra 1 as an example, the computer device, based on the prior transformation matrix 1, the first mask region 1, and the second mask region 2 corresponding to vertebra 1, can register the first mask region 1 and the second mask region 2 of vertebra 1 to obtain the initial transformation matrix 1 corresponding to vertebra 1. Similarly, for each vertebra, the computer device can register the first mask region and the second mask region of that vertebra based on the prior transformation matrix, the first mask region, and the second mask region to obtain the initial transformation matrix corresponding to that vertebra. In this way, the computer device obtains the initial transformation matrices 1 to 6 corresponding to vertebra 1 to vertebra 6, respectively.

[0111] Optionally, for each vertebra, the computer device uses the prior transformation matrix corresponding to the vertebra as a basis and registers the first mask region and the second mask region of the vertebra to update the prior transformation matrix corresponding to the vertebra, thereby obtaining the initial transformation matrix corresponding to the vertebra. For example, the computer device can use gradient descent algorithms (Powell's algorithm, evolutionary algorithm, etc.) to register the first mask region and the second mask region of the vertebra according to the prior transformation matrix corresponding to the vertebra, the first mask region, and the second mask region to obtain the initial transformation matrix corresponding to the vertebra.

[0112] S403, based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image corresponding to the first mask region of the vertebra in the first modal image, and the second sub-modal image corresponding to the second mask region of the vertebra in the second modal image, the first sub-modal image and the second sub-modal image of the vertebra are registered to obtain the target transformation matrix of the vertebra.

[0113] The first submodal image is the image in the first modal image that corresponds to the first mask region of the vertebra, that is, the image of the vertebra corresponding to that vertebra in the first modal image. The size and position of the first submodal image of the same vertebra can be the same as the first mask region. The first mask region includes only two types of pixel values, while the first submodal image has a richer set of pixel values. Optionally, the computer device can perform cropping processing on the first modal image according to the first mask region to obtain the first submodal image corresponding to each first mask region.

[0114] Similarly, the second submodal image is the image in the second modal image that corresponds to the second mask region of the vertebral body; that is, it is the image of the vertebral body in the second modal image. The size and position of the second submodal image in the second modal image of the same vertebral body are the same as those of the second mask region. The principle of the second submodal image is the same as that of the first submodal image, and will not be repeated here.

[0115] Continue with Figure 3For example, the computer device can determine the first submodal image 1 to the first submodal image 6 corresponding to the first mask region 1 to the first mask region 6, and the second submodal image 1 to the second submodal image 6 corresponding to the second mask region 1 to the second mask region 6.

[0116] Optionally, the first submodal image and the second submodal image of the same vertebra can be the same size. Further optionally, in some embodiments, submodal images corresponding to the same vertebra can overlap. For example, first submodal images 1 to 6 can overlap.

[0117] Taking vertebra 1 as an example, the computer device can register the first submodal image 1 and the second submodal image 1 of vertebra 1 based on the initial transformation matrix 1, the first submodal image 1, and the second submodal image 1 corresponding to vertebra 1, to obtain the target transformation matrix 1 corresponding to vertebra 1. The same principle applies to other vertebrae, and will not be elaborated here.

[0118] The process of obtaining the target transformation matrix can refer to the process of obtaining the initial transformation matrix. Optionally, for each vertebra, the computer device uses the initial transformation matrix corresponding to the vertebra as a basis and registers the first sub-modal image and the second sub-modal image of the vertebra to update and iterate the initial transformation matrix corresponding to the vertebra, thereby obtaining the target transformation matrix corresponding to the vertebra. For example, the computer device can use gradient descent algorithms (Powell's algorithm, evolutionary algorithm, etc.) to register the first mask region and the second mask region of the vertebra based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image, and the second sub-modal image, to obtain the target transformation matrix corresponding to the vertebra.

[0119] In the above embodiments, for each vertebra, the prior transformation matrix between the first mask region and the second mask region of the vertebra is first determined. Then, based on the prior transformation matrix corresponding to the vertebra, the first mask region and the second mask region are registered to obtain the initial transformation matrix corresponding to the vertebra. Finally, based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image in the first modal image corresponding to the first mask region of the vertebra, and the second sub-modal image in the second modal image corresponding to the second mask region of the vertebra, the first sub-modal image and the second sub-modal image of the vertebra are registered to obtain the target transformation matrix of the vertebra. In this way, on the one hand, the initial transformation matrix is ​​obtained by coarse registration first, and the target transformation matrix is ​​obtained by fine registration. On the other hand, the registration is between individual vertebrae. Therefore, the accuracy and efficiency of registration can be improved.

[0120] Figure 5 This is a flowchart illustrating the process of determining the initial transformation matrix in one embodiment. In an exemplary embodiment, such as... Figure 5As shown, S402 includes S501 to S503.

[0121] S501, based on the prior transformation matrix corresponding to the vertebra, perform a geometric transformation on the first mask region of the vertebra to obtain the first transformation region.

[0122] In this embodiment, the geometric transformation may include, but is not limited to, rotational transformation and / or translational transformation. For each vertebra, the computer device can perform a geometric transformation on the first mask region of the vertebra according to the prior transformation matrix corresponding to the vertebra to obtain the corresponding first transformation region.

[0123] For example, for the same vertebra, the computer device multiplies the prior transformation matrix of the vertebra by the first mask region of the vertebra to obtain the first transformation region of the vertebra. As another example, for the same vertebra, the computer device fine-tunes the prior transformation matrix of the vertebra, and multiplies the fine-tuned prior transformation matrix by the first mask region of the vertebra to obtain the first transformation region of the vertebra.

[0124] S502, determine the similarity between the first transformation region and the second mask region of the cone.

[0125] In this embodiment, for the same vertebral body, after obtaining the first transformed region, the similarity between the first transformed region and the second mask region can be determined. The similarity can be measured using methods including, but not limited to, the root mean square error function, mutual information, correlation coefficient, etc., between the first transformed region and the second mask region.

[0126] Taking the cone 1 as an example, assuming that the first transformation region a1 is obtained by performing a geometric transformation on the first mask region 1, the computer device will determine the similarity a1 between the first transformation region a1 and the second mask region 1 of the cone.

[0127] S503, based on similarity, register the first and second mask regions of the vertebra to obtain the initial transformation matrix corresponding to the vertebra.

[0128] Optionally, after determining the similarity, if the first stopping condition is not met, the computer device can update the prior transformation matrix based on the similarity, and perform a geometric transformation on the first mask region of the cone according to the updated prior transformation matrix to obtain a new first transformation region. It then returns to the step of determining the similarity between the first transformation region and the second mask region of the cone, repeating this iterative process until the first stopping condition is met, and using the prior transformation matrix at the point where the stopping condition is met as the initial transformation matrix. The first stopping condition can be that the similarity between the first transformation region and the second mask region is less than a first threshold, or that the number of iterations for updating the prior transformation matrix is ​​greater than a preset number.

[0129] Continuing with the example above, taking the first stopping condition being that the similarity is less than a first threshold as an example, after the computer device determines the similarity a1, if the similarity a1 is not less than the first threshold, the computer device updates the prior transformation matrix 1, and performs a geometric transformation on the first mask region 1 according to the updated prior transformation matrix 1 to obtain the first transformation region a2, and determines the similarity a2 between the first transformation region a2 and the second mask region 1. If the similarity a2 is still not less than the first threshold, the computer device continues to update the prior transformation matrix 1, and performs a geometric transformation on the first mask region 1 according to the updated prior transformation matrix 1 to obtain the first transformation region a3, and determines the similarity a3 between the first transformation region a3 and the second mask region 1. If the similarity a1 is less than the first threshold, the computer device uses the updated prior transformation matrix at this time as the initial transformation matrix 1.

[0130] In the above embodiments, since the first mask region of the vertebra can be geometrically transformed to obtain the first transformation region based on the prior transformation matrix corresponding to the vertebra, and the similarity between the first transformation region and the second mask region of the vertebra can be determined, the first mask region and the second mask region of the vertebra can be registered based on the similarity, so that the initial transformation matrix corresponding to a single vertebra can be obtained efficiently, and it is also beneficial to further register the target transformation matrix.

[0131] Figure 6 This is a schematic diagram of another process for determining the target transformation matrix in one embodiment. In an exemplary embodiment, such as... Figure 6 As shown, S403 includes S601 to S603.

[0132] S601, based on the initial transformation matrix corresponding to the vertebra, perform geometric transformation on the first submodal image of the vertebra to obtain the second transformation region.

[0133] In this embodiment, similarly, for each vertebra, the computer device can perform geometric transformation on the first sub-modal image of the vertebra according to the initial transformation matrix corresponding to the vertebra to obtain the corresponding second transformation region. The process of S601 can be referred to S501, and will not be repeated here.

[0134] S602, determine the similarity between the second transformed region and the second submodal image of the vertebral body.

[0135] Similarly, the principle of S602 is the same as that of S502. Continuing with the example of cone 1, assuming that a second transformed region b1 is obtained by geometric transformation of the first submodal image 1, the computer device will determine the similarity b1 between the first transformed region b1 and the second submodal image 1.

[0136] S603, registration is performed based on the similarity between the second transformation region and the second submodal image of the vertebral body to obtain the target transformation matrix corresponding to the vertebral body.

[0137] S603 operates on the same principle as S503. Optionally, after determining the similarity, if the second stopping condition is not met, the computer device can update the initial transformation matrix based on the similarity. Then, based on the updated initial transformation matrix, it performs a geometric transformation on the first sub-modal image of the cone to obtain a new second transformation region. It then returns to the step of determining the similarity between the second transformation region and the second sub-modal image of the cone. This process is iterated repeatedly until the second stopping condition is met, and the initial transformation matrix at which the stopping condition is met is taken as the target transformation matrix. The second stopping condition can be that the similarity between the second transformation region and the second sub-modal image is less than a second threshold, or that the number of iterations for updating the initial transformation matrix is ​​greater than a preset number.

[0138] In the above embodiments, since the first submodal image of the vertebra can be geometrically transformed to obtain the second transformation region based on the initial transformation matrix corresponding to the vertebra, and the similarity between the second transformation region and the second submodal image of the vertebra can be determined, the target transformation matrix corresponding to a single vertebra can be obtained more accurately by registering based on the similarity between the second transformation region and the second submodal image of the vertebra.

[0139] Figure 7 This is a flowchart illustrating the process of determining the prior transformation matrix in one embodiment. In an exemplary embodiment, such as... Figure 7 As shown, S401 includes S701 to S702.

[0140] S701, determine the first preset position of the first mask region of the vertebral body and the second preset position corresponding to the first preset position in the second mask region.

[0141] In this embodiment, the first preset position and the second preset position can be the same preset position. The preset position can be the centroid, the center, or other positions. Taking the centroid as an example, for the same vertebra, the first preset position is the centroid in the first masking region of the vertebra, and the second preset position is the centroid in the second masking region of the vertebra.

[0142] S702, determine the prior transformation matrix based on the first preset position and the second preset position.

[0143] In this embodiment, optionally, for each vertebra, the computer device aligns the first preset position with the second preset position to determine the prior transformation matrix corresponding to that vertebra. That is, the computer device can align the first preset position in the first mask region with the second preset position in the second mask region, and determine the prior transformation matrix based on the direction and displacement of movement during the alignment process.

[0144] Optionally, the computer device can determine the differences in each direction of the coordinate system between the first preset position in the first mask region and the second preset position in the second mask region, and determine the prior transformation matrix of the cone based on these differences. For example, the computer device can use the differences in the X, Y, and Z directions of the coordinate system between the centroid of the first mask region 1 and the centroid of the second mask region 1 as the translation vectors of the prior transformation matrix of the cone 1. In this way, the computer device can determine how much displacement in each of the X, Y, and Z directions is needed to align the first and second preset positions.

[0145] In the above embodiments, since a first preset position in the first mask region of the vertebral body and a second preset position in the second mask region corresponding to the first preset position are determined, the prior transformation matrix can be efficiently determined based on the first preset position and the second preset position, which is beneficial for obtaining the initial transformation matrix.

[0146] Figure 8 This is a schematic diagram illustrating the process of determining the target transformation matrix in one embodiment, such as... Figure 8 As shown, for each vertebra, after the computer device determines the first mask region and the second mask region, center alignment can be performed. Center alignment is also known as… Figure 7 The process shown aims to obtain the prior transformation matrix of the vertebral body. After obtaining the prior transformation matrix, the computer device performs mask rigid coarse registration, which is also known as mask rigid coarse registration. Figure 5 The process shown aims to obtain the initial transformation matrix of the vertebral body. Afterwards, the computer equipment performs fine registration. Fine registration is the process of geometrically transforming the first submodal image of the vertebral body according to the initial transformation matrix to obtain the second transformation region, and then using the second transformation region and the second submodal image to determine the target transformation matrix of the vertebral body. This process can be referenced... Figure 6 .

[0147] In the above process, the initial transformation matrix of the cone is obtained based on the first and second mask regions, which contain only two types of pixel values, thus improving the efficiency of obtaining the initial transformation matrix. The process of obtaining the target transformation matrix is ​​based on the first and second sub-modal images, which contain richer information about pixel values, thereby improving the accuracy of obtaining the target transformation matrix.

[0148] Figure 9 This is a schematic diagram of a process for determining the fused deformation field in one embodiment. In an exemplary embodiment, such as... Figure 9 As shown, S203 includes S901 to S903.

[0149] S901, Generate Gaussian distribution maps corresponding to the second submodal images of each vertebra; the Gaussian distribution maps are used to characterize the probability distribution of the rigidity of each pixel in the second submodal image.

[0150] In this embodiment, the second sub-modal image of the vertebral body is the image corresponding to the second mask region of the vertebral body in the second modal image of the vertebral body. The description of the second sub-modal image can be found in S403 above, and will not be repeated here.

[0151] The computer equipment can generate a Gaussian distribution map corresponding to each second submodal image. The Gaussian distribution map is used to characterize the probability distribution of rigidity of each pixel in the second submodal image. That is, the Gaussian distribution map of the same cone is the same size as the second submodal image, includes all pixels of the second submodal image, and the pixel value of each pixel in the Gaussian distribution map is the rigidity probability value corresponding to that pixel. The rigidity probability value indicates the probability that the corresponding pixel belongs to a rigid region.

[0152] Optionally, the rigidity probability value of a pixel in the second submodal image is related to the distance between the pixel and the region where the cone is located. Further, optionally, the rigidity probability value of a pixel is negatively correlated with the distance between the pixel and the cone. In other words, the closer a pixel in the second submodal image is to the region where the cone is located, the higher the rigidity probability value of that pixel.

[0153] The target transformation matrix obtained after the above segmented rigid registration only reflects the rigid transformation of the bony region of the vertebral body. Regions outside the vertebral body, such as blood vessels, spinal cord, and intervertebral discs, are elastic deformation areas. Since spinal registration focuses on the surrounding area, and the deformation of the surrounding soft tissues is influenced by the vertebrae, the rigid transformation of a single vertebra can be extended to other non-rigid soft tissue regions using a Gaussian probability distribution weighted approach. This results in regions closer to the vertebral body exhibiting stronger "rigidity," while regions farther from the vertebral body exhibit weaker "rigidity." Therefore, optionally, the computer device can generate Gaussian distribution maps corresponding to the second submodal images of each vertebra based on a Gaussian distribution function.

[0154] S902, based on the target transformation matrix of each vertebra, determines the rigid deformation field of each vertebra.

[0155] In this embodiment, the computer device determines the rigid deformation field of each vertebra. The rigid deformation field of the vertebra is used to indicate the change in rigidity of the vertebra between the first modal image and the second modal image. Continuing the example above, the computer device determines the rigid deformation field 1 of vertebra 1 based on the target transformation matrix 1, and so on.

[0156] Optionally, for each vertebra, the computer device can determine the translation vector and rotation vector based on the target transformation matrix of the vertebra, and obtain the rigid deformation field of the vertebra based on the translation vector and rotation vector. For example, the computer device can determine the displacement field of the rigid deformation field based on the translation vector, and determine the deformation field of the rigid deformation field based on the rotation vector.

[0157] S903, based on the rigid deformation field and Gaussian distribution map of each vertebra, determine the fused deformation field between the first modal image and the second modal image.

[0158] In this embodiment, optionally, the computer device inputs the rigid deformation field and Gaussian distribution map of each vertebra into the trained fusion model to obtain the fused deformation field between the first modal image and the second modal image.

[0159] In the above embodiments, since Gaussian distribution maps corresponding to the second submodal images of each vertebra can be generated, and these Gaussian distribution maps are used to characterize the probability distribution of the rigidity of each pixel in the second submodal image, and the rigid deformation field of each vertebra can be determined based on the target transformation matrix of each vertebra, the fused deformation field between the first modal image and the second modal image can be determined efficiently based on the rigid deformation field and Gaussian distribution maps of each vertebra.

[0160] Figure 10 This is a flowchart illustrating the process of determining a rigid deformation field in one embodiment. In an exemplary embodiment, such as... Figure 10 As shown, S902 includes S1001 to S1003.

[0161] S1001, for each vertebra, determine the first coordinates of each pixel in the first submodal image of the vertebra.

[0162] In this embodiment, the first submodal image of the vertebral body is the image corresponding to the first mask region of the vertebral body in the first modal image of the vertebral body. The description of the first submodal image can be referred to in S403 above, and will not be repeated here.

[0163] For each vertebra, the computer device can obtain the first coordinates of each pixel in the first submodal image corresponding to that vertebra. Taking vertebra 1 as an example, the computer device can obtain the first coordinates of each pixel in the first submodal image 1.

[0164] S1002, for each pixel, determine the second coordinate of the pixel based on the product between the first coordinate of the pixel and the corresponding element in the target transformation matrix.

[0165] In this embodiment, the second coordinates are the coordinates of each pixel in the second submodal image. The target transformation matrix may include elements corresponding to each pixel in the first or second submodal image.

[0166] Continuing with the example of cone 1, assuming that both the first submodal image 1 and the second submodal image 1 include pixels A to C, the computer device can determine the second coordinate of pixel A in the second submodal image 1 based on the product of the first coordinate of pixel A in the first submodal image 1 and the corresponding element of pixel A in the target transformation matrix. For example, the computer device can directly use the product of the first coordinate of pixel A and the corresponding element of pixel A in the target transformation matrix as the second coordinate of pixel A in the second submodal image 1.

[0167] The same logic applies to pixels B and C, which will not be elaborated further here. In this way, the computer device determines the first and second coordinates corresponding to pixels A through C, respectively. It can be understood that the first coordinate is the pixel's coordinate in the first submodal image, and the second coordinate is the pixel's coordinate in the second submodal image.

[0168] S1003, determine the rigid deformation field of the cone based on the first and second coordinates of each pixel.

[0169] In this embodiment, the rigid deformation field of the vertebra can be used to indicate the amount of change of corresponding pixels between the first submodal image and the second submodal image of the vertebra.

[0170] Optionally, the computer device can determine the rigid deformation field of the cone based on the change between the first and second coordinates of each pixel. The change between the first and second coordinates can also be understood as the change between the first and second sub-modal images of the same cone at corresponding pixels, i.e., at corresponding positions.

[0171] Optionally, the computer device uses the change between the first and second coordinates of each pixel as the pixel value of the corresponding pixel in the rigid deformation field. Continuing with the example of cone 1, the computer device determines the change in pixel A based on its first and second coordinates. Similarly, the computer device can also determine the changes in pixel B and pixel C. Then, the computer device can determine the rigid deformation field of cone 1 based on the changes in pixels A through C. That is, the rigid deformation field of cone 1 includes pixels A through C, and in the rigid deformation field of cone 1, the pixel value corresponding to pixel A is the change in pixel A, the pixel value corresponding to pixel B is the change in pixel B, and the pixel value corresponding to pixel C is the change in pixel C.

[0172] In the above embodiments, since the first coordinates of each pixel in the first submodal image of each vertebra are determined for each vertebra, and the second coordinates of each pixel can be determined by the product between the first coordinates of the pixel and the corresponding element in the target transformation matrix, the rigid deformation field of the vertebra can be determined efficiently and accurately based on the first and second coordinates of each pixel.

[0173] Figure 11 This is a schematic diagram of another process for determining the fused deformation field in one embodiment. In an exemplary embodiment, such as... Figure 11 As shown, S903 includes S1101 to S1103.

[0174] S1101, for each vertebra, determine the aggregate diagram of the vertebra based on the rigid deformation field and Gaussian distribution map of the vertebra.

[0175] In this embodiment, optionally, the computer device can fuse the rigid deformation field and Gaussian distribution map of the same vertebra to obtain a composite map of the vertebra. The fusion method may include, but is not limited to, weighted calculation or summation.

[0176] In an exemplary embodiment, the computer device can also determine the aggregated image of the cone based on the product of the pixel value of a first pixel in the rigid deformation field of the cone and the pixel value of a second pixel in the Gaussian distribution map of the cone. Here, the first pixel corresponds to the second pixel; that is, the position of the first pixel in the rigid deformation field is the same as the position of the second pixel in the Gaussian distribution map. In other words, the computer device can multiply the rigid deformation field of the cone and the Gaussian distribution map point-by-point at the corresponding pixels to obtain the aggregated image of the cone.

[0177] For example, the rigid deformation field of cone 1 includes pixels A, B, and C, and the Gaussian distribution map of cone 1 also includes pixels A, B, and C. Thus, the computer device multiplies the pixel value of pixel A in the rigid deformation field of cone 1 with the pixel value of pixel A in the Gaussian distribution map of cone 1 to obtain the pixel value of pixel A in the aggregated map of cone 1, and so on. Afterward, the computer device obtains an aggregated map of each cone with the same size as the Gaussian distribution map.

[0178] In the above embodiments, since the first pixel corresponds to the second pixel, the aggregate map of the vertebra can be determined efficiently and accurately by multiplying the pixel value of the first pixel in the rigid deformation field of the vertebra with the pixel value of the second pixel in the Gaussian distribution map of the vertebra.

[0179] S1102, obtain the Gaussian distribution map of the second modality image based on the Gaussian distribution map of each vertebra, and obtain the aggregation map of the second modality image based on the aggregation map of each vertebra.

[0180] In this embodiment, optionally, the computer device can arrange the Gaussian distribution maps of each vertebra in the second modal image to obtain the Gaussian distribution map of the second modal image, and arrange the aggregated maps of each vertebra to obtain the aggregated map of the second modal image. Figure 3 For example, computer equipment can be arranged in the order of vertebrae 1 to vertebrae 6, with Gaussian distributions of vertebrae 1 to vertebrae 6 arranged sequentially. Figure 1 The Gaussian distribution map of the second modality image is obtained. Similarly, the computer device arranges the aggregated images of vertebrae 1 to vertebrae 6 in the order of vertebrae 1 to vertebrae 6 to obtain the aggregated image of the second modality image.

[0181] In some embodiments, the aggregated image of the second modality is the same size as the second modality image. The position of each vertebra in the second modality image can be the position of the second sub-modality image of the vertebra in the second modality image. In this way, the computer device can arrange the Gaussian distribution images of each vertebra in the second modality image to obtain the Gaussian distribution image of the second modality image, and arrange the aggregated images of each vertebra to obtain the aggregated image of the second modality image.

[0182] For example, the Gaussian distribution map of each vertebra is placed at the second submodal image of that vertebra to obtain the Gaussian distribution map of the second modal image, and the aggregate map of each vertebra is placed at the second submodal image of that vertebra to obtain the aggregate map of the second modal image.

[0183] In the above embodiments, since the Gaussian distribution map and the aggregate map of each vertebra are arranged according to the position of each vertebra in the second modal image, the Gaussian distribution map and the aggregate map of the second modal image can be obtained accurately according to the position of each vertebra.

[0184] S1103, Determine the fused deformation field based on the Gaussian distribution map and the aggregation map of the second modal image.

[0185] In this embodiment, the computer device can fuse the Gaussian distribution map of the second modality image and the aggregate map of the second modality image to determine the fused deformation field. The fusion method may include, but is not limited to, weighted calculation or summation.

[0186] In an exemplary embodiment, similar to the principle of S1101, the computer device can also determine the fused deformation field based on the product of the pixel value of the third pixel in the Gaussian distribution map of the second modality image and the pixel value of the fourth pixel in the aggregated map of the second modality image. Here, the third pixel corresponds to the fourth pixel; that is, the position of the third pixel in the Gaussian distribution map is the same as the position of the fourth pixel in the aggregated map. In other words, the computer device can multiply the Gaussian distribution map and the aggregated map of the second modality image point by point at the corresponding pixels to efficiently and accurately obtain the determined fused deformation field.

[0187] In the above embodiments, for each vertebra, the aggregation map of the vertebra is determined based on the rigid deformation field and Gaussian distribution map of the vertebra, and the Gaussian distribution map of the second modality image is obtained based on the Gaussian distribution map of each vertebra, and the aggregation map of the second modality image is obtained based on the aggregation map of each vertebra. In this way, based on the Gaussian distribution map of the second modality image and the aggregation map of the second modality image, multiple rigid deformation regions can be extended to other soft tissue regions and then weighted and fused to form a new elastic deformation field of the whole image, so as to determine a more accurate fused deformation field.

[0188] Figure 12 This is a schematic diagram of the process for determining a first mask region and a second mask region in one embodiment. In an exemplary embodiment, such as... Figure 12 As shown, S201 includes S1201 to S1202.

[0189] S1201, preprocess the third modal image to obtain the first modal image, and preprocess the fourth modal image to obtain the second modal image.

[0190] In this embodiment, optionally, the computer device can acquire third-modality and fourth-modality images of the spine. Optionally, the computer device can acquire third-modality or fourth-modality images sent by a medical scanning device, or it can acquire third-modality or fourth-modality images from other devices or storage.

[0191] It is understandable that the third modality of the image is the same as that of the first modality, and the fourth modality of the image is the same as that of the second modality. For example, the third modality can be an MR image, and the fourth modality can be a CT image.

[0192] Preprocessing includes, but is not limited to, cropping, noise reduction, filtering, enhancement, and resolution adjustment. The method of preprocessing the first modality image can be the same as or different from the method of preprocessing the second modality image. For example, a computer device can reduce the resolution of the first and second modality images respectively to obtain a third and a fourth modality image.

[0193] S1202, input the first modal image into the vertebral segmentation model to obtain each first mask region, and input the second modal image into the vertebral segmentation model to obtain each second mask region.

[0194] Furthermore, the computer device inputs the first modal image into the vertebral segmentation model to obtain the first mask region corresponding to each vertebra output by the vertebral segmentation model, and inputs the second modal image into the vertebral segmentation model to obtain the second mask region corresponding to each vertebra output by the vertebral segmentation model.

[0195] In the above embodiments, since the third modal image can be preprocessed to obtain the first modal image, and the fourth modal image can be preprocessed to obtain the second modal image, after inputting the first modal image into the vertebral segmentation model and the second modal image into the vertebral segmentation model, each first mask region and each second mask region can be obtained, thereby determining the region corresponding to a single vertebra in the two modalities. In the above process, it is not necessary for the images of each modality to have the same scanning field of view, and it is also beneficial for the registration of a single vertebra.

[0196] Figure 13 Here is a flowchart of a registration method in one embodiment, such as... Figure 13As shown, continuing with the example of using the first modality image as an MR image and the second modality image as a CT image, this embodiment employs a registration method combining deep learning segmentation and traditional registration. First, a vertebral segmentation model is trained using deep learning. This model is then used to segment the first and second modal images into single vertebral bodies to determine the first and second mask regions. Next, rigid registration of the segmented single vertebral bodies is performed based on the first and second mask regions to obtain the target transformation matrix for each vertebra. Then, the rigid deformation field for each vertebra is obtained based on its target transformation matrix, for example, by converting the rotation matrix in the target transformation matrix into the deformation field. Finally, the rigid deformation field is extended to other soft tissue regions and then weighted and fused to form a new elastic deformation field for the entire image, i.e., the fused deformation field.

[0197] Figure 14 This is a schematic diagram of the registration method in one embodiment, such as... Figure 14 As shown, computer devices can execute this registration method according to the following procedure.

[0198] S1401, preprocess the third modal image to obtain the first modal image, and preprocess the fourth modal image to obtain the second modal image.

[0199] S1402, input the first modal image into the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image, and input the second modal image into the vertebral segmentation model to obtain the second mask region of each vertebra in the second modal image.

[0200] S1403, for each vertebra, determine the first preset position of the first mask region of the vertebra and the second preset position corresponding to the first preset position in the second mask region, and determine the prior transformation matrix based on the first preset position and the second preset position.

[0201] S1404. Based on the prior transformation matrix corresponding to the vertebra, perform a geometric transformation on the first mask region of the vertebra to obtain the first transformation region. Determine the similarity between the first transformation region and the second mask region of the vertebra. Based on the similarity, register the first mask region and the second mask region of the vertebra to obtain the initial transformation matrix corresponding to the vertebra.

[0202] S1405, based on the initial transformation matrix corresponding to the vertebral body, perform geometric transformation on the first submodal image of the vertebral body to obtain the second transformation region.

[0203] S1406, determine the similarity between the second transformed region and the second submodal image of the vertebral body.

[0204] S1407, registration is performed based on the similarity between the second transformation region and the second submodal image of the vertebral body to obtain the target transformation matrix corresponding to the vertebral body.

[0205] S1408, Generate Gaussian distribution maps corresponding to the second sub-modal images of each vertebra. The Gaussian distribution map is used to characterize the probability distribution of rigidity of each pixel in the second sub-modal image.

[0206] S1409, for each vertebra, determine the first coordinates of each pixel in the first submodal image of the vertebra.

[0207] S1410, For each pixel, determine the second coordinate of the pixel based on the product between the first coordinate of the pixel and the corresponding element in the target transformation matrix.

[0208] S1411, determine the rigid deformation field of the cone based on the first and second coordinates of each pixel.

[0209] S1412, For each vertebra, the aggregated image of the vertebra is determined based on the product of the pixel value of the first pixel in the rigid deformation field of the vertebra and the pixel value of the second pixel in the Gaussian distribution map of the vertebra. The first pixel corresponds to the second pixel.

[0210] S1413, based on the position of each vertebra in the second modal image, arrange the Gaussian distribution map of each vertebra to obtain the Gaussian distribution map of the second modal image, and arrange the aggregation map of each vertebra to obtain the aggregation map of the second modal image.

[0211] S1414, the fused deformation field is determined based on the product of the pixel value of the third pixel in the Gaussian distribution map of the second modality image and the pixel value of the fourth pixel in the aggregated map of the second modality image. The third pixel corresponds to the fourth pixel.

[0212] The steps S1401 to S1414 can be referred to in the above embodiments, and will not be repeated here.

[0213] Figure 15 This is a registration result image. Figure 16 This is another registration result diagram. Figure 15 The image shows a grayscale overlay after registration of CT and MR multimodal images, where... Figure 15 (a) is a diagram showing the effect of CT and MR images of the cervical spine after registration. Figure 15 (b) The effect after registration with CT and MR images of the neck and chest.

[0214] Figure 16 This shows a multimodal overlay effect diagram of the lumbar spine. Figure 16 (a) shows the contour display effect after CT and MR images are registered. Figure 16 (b) shows the checkerboard display effect after the CT and MR images are registered.

[0215] In summary, firstly, this application presents a method for generating a global elastic transformation based on Gaussian weighted multi-rigidity transformation fusion. This method effectively fuses multiple three-dimensional rigid deformation fields into a single deformation field that simultaneously matches the individual deformation fields of all vertebrae in the fused image. This ensures both vertebral rigidity and preserves the natural flexibility of the surrounding soft tissue regions, resulting in faster operation compared to other interpolation fusion methods. Secondly, existing technologies utilize deep learning for end-to-end registration, which requires high data distribution and volume, and is difficult to learn. In contrast, the vertebral segmentation model in this application only needs to segment the vertebrae, making it easier to learn and requiring no paired or identically viewed images. Thirdly, this application presents a fully automated registration method that eliminates the need for manual interaction and can adapt to any scanning field of view, significantly improving the software's applicability and ease of use.

[0216] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0217] Based on the same inventive concept, this application also provides a registration apparatus for implementing the registration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more registration apparatus embodiments provided below can be found in the limitations of the registration method described above, and will not be repeated here.

[0218] Figure 17 This is a structural block diagram of a registration device in one embodiment. In an exemplary embodiment, such as... Figure 17 As shown, a registration device 1700 is provided, including: a segmentation module 1701, a registration module 1702, and a determination module 1703, wherein:

[0219] The segmentation module 1701 is used to segment the first modal image and the second modal image of the spine based on the vertebral body segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image.

[0220] The registration module 1702 is used to register the first mask region and the second mask region of each vertebra to obtain the target transformation matrix.

[0221] The determination module 1703 is used to determine the fusion deformation field between the first modal image and the second modal image based on the target transformation matrix of each vertebra; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

[0222] In the aforementioned registration device, since the first and second modal images of the spine can be segmented based on the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image, the spine can be automatically segmented using the vertebral segmentation model. Furthermore, for each vertebra, registration of the first and second mask regions allows for registration based on the segmented individual vertebrae, improving the accuracy and efficiency of the obtained target transformation matrix. Furthermore, based on the target transformation matrix of each vertebra, the fused deformation field between the first and second modal images can be determined for registration. On the one hand, the fused deformation field includes both rigid and non-rigid deformation fields; therefore, registration of both rigid and non-rigid deformation fields is achieved, improving registration accuracy. On the other hand, registration between multimodal images can be automatically completed without any manual operation, and the images do not need to have the same scanning field of view, improving registration efficiency.

[0223] Optionally, the registration module 1702 includes:

[0224] The first determining unit is used to determine, for each vertebra, the prior transformation matrix between the first mask region and the second mask region of the vertebra.

[0225] The first registration unit is used to register the first and second mask regions of the vertebra based on the prior transformation matrix corresponding to the vertebra, the first mask region, and the second mask region, to obtain the initial transformation matrix corresponding to the vertebra.

[0226] The second registration unit is used to register the first sub-modal image and the second sub-modal image of the vertebra based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image in the first modal image corresponding to the first mask region of the vertebra, and the second sub-modal image in the second modal image corresponding to the second mask region of the vertebra, to obtain the target transformation matrix of the vertebra.

[0227] Optionally, the first registration unit is used to perform a geometric transformation on the first mask region of the vertebra based on the prior transformation matrix corresponding to the vertebra to obtain a first transformation region; determine the similarity between the first transformation region and the second mask region of the vertebra; and register the first mask region and the second mask region of the vertebra based on the similarity to obtain the initial transformation matrix corresponding to the vertebra.

[0228] Optionally, the second registration unit is used to perform geometric transformation on the first sub-modal image of the vertebral body according to the initial transformation matrix corresponding to the vertebral body to obtain the second transformation region; determine the similarity between the second transformation region and the second sub-modal image of the vertebral body; and perform registration based on the similarity between the second transformation region and the second sub-modal image of the vertebral body to obtain the target transformation matrix corresponding to the vertebral body.

[0229] Optionally, the first determining unit is used to determine a first preset position in the first mask region of the vertebra and a second preset position in the second mask region corresponding to the first preset position; and to determine a priori transformation matrix based on the first preset position and the second preset position.

[0230] Optionally, the determination module 1703 includes:

[0231] The generation unit is used to generate a Gaussian distribution map corresponding to the second submodal image of each vertebra; the Gaussian distribution map is used to characterize the probability distribution of the rigidity of each pixel in the second submodal image; the second submodal image of the vertebra is the image corresponding to the second mask region of the vertebra in the second modal image of the vertebra.

[0232] The second determining unit is used to determine the rigid deformation field of each vertebra based on the target transformation matrix of each vertebra.

[0233] The third determining unit is used to determine the fused deformation field between the first modal image and the second modal image based on the rigid deformation field and Gaussian distribution map of each vertebra.

[0234] Optionally, the second determining unit is used to determine the first coordinates of each pixel in the first submodal image of the vertebra for each vertebra; for each pixel, determine the second coordinates of the pixel based on the product between the first coordinates of the pixel and the corresponding element in the target transformation matrix; and determine the rigid deformation field of the vertebra based on the first coordinates and the second coordinates of each pixel; the first submodal image of the vertebra is the image in the first modal image of the vertebra that corresponds to the first mask region of the vertebra.

[0235] Optionally, the third determining unit includes:

[0236] The first defined sub-unit is used to determine the aggregate diagram of each vertebra based on the rigid deformation field and Gaussian distribution map of the vertebra.

[0237] The second determining subunit is used to obtain the Gaussian distribution map of the second modality image based on the Gaussian distribution map of each vertebra, and to obtain the aggregate map of the second modality image based on the aggregate map of each vertebra.

[0238] The third determining sub-unit is used to determine the fused deformation field based on the Gaussian distribution map and the aggregation map of the second modal image.

[0239] Optionally, the first determining sub-unit is used to determine the aggregate map of the cone based on the product between the pixel value of the first pixel in the rigid deformation field of the cone and the pixel value of the second pixel in the Gaussian distribution map of the cone; the first pixel corresponds to the second pixel.

[0240] Optionally, the second determining subunit is used to arrange the Gaussian distribution maps of each vertebra in the second modal image to obtain the Gaussian distribution map of the second modal image, and to arrange the aggregation map of each vertebra to obtain the aggregation map of the second modal image.

[0241] Optionally, the third determining subunit is used to determine the fused deformation field based on the product between the pixel value of the third pixel in the Gaussian distribution map of the second modal image and the pixel value of the fourth pixel in the aggregate map of the second modal image; the third pixel corresponds to the fourth pixel.

[0242] Optionally, the segmentation module 1701 includes:

[0243] The preprocessing unit is used to preprocess the third modal image to obtain the first modal image, and to preprocess the fourth modal image to obtain the second modal image.

[0244] The segmentation unit is used to input the first modal image into the vertebral segmentation model to obtain each first mask region, and to input the second modal image into the vertebral segmentation model to obtain each second mask region.

[0245] Each module in the above-mentioned registration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0246] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0247] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0248] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0249] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0250] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0251] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A registration method, characterized in that, The method includes: Based on the vertebral segmentation model, the first modal image and the second modal image of the spine are segmented to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image. For each of the vertebrae, the first mask region and the second mask region of the vertebrae are registered to obtain the target transformation matrix; Based on the target transformation matrix of each vertebra, the fusion deformation field between the first modal image and the second modal image is determined; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

2. The method according to claim 1, characterized in that, For each of the vertebrae, the first mask region and the second mask region of the vertebrae are registered to obtain the target transformation matrix, including: For each of the vertebrae, determine the prior transformation matrix between the first mask region and the second mask region of the vertebrae; Based on the prior transformation matrix corresponding to the vertebra, the first mask region and the second mask region of the vertebra are registered to obtain the initial transformation matrix corresponding to the vertebra. Based on the initial transformation matrix corresponding to the vertebra, the first sub-modal image in the first modal image corresponding to the first mask region of the vertebra, and the second sub-modal image in the second modal image corresponding to the second mask region of the vertebra, the first sub-modal image and the second sub-modal image of the vertebra are registered to obtain the target transformation matrix of the vertebra.

3. The method according to claim 2, characterized in that, The step of registering the first sub-modal image and the second sub-modal image of the vertebral body according to the initial transformation matrix corresponding to the vertebral body, the first sub-modal image in the first modal image corresponding to the first mask region of the vertebral body, and the second sub-modal image in the second modal image corresponding to the second mask region of the vertebral body to obtain the target transformation matrix of the vertebral body includes: Based on the initial transformation matrix corresponding to the vertebral body, a geometric transformation is performed on the first submodal image of the vertebral body to obtain the second transformation region; Determine the similarity between the second transformed region and the second submodal image of the vertebral body; Registration is performed based on the similarity between the second transformation region and the second submodal image of the vertebral body to obtain the target transformation matrix corresponding to the vertebral body.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the fusion deformation field between the first modal image and the second modal image based on the target transformation matrix of each of the vertebrae includes: A Gaussian distribution map corresponding to the second sub-modal image of each of the cones is generated; the Gaussian distribution map is used to characterize the probability distribution of the rigidity of each pixel in the second sub-modal image; the second sub-modal image of the cone is the image corresponding to the second mask region of the cone in the second modal image of the cone; Based on the target transformation matrix of each vertebra, the rigid deformation field of each vertebra is determined; Based on the rigid deformation field and Gaussian distribution map of each vertebra, the fused deformation field between the first modal image and the second modal image is determined.

5. The method according to claim 4, characterized in that, The determination of the rigid deformation field of each vertebra based on the target transformation matrix of each vertebra includes: For each of the vertebrae, the first coordinates of each pixel in the first sub-modal image of the vertebrae are determined; the first sub-modal image of the vertebrae is the image in the first modal image of the vertebrae that corresponds to the first mask region of the vertebrae. For each pixel, the second coordinate of the pixel is determined by the product of the first coordinate of the pixel and the element corresponding to the pixel in the target transformation matrix; The rigid deformation field of the cone is determined based on the first and second coordinates of each pixel.

6. The method according to claim 5, characterized in that, The step of determining the fused deformation field between the first modal image and the second modal image based on the rigid deformation field and Gaussian distribution map of each vertebra includes: For each of the vertebrae, a composite diagram of the vertebrae is determined based on the rigid deformation field and Gaussian distribution map of the vertebrae; The Gaussian distribution map of the second modality image is obtained based on the Gaussian distribution map of each of the vertebrae, and the aggregate map of the second modality image is obtained based on the aggregate map of each of the vertebrae; The fused deformation field is determined based on the Gaussian distribution map of the second modal image and the aggregate map of the second modal image.

7. The method according to claim 6, characterized in that, The step of determining the convergence diagram of the vertebral body based on the rigid deformation field and Gaussian distribution map of the vertebral body includes: The aggregate map of the vertebra is determined by multiplying the pixel value of the first pixel in the rigid deformation field of the vertebra with the pixel value of the second pixel in the Gaussian distribution map of the vertebra; the first pixel corresponds to the second pixel.

8. The method according to claim 6, characterized in that, The step of obtaining the Gaussian distribution map of the second modality image based on the Gaussian distribution map of each of the vertebrae, and obtaining the aggregated map of the second modality image based on the aggregated map of each of the vertebrae, includes: Based on the position of each vertebra in the second modal image, Gaussian distribution maps of each vertebra are arranged to obtain a Gaussian distribution map of the second modal image, and aggregate maps of each vertebra are arranged to obtain an aggregate map of the second modal image.

9. The method according to claim 6, characterized in that, Determining the fused deformation field based on the Gaussian distribution map of the second modal image and the aggregate map of the second modal image includes: The fused deformation field is determined by multiplying the pixel value of the third pixel in the Gaussian distribution map of the second modal image with the pixel value of the fourth pixel in the aggregate map of the second modal image; the third pixel corresponds to the fourth pixel.

10. The method according to any one of claims 1-3, characterized in that, The segmentation of the first and second modal images of the spine based on the vertebral body segmentation model yields a first mask region for each vertebra in the first modal image and a second mask region for each vertebra in the second modal image, including: The first modal image is obtained by preprocessing the third modal image, and the second modal image is obtained by preprocessing the fourth modal image. The first modal image is input into the vertebral segmentation model to obtain each of the first mask regions, and the second modal image is input into the vertebral segmentation model to obtain each of the second mask regions.

11. A registration device, characterized in that, The device includes: The segmentation module is used to segment the first modal image and the second modal image of the spine based on the vertebral segmentation model to obtain the first mask region of each vertebra in the first modal image and the second mask region of each vertebra in the second modal image. The registration module is used to register the first mask region and the second mask region of each vertebra to obtain the target transformation matrix. The determination module is used to determine the fusion deformation field between the first modal image and the second modal image based on the target transformation matrix of each of the vertebrae; the fusion deformation field is used to register the first modal image and the second modal image, and the fusion deformation field includes a rigid deformation field and a non-rigid deformation field.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images