MR Image and CT Image Registration Method, Apparatus, Computer Device, and Storage Medium

Abstract

The present invention is applicable to the field of computers and provides a method for registering MR images and CT images, including: obtaining the MR image and the CT image to be registered; generating a simulated CT image according to the MR image; determining a current displacement field according to the simulated CT image and the CT image to be registered; processing the current displacement field and the previous displacement field based on a diffeomorphic algorithm to generate a superposed displacement field; performing regularization processing on the superposed displacement field to generate a smooth deformation field; generating a registration pending confirmation image according to the smooth deformation field and the simulated CT image; until a preset condition is satisfied, confirming the registration pending confirmation image as the registered image, otherwise returning to the step of determining the current displacement field according to the simulated CT image and the CT image to be registered for iterative processing. The registration method provided by the embodiments of the present invention has a good registration effect on images with large deformations by processing the MR image as a simulated CT image and then performing iterative optimization registration.

Description

Technical Field

[0001] The present invention belongs to the field of computers, and particularly relates to an MR image and CT image registration method, device, computer device, and storage medium. Background Art

[0002] There are various imaging techniques for medical images, such as CT (Computed Tomography) imaging and MR (Magnetic Resonance) imaging. When performing medical image analysis, multiple images of the same patient are often put together for comparative analysis to obtain dynamic information such as the degree of lesion of the patient's lesion. When performing quantitative comparative analysis on different images, image registration is first required.

[0003] In the prior art, the demons algorithm is usually used for registering medical images. The Demons algorithm is a fully automatic non-rigid registration method based on gray information. Specifically, the algorithm uses the gray gradient information of the reference image to determine the movement of each pixel of the floating image. At the same time, Gaussian filtering is used to smooth the obtained offset for regularization, so that the transformation of the local information of the image to be registered is continuous within the global range. However, the demons algorithm has poor effect in processing images with large deformations, and the compression and deformation of some organs or tissues are irreversible, so the effect of the registered image is not ideal.

[0004] It can be seen that the existing medical image registration method still has the technical problem of poor registration effect for images with large deformations and the registered image not being ideal enough. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide an MR image and CT image registration method, aiming to solve the technical problems existing in the existing medical image registration method, namely poor registration effect for images with large deformations and the registered image not being ideal enough.

[0006] The embodiments of the present invention are implemented as follows. An MR image and CT image registration method includes:

[0007] Obtain the MR image and CT image to be registered;

[0008] Generate a simulated CT image according to the MR image to be registered;

[0009] Determine the current displacement field according to the simulated CT image and the CT image to be registered;

[0010] Process the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superimposed displacement field;

[0011] Regularize the superposition state displacement field to generate a smooth deformation field;

[0012] Generate a registration pending confirmation image according to the smooth deformation field and the simulated CT image;

[0013] When it is determined that the preset registration condition is not satisfied, determine the registration pending confirmation image as the simulated CT image, and return to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered;

[0014] When it is determined that the preset registration condition is satisfied, confirm that the registration pending confirmation image is the registered image,

[0015] Wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0016] Another object of the embodiments of the present invention is to provide an MR image and CT image registration device, including:

[0017] An image acquisition unit for acquiring an MR image and a CT image to be registered;

[0018] A simulated CT image generation unit for generating a simulated CT image according to the MR image to be registered;

[0019] A displacement field determination unit for determining a current displacement field according to the simulated CT image and the CT image to be registered;

[0020] A superposition state processing unit for processing the current displacement field and the previous displacement field based on the diffeomorphism algorithm to generate a superposition state displacement field;

[0021] A regularization processing unit for regularizing the superposition state displacement field to generate a smooth deformation field;

[0022] A registration pending confirmation image generation unit for generating a registration pending confirmation image according to the smooth deformation field and the simulated CT image;

[0023] An iterative processing unit for, when it is determined that the preset registration condition is not satisfied, determining the registration pending confirmation image as the simulated CT image, and returning to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered;

[0024] A registered image determination unit for, when it is determined that the preset registration condition is satisfied, confirming that the registration pending confirmation image is the registered image,

[0025] Wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0026] Another object of the embodiments of the present invention is to provide a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method for registering MR images and CT images as described above.

[0027] Another object of the embodiments of the present invention is to provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method for registering MR images and CT images as described above.

[0028] The method for registering MR images and CT images provided by the embodiments of the present invention, after obtaining the MR image and CT image to be registered, generates a simulated CT image by processing the MR image to be registered, then determines the current displacement field according to the simulated CT image and the CT image to be registered, and generates a superposed displacement field by processing the displacement field based on the differential diffeomorphism algorithm, generates a smooth deformation field by processing the superposed displacement field using regularization processing, and then generates a registration pending confirmation image from the smooth deformation field and the simulated CT image, and repeats the above steps multiple times until a preset condition is met, and then uses the current registration pending confirmation image as the registered image. The method for registering MR images and CT images provided by the embodiments of the present invention is based on the differential diffeomorphism algorithm, is applicable to images with large or small deformations, the originally separated anatomical structures still remain separated, and the continuous anatomical structures still remain continuous. The registration result is reversible and smoothly expresses the differences in anatomical structures between images, and the registration effect is better. Description of the Drawings

[0029] Figure 1 is a flowchart of the steps of the method for registering MR images and CT images provided by the embodiments of the present invention;

[0030] Figure 2 is a flowchart of the steps of the method for generating a simulated CT image provided by the embodiments of the present invention;

[0031] Figure 3 is a flowchart of the steps of the method for determining the displacement field provided by the embodiments of the present invention;

[0032] Figure 4 is a flowchart of the steps of processing to generate a superposed displacement field provided by the embodiments of the present invention;

[0033] Figure 5 is a flowchart of the steps of generating a smooth deformation field by regularization processing provided by the embodiments of the present invention;

[0034] Figure 6Schematic diagram of the MR image and CT image registration device provided by the embodiment of the present invention;

[0035] Figure 7 Schematic diagram of the simulated CT image generation unit provided by the embodiment of the present invention;

[0036] Figure 8 Schematic diagram of the displacement field determination unit provided by the embodiment of the present invention;

[0037] Figure 9 Schematic diagram of the superposition state processing unit provided by the embodiment of the present invention;

[0038] Figure 10 Schematic diagram of the regularization processing unit provided by the embodiment of the present invention;

[0039] Figure 11 Comparison data of technical effects of different registration methods provided by the embodiment of the present invention;

[0040] Figure 12(a) is the effect diagram of the registered image of the registration method provided by the embodiment of the present invention;

[0041] Figure 12(b) is the effect diagram of the registered image of the multi-modal registration method based on local phase difference;

[0042] Figure 12(c) is the effect diagram of the registered image of the multi-modal registration method based on mutual information. Detailed implementation manners

[0043] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, the first xx script may be referred to as the second xx script, and similarly, the second xx script may be referred to as the first xx script.

[0045] In the embodiment of the present invention, in order to solve the technical problem that the existing medical registration method has a poor registration effect on images with large deformations, the image to be registered is processed into a simulated CT image, and then based on the diffeomorphic algorithm and regularization processing, the simulated CT image and the CT image to be registered are iteratively registered and optimized multiple times, so as to effectively improve the final registration effect of the image.

[0046] AsFigure 1 As shown, in one embodiment, a method for registering MR images and CT images is proposed, which may specifically include the following steps:

[0047] Step S102, obtain the MR image and CT image to be registered.

[0048] In the embodiments of the present invention, MR images and CT images are two common diagnostic images in medicine, which are generated by MR imaging technology and CT scanning technology respectively.

[0049] Step S104, generate a simulated CT image according to the MR image to be registered.

[0050] In the embodiments of the present invention, generating a simulated CT image according to the MR image to be registered means converting the MR image into a CT image.

[0051] As a feasible embodiment of the present invention, for the specific steps of generating a simulated CT image according to the MR image to be registered, please refer to Figure 2 and its explanation.

[0052] Step S106, determine the current displacement field according to the simulated CT image and the CT image to be registered.

[0053] In the embodiments of the present invention, the current displacement field can be understood as the difference between corresponding points of the images, that is, the phase difference between corresponding points of the simulated CT image and the CT image to be registered.

[0054] As a feasible embodiment of the present invention, for the specific steps of determining the current displacement field according to the simulated CT image and the CT image to be registered, please refer to Figure 3 and its explanation.

[0055] Step S108, process the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superimposed displacement field.

[0056] In the embodiments of the present invention, the diffeomorphic algorithm is used for the exponential mapping of the deformation field, and the current deformation field is superimposed with the previous deformation field, that is, after calculating the current displacement field, the total displacement is increased by updating the field.

[0057] In the embodiments of the present invention, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0058] As a feasible embodiment of the present invention, for the specific steps of processing the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superimposed displacement field, please refer to Figure 4 and its explanation.

[0059] Step S110: Regularize the superposition state displacement field to generate a smooth deformation field.

[0060] In the embodiment of the present invention, by regularizing the superposition state displacement field, a smoother deformation field can be obtained, and at the same time, the influence of image noise on the registration output can be reduced.

[0061] As a feasible embodiment of the present invention, regularization is achieved by applying a low-pass filter to each component of the displacement field. For the specific steps of regularizing the superposition state displacement field to generate a smooth deformation field, please refer to Figure 5 and its explanation.

[0062] Step S112: Generate a registration pending confirmation image according to the smooth deformation field and the simulated CT image.

[0063] In the embodiment of the present invention, the smooth deformation field can be understood as the amount of deformation. After directly superimposing the amount of deformation on the original simulated CT image, the optimized simulated CT image, that is, the registration pending confirmation image, can be obtained.

[0064] In the embodiment of the present invention, determining the displacement field - determining the superposition state displacement field - determining the smooth displacement field can be understood as a complete optimization cycle for the simulated CT image. For the places involving optimization in the following, the explanation here can be referred to.

[0065] Step S114: Determine whether the preset registration condition is satisfied. When it is determined that the preset condition is satisfied, execute step S116; when it is determined that the preset condition is not satisfied, execute step S118.

[0066] In the embodiment of the present invention, the preset registration condition is used to determine whether the condition for optimizing the simulated CT image is reached. The preset condition can be judged from the perspective of the number of optimization times, for example, judging whether the number of optimizations reaches a preset threshold, or it can also be directly judged from the registration pending confirmation image, judging whether the registration effect meets the preset condition.

[0067] Step S116: Determine the registration pending confirmation image as the simulated CT image, and return to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered.

[0068] In the embodiment of the present invention, when the preset registration condition is not satisfied, it means that the simulated CT image still needs to be further optimized.

[0069] In an embodiment of the present invention, determining the image to be confirmed for registration as a simulated CT image can be understood as an iterative process. The optimized simulated CT image is iterated and further optimized. Since the algorithm used in the iteration is convergent, during the continuous superposition based on the diffeomorphic algorithm and under the regularization constraint, the solution will be regular and invertible, and also optimal in terms of local phase difference. At the same time, because the phase is monotonic and smooth, the mismatch of local structures will automatically lead to the update of the non-zero field, which helps to improve the displacement estimation and adapt the structure, that is, the registration effect of the registered image can be continuously improved.

[0070] Step S118, confirm that the image to be confirmed for registration is the registered image.

[0071] In an embodiment of the present invention, when the preset registration condition is met, it means that the current image to be confirmed for registration has achieved the expected registration effect, and the current image to be confirmed for registration can be directly determined as the registered image.

[0072] The method for registering MR images and CT images provided by the embodiments of the present invention includes: after obtaining the MR image and CT image to be registered, generating a simulated CT image by processing the MR image to be registered, then determining the current displacement field according to the simulated CT image and the CT image to be registered, generating a superposed displacement field by processing the displacement field based on the differential registration algorithm, generating a smooth deformation field by processing the superposed displacement field using regularization, and then generating an image to be confirmed for registration from the smooth deformation field and the simulated CT image. Repeat the above steps multiple times until the preset condition is met, and then the current image to be confirmed for registration is the registered image. The method for registering MR images and CT images provided by the embodiments of the present invention is based on the diffeomorphic algorithm and is applicable to images with large or small deformations. The originally separated anatomical structures still remain separated, and the continuously connected anatomical structures still remain continuous. The registration result is reversible and smoothly expresses the differences in anatomical structures between images, and the registration effect is better.

[0073] In one embodiment, as Figure 2 shown, a method for generating a simulated CT image according to the MR image to be registered is provided. The specific steps include:

[0074] Step S202, process the MR image to obtain multiple mdixon sequence images of the MR image.

[0075] In an embodiment of the present invention, the MDixon is a fat suppression technique that uses a spin echo sequence to acquire two echo signals, namely the in-phase and out-of-phase echo signals of water and fat protons, respectively, at different echo times. By adding and subtracting the two different signals, the effect of separating water and fat can be achieved, that is, multiple MDixon sequence images can be obtained. The MDixon sequence images include a fat map, a water map, an in-phase map, and an out-of-phase map.

[0076] Step S204: Determine the feature information of each image point in the MR image according to multiple MDixon sequence images.

[0077] In an embodiment of the present invention, the feature information includes the spatial coordinates of each image point and the MR value in multiple MDixon sequence images.

[0078] In an embodiment of the present invention, the spatial coordinates of each image point in the MR image can be represented by a three-dimensional coordinate. At the same time, the MR value corresponding to the image point in each MDixon sequence image and the spatial coordinates of the image point are combined as the feature information of the image point.

[0079] Step S206: Determine the probability of the classification category corresponding to each image point according to the feature information of each image point and the four-class medical image classification model trained based on the hierarchical support vector machine algorithm.

[0080] In an embodiment of the present invention, the classification categories include fat, soft tissue, air, and bone, that is, after inputting a set of feature information, the probability that the image point is fat, soft tissue, air, or bone can be directly determined according to the feature information.

[0081] As a preferred embodiment of the present invention, since the classification model trained by the support vector machine algorithm is used for binary classification, in order to implement the four-class medical image classification model, we use the hierarchical support vector machine algorithm for training, that is, first train multiple binary medical image classification models, and then construct a decision tree from the multiple binary medical image classification models to generate a four-class medical image classification model.

[0082] Step S208: Determine the CT value corresponding to each image point according to the probability of the classification category corresponding to each image point.

[0083] In an embodiment of the present invention, the CT value of each image point is determined by performing a weighted operation on the standard CT values of each classification category using the probability of the classification category corresponding to each image point as the weight.

[0084] Step S210: Generate a simulated CT image according to the CT value.

[0085] In the embodiments of the present invention, the CT value generally ranges from -1000 Hu to 1000 Hu. Among them, commonly, the CT value of dense bone is 1000 Hu, the CT value of water is 0 Hu, and the CT value of air is -1000 Hu. After knowing the CT value of each image point, a CT image can be simply determined, and this CT image is the simulated CT image.

[0086] A method for generating a simulated CT image provided by an embodiment of the present invention processes the MR image into multiple mdixon sequence images, including a fat map, a water map, a positive phase map, and an opposed phase map, and then determines the feature information of each image point in the original MR image according to each sequence image. The feature information includes the spatial coordinates of each image point and the MR value in multiple mdixon sequence images, and determines the probability of the corresponding classification category of each image point according to the feature information of each image point and a pre-trained four-element medical image classification model generated based on the hierarchical support vector machine algorithm. The classification categories include fat, soft tissue, air, and bone, and determines the corresponding CT value according to the probability of each classification category, so as to directly determine the simulated CT image. The method for generating a simulated CT image provided by an embodiment of the present invention can automatically process the MR image after inputting the MR scan image, directly output the corresponding CT image, effectively improves the processing efficiency, and the four-element medical image classification model adopted in the processing process is pre-trained and generated through a large number of sample data, effectively ensuring the accuracy rate in the processing process and improving the effect of the generated CT image.

[0087] In one embodiment, as Figure 3 shown, a method for determining a displacement field is provided, and the specific steps include:

[0088] Step S302, determining an orthogonal filter according to a polarity separable function.

[0089] In the embodiments of the present invention, it is necessary to determine an orthogonal filter through a polarity separable function in the direction η ∈ R 3 .

[0090] As a feasible embodiment of the present invention, the expression of the polarity separable function is as follows:

[0091]

[0092] where ω ∈ R 3 is a frequency vector. When λ > 0, χ + (λ) = 1, otherwise χ + (λ) = 0, is a unit vector supporting ω, and R is a radial basis function centered on ρ > 0, defined as:

[0093]

[0094] Step S304: Determine the local phases and the local phase difference in a specific frequency and a specific direction in the simulated CT image and the CT image to be registered according to the orthogonal filter.

[0095] In the embodiment of the present invention, the local phase difference (Δφ k ) is calculated as follows:

[0096]

[0097] where q f (x; k) is a filtering operation, which is defined as q f (x; k) = (f * h k )(x), w is the simulated CT image, f is the CT image to be registered, h k (x) is a filter centered at the origin in the spatial domain, and * is a common convolution operation.

[0098] Step S306: Determine the displacement field at different scales according to the weighted least squares method and the local phase difference.

[0099] In the embodiment of the present invention, the displacement field Θ D (f, w) is calculated as follows:

[0100]

[0101] where c k reflects the reliability of each pixel estimation, c k (x) = A f (x; k) A m (x; k), and the global certainty mapping relationship related to the estimation quality of Θ D is:

[0102]

[0103] A method for determining a displacement field provided by an embodiment of the present invention, after determining an orthogonal filter according to a polarity separable function, determines the phase difference between a simulated CT image and a CT image to be registered, and finally determines the displacement field at different scales based on the weighted least squares method, realizing a rough-to-fine displacement estimation, that is, determining the displacement field on different scale bands of the simulated CT image and the CT image to be registered.

[0104] In one embodiment, as Figure 4 shown, a method for processing and generating a superposition state displacement field based on a diffeomorphic algorithm is provided, and the specific steps include:

[0105] Step S402: Determine the weights of the current displacement field and the previous displacement field based on the certainty of the current displacement field and the superposition certainty mapping.

[0106] In an embodiment of the present invention, the weight of the previous displacement field D a is set to 1, and the weight of the current displacement field D u is where c u is the certainty of the current displacement field D u , calculated according to Θ c , and c a is the superposition certainty mapping, updated according to its own certainty.

[0107] Step S404: Based on the diffeomorphism algorithm, perform the exponential mapping of the deformation field and determine the superposition state displacement field according to the weights.

[0108] In an embodiment of the present invention, by introducing the diffeomorphism flow exp(), the exponential mapping of the deformation field is performed based on the diffeomorphism algorithm. The idea is to adapt the Lie group using an optimization method, thereby restricting the possible solutions to diffeomorphic transformations.

[0109] In an embodiment of the present invention, the superposition state displacement field Φ D (D a ,D u ,c a ,c u ) is calculated as follows:

[0110]

[0111] Where, in an embodiment of the present invention, since the superposition step involves attenuation regarding certainty, the original use of a small number of recursive combinations of the field D u itself to estimate the exponential mapping exp(D u ) is changed to (since in any vector field D, exp(0D u ) = Id), when c a >> c u , the superposition gradually disappears)

[0112] The embodiment of the present invention provides a method for processing and generating a superposition state displacement field based on the diffeomorphism algorithm. The diffeomorphism algorithm is used to perform the exponential mapping of the deformation field and add the current deformation field and the previous deformation field to determine the superposition state displacement field, and the processing formula is specifically disclosed. Together with the following regularization processing, the optimization of the deformation field is restricted, which helps to improve displacement estimation and adapt the structure.

[0113] In one embodiment, as Figure 5As shown, a method for generating a smooth deformation field through regularization processing is provided, and the specific steps include:

[0114] Step S502: Determine a positive function and a filter according to the displacement field selection area.

[0115] In an embodiment of the present invention, the filter is usually a Gaussian kernel with a variance σ Ψ >0.

[0116] Step S504: Determine a normalized convolution kernel according to the positive function and the filter, and perform convolution processing on the displacement field.

[0117] In an embodiment of the present invention, the expression of the normalized convolution kernel s is as follows:

[0118]

[0119] where h is the determined positive function and g is the determined filter.

[0120] In an embodiment of the present invention, after determining the normalized convolution kernel, it is necessary to further perform normalization processing on all vectors in the displacement field.

[0121] Step S506: Determine a smooth deformation field according to the displacement field after convolution processing and the filter.

[0122] In an embodiment of the present invention, in addition to determining the smooth deformation field Ψ D through regularization processing, considering the deterministic mapping mentioned above to ensure high position certainty and maintain the correspondence between the displacement vector and its corresponding determined position, the deterministic mapping is regularized in the same way as the displacement field mapping, that is, it is also necessary to perform regularization processing on the superimposed deterministic mapping c a to determine Ψ c , and the Ψ D , Ψ c has the following calculation formula:

[0123] Ψ D (D a ,c a ) = D a *c a g, Ψ c (c a ,c a ) = c a *c a g

[0124] An embodiment of the present invention provides a method for generating a smooth deformation field through regularization processing, including determining a positive function and a filter. After determining the normalized convolution kernel, convolution processing is performed on each vector of the displacement field. Finally, to ensure a high degree of certainty of the position, the correspondence between the displacement vector and its corresponding determined position is maintained, and at the same time, regularization processing is performed on the displacement field and the superimposed deterministic mapping, which, together with the aforementioned diffeomorphic algorithm, restricts the optimization of the deformation field, helps to improve displacement estimation, and adapts the structure.

[0125] As Figure 6 shown, in one embodiment, an MR image and CT image registration device is provided. The MR image and CT image registration device may specifically include an image acquisition unit 610, a simulated CT image generation unit 620, a displacement field determination unit 630, a superposition state processing unit 640, a regularization processing unit 650, a registration pending confirmation image generation unit 660, an iterative processing unit 670, and a registered image determination unit 680.

[0126] The image acquisition unit 610 is configured to acquire the MR image and CT image to be registered.

[0127] In an embodiment of the present invention, the MR image and CT image are two common diagnostic images in medicine, which are generated by MR imaging technology and CT scanning technology respectively.

[0128] The simulated CT image generation unit 620 is configured to generate a simulated CT image according to the MR image to be registered.

[0129] In an embodiment of the present invention, generating a simulated CT image according to the MR image to be registered means converting the MR image into a CT image.

[0130] As a feasible embodiment of the present invention, for the specific structural schematic diagram of the simulated CT image generation unit, please refer to Figure 7 and its explanatory notes.

[0131] The displacement field determination unit 630 is configured to determine the current displacement field according to the simulated CT image and the CT image to be registered.

[0132] In an embodiment of the present invention, the current displacement field can be understood as the difference between corresponding points of the images, that is, the phase difference between corresponding points of the simulated CT image and the CT image to be registered.

[0133] As a feasible embodiment of the present invention, for the specific structural schematic diagram of the displacement field determination unit, please refer to Figure 8 and its explanatory notes.

[0134] The superposition state processing unit 640 is configured to process the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superposition state displacement field.

[0135] In an embodiment of the present invention, the exponential mapping of the deformation field is performed by using the diffeomorphic algorithm, and the current deformation field is superimposed on the previous deformation field, that is, after calculating the current displacement field, the total displacement is increased by updating the field.

[0136] In an embodiment of the present invention, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0137] As a feasible embodiment of the present invention, for the specific structural schematic diagram of the superposition state processing unit, please refer to Figure 9 and its explanatory notes.

[0138] The regularization processing unit 650 is configured to perform regularization processing on the superposition state displacement field to generate a smooth deformation field.

[0139] In an embodiment of the present invention, by performing regularization processing on the superposition state displacement field, a smoother deformation field can be obtained, and at the same time, the influence of image noise on the registration output can be reduced.

[0140] As a feasible embodiment of the present invention, the regularization processing is realized by applying a low-pass filter to each component of the displacement field. For the specific structural schematic diagram of the regularization processing unit, please refer to Figure 10 and its explanatory notes.

[0141] The image to be registered generation unit 660 is configured to generate an image to be registered according to the smooth deformation field and the simulated CT image.

[0142] In an embodiment of the present invention, the smooth deformation field can be understood as the amount of deformation. After directly superimposing the amount of deformation on the original simulated CT image, an optimized simulated CT image, that is, an image to be registered, can be obtained.

[0143] In an embodiment of the present invention, determining the displacement field - determining the superposition state displacement field - determining the smooth displacement field can be understood as a complete optimization cycle for the simulated CT image.

[0144] The iterative processing unit 670 is configured to determine the image to be registered as the simulated CT image when it is determined that the preset registration condition is not satisfied.

[0145] In an embodiment of the present invention, the preset registration condition is used to determine whether the condition for optimizing the simulated CT image is met. The preset condition can be judged from the perspective of the number of optimization times. For example, it is judged whether the number of optimizations reaches a preset threshold, or it can also be directly judged from the image to be confirmed for registration, that is, whether the registration effect meets the preset condition.

[0146] In an embodiment of the present invention, when the preset registration condition is not met, it means that the simulated CT image still needs to be further optimized.

[0147] In an embodiment of the present invention, determining the image to be confirmed for registration as the simulated CT image can be understood as an iterative process. The optimized simulated CT image is iterated and further optimized. Since the algorithm used in the iteration is convergent, in the process of continuously superimposing based on the diffeomorphic algorithm and under the limitation of regularization, the solution will be regular and reversible, and is also optimal in terms of local phase difference. At the same time, because the phase is monotonic and smooth, the mismatch of local structures will automatically lead to the update of the non-zero field, which helps to improve the displacement estimation and adapt the structure, that is, the registration effect of the registered image can be continuously improved.

[0148] The registration image determination unit 680 is configured to confirm the image to be confirmed for registration as the registered image when it is judged that the preset registration condition is met.

[0149] In an embodiment of the present invention, when the preset registration condition is met, it means that the current image to be confirmed for registration has reached the expected registration effect, and the current image to be confirmed for registration can be directly determined as the registered image.

[0150] An MR image and CT image registration device provided by an embodiment of the present invention, after obtaining the MR image and CT image to be registered, generates a simulated CT image by processing the MR image to be registered, then determines the current displacement field according to the simulated CT image and the CT image to be registered, generates a superimposed displacement field by processing the displacement field based on the differential registration algorithm, generates a smooth deformation field by processing the superimposed displacement field using regularization processing, and then generates an image to be confirmed for registration from the smooth deformation field and the simulated CT image. The above steps are repeated multiple times until the preset condition is met, and the current image to be confirmed for registration is used as the registered image. The MR image and CT image registration method provided by the embodiment of the present invention is based on the diffeomorphic algorithm and is applicable to images with large or small deformations. The originally separated anatomical structures still remain separated, and the continuously connected anatomical structures still remain continuous. The registration result is reversible and smoothly expresses the differences in anatomical structures between images, and the registration effect is better.

[0151] Such as Figure 7As shown in the figure, in one embodiment, the simulated CT image generation unit 620 specifically includes: an MR image processing module 701, a feature information determination module 702, a classification module 703, a CT value determination module 704, and a simulated CT image generation module 705.

[0152] The MR image processing module 701 is configured to process the MR image to obtain a plurality of mdixon sequence images of the MR image.

[0153] In an embodiment of the present invention, the mdixon is a fat suppression technique that uses a spin echo sequence to separately collect two echo signals, namely the in-phase and out-of-phase signals of water and fat protons, at different echo times. By adding and subtracting the two different signals, the effect of separating water and fat can be achieved, that is, a plurality of mdixon sequence images can be obtained. The mdixon sequence images include a fat map, a water map, an in-phase map, and an out-of-phase map.

[0154] The feature information determination module 702 is configured to determine the feature information of each image point in the MR image according to a plurality of mdixon sequence images.

[0155] In an embodiment of the present invention, the feature information includes the spatial coordinates of each image point and the MR value in a plurality of mdixon sequence images.

[0156] In an embodiment of the present invention, the spatial coordinates of each image point in the MR image can be represented by a three-dimensional coordinate. At the same time, the MR value corresponding to the image point in each mdixon sequence image is combined with the spatial coordinates of the image point as the feature information of the image point.

[0157] The classification module 703 is configured to determine the probability of the classification category corresponding to each image point according to the feature information of each image point and a four-class medical image classification model trained based on the hierarchical support vector machine algorithm.

[0158] In an embodiment of the present invention, the classification categories include fat, soft tissue, air, and bone, that is, after inputting a set of feature information, the probability that the image point is fat, soft tissue, air, or bone can be directly determined according to the feature information.

[0159] As a preferred embodiment of the present invention, since the classification model trained by the support vector machine algorithm is used for binary classification, in order to implement a four-class medical image classification model, we use the hierarchical support vector machine algorithm for training, that is, first train and generate a plurality of binary medical image classification models, and then construct a decision tree from the plurality of binary medical image classification models to generate a four-class medical image classification model.

[0160] The CT value determination module 704 is configured to determine the CT value corresponding to each image point according to the probability of the classification category corresponding to each image point.

[0161] In an embodiment of the present invention, the standard CT values of each classification category are weighted according to the probability of the classification category corresponding to each image point as a weight, so as to determine the CT value of each image point.

[0162] The simulated CT image generation module 705 is configured to generate a simulated CT image according to the CT value.

[0163] In an embodiment of the present invention, the CT value generally ranges from -1000 Hu to 1000 Hu. Among them, commonly, the CT value of dense bone is 1000 Hu, the CT value of water is 0 Hu, and the CT value of air is -1000 Hu. After knowing the CT value of each image point, a CT image can be simply determined, and this CT image is the simulated CT image.

[0164] An embodiment of the present invention provides a specific structural schematic diagram of a simulated CT image generation unit. By processing the MR image into multiple mdixon sequence images, including a fat map, a water map, a positive phase map, and an opposed phase map, and then determining the feature information of each image point in the original MR image according to each sequence image, the feature information includes the spatial coordinates of each image point and the MR value in multiple mdixon sequence images, and determining the probability of the classification category corresponding to each image point according to the feature information of each image point and a pre-trained four-element medical image classification model generated based on the hierarchical support vector machine algorithm, the classification categories include fat, soft tissue, air, and bone, and determining the corresponding CT value according to the probability of each classification category, so as to directly determine the simulated CT image. The simulated CT image generation method provided by the embodiment of the present invention can automatically process the MR image after inputting the MR scan image, directly output the corresponding CT image, effectively improve the processing efficiency, and the four-element medical image classification model adopted in the processing process is pre-trained through a large number of sample data, effectively ensuring the accuracy rate in the processing process and improving the effect of the generated CT image.

[0165] As Figure 8 shown, in one embodiment, the displacement field determination unit 630 specifically includes: an orthogonal filter determination module 801, a local phase difference determination module 802, and a displacement field determination module 803.

[0166] The orthogonal filter determination module 801 is configured to determine an orthogonal filter according to a polarity separable function.

[0167] In an embodiment of the present invention, it is necessary to determine an orthogonal filter through a polarity separable function in the direction η∈R 3 on.

[0168] As a feasible embodiment of the present invention, the expression of the polarity separable function is as follows:

[0169]

[0170] where ω ∈ R 3 is the frequency vector. In the case of λ > 0, χ + (λ) = 1, otherwise χ + (λ) = 0, is the unit vector supporting ω, and R is a radial basis function centered at ρ > 0, defined as:

[0171]

[0172] The local phase difference determination module 802 is configured to determine the local phase and the local phase difference at a specific frequency and a specific direction in the simulated CT image and the CT image to be registered according to the orthogonal filter.

[0173] In the embodiment of the present invention, the calculation formula of the local phase difference (Δφ k ) is as follows:

[0174]

[0175] where q f (x; k) is a filtering operation, which is defined as q f (x; k) = (f * h k )(x), w is the simulated CT image, f is the CT image to be registered, h k (x) is a filter centered at the origin in the spatial domain, and * is a common convolution operation.

[0176] The displacement field determination module 803 is configured to determine the displacement field at different scales according to the weighted least squares method and the local phase difference.

[0177] In the embodiment of the present invention, the calculation formula of the displacement field Θ D (f, w) is as follows:

[0178]

[0179] where c k reflects the reliability of each pixel estimation, c k (x) = A f (x; k)A m (x; k), and the global deterministic mapping relationship related to the estimation quality of Θ D is:

[0180]

[0181] An embodiment of the present invention provides a specific structural schematic diagram of a displacement field determination unit. After determining the orthogonal filter according to the polarity separable function, the phase difference between the simulated CT image and the CT image to be registered is determined, and finally the displacement field at different scales is determined based on the weighted least squares method, realizing rough-to-fine displacement estimation, that is, determining the displacement field on different scale bands of the simulated CT image and the CT image to be registered.

[0182] As Figure 9 shown, in one embodiment, the superposition state processing unit 640 specifically includes: a weight determination module 901 and a superposition state displacement field determination module 902.

[0183] The weight determination module 901 is used to determine the weights of the current displacement field and the previous displacement field according to the certainty of the current displacement field and the superposition certainty mapping.

[0184] In the embodiment of the present invention, the weight of the previous displacement field D a is set to 1, and the weight of the current displacement field D u is where c u is the certainty of the current displacement field D u , calculated according to Θ c , and c a is the superposition certainty mapping, which is updated according to its own certainty.

[0185] The superposition state displacement field determination module 902 is used to perform exponential mapping of the deformation field and determine the superposition state displacement field based on the weight by means of the diffeomorphism algorithm.

[0186] In the embodiment of the present invention, by introducing the diffeomorphism flow exp(), exponential mapping of the deformation field is performed based on the diffeomorphism algorithm. The idea is to adapt the Lie group using an optimization method, thereby restricting the possible solutions to differential transformations.

[0187] In the embodiment of the present invention, the formula for the superposition state displacement field Φ D (D a , D u , c a , c u ) is as follows:

[0188]

[0189] Among them, in the embodiment of the present invention, since the superposition step involves attenuation regarding certainty, the original use of a small amount of recursive combination of the field D u itself to estimate the exponential mapping exp(D u ) is changed to (Since in any vector field D, exp(0D u ) = Id), when c a >> c u , the superposition gradually disappears)

[0190] An embodiment of the present invention provides a specific structural schematic diagram of a superposition state processing unit, which uses a diffeomorphic algorithm to perform exponential mapping of a deformation field and adds the current deformation field to the previous deformation field to determine a superposition state displacement field, and specifically discloses a processing formula. Together with the following regularization processing unit, it restricts the optimization of the deformation field, helps to improve displacement estimation, and makes the structure adaptable.

[0191] As Figure 10 shown, in one embodiment, the regularization processing unit 650 specifically includes: a positive function and filter determination module 1001, a convolution processing module 1002, and a smooth deformation field determination module 1003.

[0192] The positive function and filter determination module 1001 is used to determine a positive function and a filter according to the displacement field selection area.

[0193] In an embodiment of the present invention, the filter is usually a Gaussian kernel with a variance σ Ψ > 0.

[0194] The convolution processing module 1002 is used to determine a normalized convolution kernel according to the positive function and the filter and perform convolution processing on the displacement field.

[0195] In an embodiment of the present invention, the expression of the normalized convolution kernel s is as follows:

[0196]

[0197] where h is the determined positive function and g is the determined filter.

[0198] In an embodiment of the present invention, after determining the normalized convolution kernel, it is necessary to further normalize all vectors in the displacement field.

[0199] The smooth deformation field determination module 1003 is used to determine a smooth deformation field according to the displacement field after convolution processing and the filter.

[0200] In an embodiment of the present invention, in addition to determining the smooth deformation field Ψ D through regularization processing, considering the deterministic mapping mentioned above at the same time to ensure high certainty of the position and maintain the correspondence between the displacement vector and its corresponding determined position, the deterministic mapping is regularized in the same way as the displacement field mapping, that is, it is also necessary to perform regularization processing on the superposition deterministic mapping c a to determine Ψ c , the ΨD , Ψ c The calculation formula of

[0201] Ψ D (D a , c a ) = D a * c a g, Ψ c (c a , c a ) = c a * c a g

[0202] An embodiment of the present invention provides a specific structural schematic diagram of a regularization processing unit, including determining a positive function and a filter. After determining the normalized convolution kernel, convolution processing is performed on each vector of the displacement field. Finally, to ensure a high degree of certainty of the position, the correspondence between the displacement vector and its corresponding determined position is maintained, and at the same time, regularization processing is performed on the displacement field and the superimposed deterministic mapping, which, together with the aforementioned superposition state processing unit, restricts the optimization of the deformation field, helps to improve displacement estimation, and adapts the structure.

[0203] In one embodiment, a computer device is proposed. The computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0204] Obtain the MR image and CT image to be registered;

[0205] Generate a simulated CT image according to the MR image to be registered;

[0206] Determine the current displacement field according to the simulated CT image and the CT image to be registered;

[0207] Process the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superposition state displacement field;

[0208] Perform regularization processing on the superposition state displacement field to generate a smooth deformation field;

[0209] Generate a registration pending confirmation image according to the smooth deformation field and the simulated CT image;

[0210] When it is determined that the preset registration condition is not satisfied, determine the registration pending confirmation image as the simulated CT image, and return to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered;

[0211] When it is determined that the preset registration condition is satisfied, confirm the registration pending confirmation image as the registered image,

[0212] Wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0213] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor is caused to perform the following steps:

[0214] Obtain an MR image and a CT image to be registered;

[0215] Generate a simulated CT image according to the MR image to be registered;

[0216] Determine the current displacement field according to the simulated CT image and the CT image to be registered;

[0217] Process the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superposed displacement field;

[0218] Perform regularization processing on the superposed displacement field to generate a smooth deformation field;

[0219] Generate an image to be confirmed for registration according to the smooth deformation field and the simulated CT image;

[0220] When it is determined that the preset registration condition is not satisfied, determine the image to be confirmed for registration as the simulated CT image, and return to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered;

[0221] When it is determined that the preset registration condition is satisfied, confirm the image to be confirmed for registration as the registered image,

[0222] Wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0.

[0223] In order to intuitively characterize the technical performance of the above MR image and CT image registration method, according to the above disclosed method, an embodiment is completed for the volunteer data provided by Case Western Reserve University in the United States. This data includes a total of 7 sets of lower abdominal MR image and CT image data of patients. In addition, the technical solution disclosed in the present invention is compared with the existing multi-modal registration based on local phase difference and the multi-modal registration based on mutual information from three performance indicators: mutual information, cross-correlation, and sum of squared pixel differences. For specific experimental data, please refer to the following Figure 11Please refer to FIGS. 12(a), 12(b), and 12(c) below for the registration effect diagrams of the three registration methods. Among them, FIG. 12(a) is the effect diagram of the registered image using the registration method provided in the embodiment of the present invention, FIG. 12(b) is the effect diagram of the registered image of the multi-modal registration method based on local phase difference, and FIG. 12(c) is the effect diagram of the registered image of the multi-modal registration method based on mutual information.

[0224] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0225] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0226] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0227] The above-described embodiments merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention shall be subject to the appended claims.

[0228] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An MR image and CT image registration method, characterized in that, Comprising: Obtaining an MR image and a CT image to be registered; Generating a simulated CT image according to the MR image to be registered; Determining a displacement field according to the phase difference between the simulated CT image and the CT image to be registered; Processing the current displacement field and the previous displacement field based on a diffeomorphic algorithm to generate a superposed displacement field; Regularizing the superposed displacement field to generate a smooth deformation field; Generating a registration pending confirmation image according to the smooth deformation field and the simulated CT image; When it is determined that the preset registration condition is not satisfied, determining the registration pending confirmation image as the simulated CT image, and returning to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered; When it is determined that the preset registration condition is satisfied, confirming the registration pending confirmation image as the registered image, wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0; The step of processing the current displacement field and the previous displacement field based on a diffeomorphic algorithm to generate a superposed displacement field includes: Determining the weights of the current displacement field and the previous displacement field according to the certainty of the current displacement field and the superposed certainty mapping; Based on the diffeomorphic algorithm, performing exponential mapping of the deformation field and determining weights to generate a superposed displacement field.

2. The MR image and CT image registration method according to claim 1, characterized in that The step of generating a simulated CT image according to the MR image to be registered includes: Processing the MR image to obtain a plurality of mdixon sequence images of the MR image, the plurality of mdixon sequence images including a fat map, a water map, a positive phase map, and an opposed phase map; Determining the feature information of each image point in the MR image according to the plurality of mdixon sequence images, the feature information including the spatial coordinates of each image point and the MR value in the plurality of mdixon sequence images; Determining the probability of the classification category corresponding to each image point according to the feature information of each image point and a four - element medical image classification model trained based on a hierarchical support vector machine algorithm, the classification categories including fat, soft tissue, air, and bone; Determining the CT value corresponding to each image point according to the probability of the classification category corresponding to each image point; Generating a simulated CT image according to the CT value.

3. The MR image and CT image registration method according to claim 1, characterized in that The step of determining a displacement field according to the phase difference between the simulated CT image and the CT image to be registered includes: Determining an orthogonal filter according to a polarity - separable function; Determining the local phase and the local phase difference of the simulated CT image and the CT image to be registered at a specific frequency and a specific direction according to the orthogonal filter; Determining the displacement field at different scales according to the weighted least - squares method and the local phase difference.

4. The MR image and CT image registration method according to claim 1, wherein The step of regularizing the superposed displacement field to generate a smooth deformation field specifically includes: Determining a positive function and a filter according to the displacement field selection area; Determining a normalized convolution kernel according to the positive function and the filter and performing convolution processing on the displacement field; Determining a smooth deformation field according to the displacement field after convolution processing and the filter.

5. An MR image and CT image registration device, characterized in that, Comprising: An image acquisition unit for acquiring an MR image and a CT image to be registered; A simulated CT image generation unit for generating a simulated CT image according to the MR image to be registered; A displacement field determination unit for determining a displacement field according to the phase difference between the simulated CT image and the CT image to be registered; A superposition state processing unit for processing the current displacement field and the previous displacement field based on the diffeomorphic algorithm to generate a superposition state displacement field; A regularization processing unit for regularizing the superposition state displacement field to generate a smooth deformation field; A registration pending image generation unit for generating a registration pending image according to the smooth deformation field and the simulated CT image; An iterative processing unit for, when it is determined that the preset registration condition is not satisfied, determining the registration pending image as the simulated CT image and returning to the step of determining the displacement field according to the phase difference between the simulated CT image and the CT image to be registered; A registered image determination unit for, when it is determined that the preset registration condition is satisfied, confirming the registration pending image as the registered image; wherein, when the current displacement field is determined for the first time according to the simulated CT image and the CT image to be registered, the previous displacement field is 0; The superposition state processing unit is specifically configured to determine the weights of the current displacement field and the previous displacement field according to the certainty of the current displacement field and the superposition certainty mapping, and determine the superposition state displacement field based on the exponential mapping of the deformation field and the weights of the diffeomorphic algorithm.

6. The MR image and CT image registration device according to claim 5, characterized in that, The simulated CT image generation unit includes: An MR image processing module for processing the MR image to obtain a plurality of mdixon sequence images of the MR image, the plurality of mdixon sequence images including a fat map, a water map, a positive phase map, and an opposed phase map; A feature information determination module for determining the feature information of each image point in the MR image according to the plurality of mdixon sequence images, the feature information including the spatial coordinates of each image point and the MR value in the plurality of mdixon sequence images; A classification module for determining the probability of the classification category corresponding to each image point according to the feature information of each image point and a four - element medical image classification model trained based on the hierarchical support vector machine algorithm, the classification categories including fat, soft tissue, air, and bone; A CT value determination module for determining the CT value corresponding to each image point according to the probability of the classification category corresponding to each image point; and A simulated CT image generation module for generating a simulated CT image according to the CT value.

7. The MR image and CT image registration device according to claim 5, characterized in that, The displacement field determination unit includes: An orthogonal filter determination module for determining an orthogonal filter according to a polarity - separable function; A local phase difference determination module for determining the local phase and the local phase difference in a specific frequency and a specific direction in the simulated CT image and the CT image to be registered according to the orthogonal filter; A displacement field determination module for determining the displacement field at different scales according to the weighted least - squares method and the local phase difference.

8. A computer device, characterized in that, It includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the processor is caused to execute the steps of the method for registering the MR image and the CT image according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the processor is caused to execute the steps of the method for registering the MR image and the CT image according to any one of claims 1 to 4.

Images