Image registration method and device

Determine rotation parameters and template matching through neural networks to determine translation parameters, combined with binarization processing, the problem of large time consumption of 2D/3D image registration in the prior art is solved, and fast and effective image registration is achieved, suitable for images Guide real-time needs in treatment.

CN112308764BActive Publication Date: 2024-11-26HANGZHOU SANTAN MEDICAL TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011084835.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-12
Publication Date
2024-11-26
Estimated Expiration
2040-10-12

AI Technical Summary

Technical Problem

The existing 2D/3D image registration algorithms consume a lot of time in image guidance treatment and are difficult to meet the real-time requirements, especially due to the high computational complexity caused by frequent use of DRR algorithms to generate DRR images to be matched.

Method used

By inputting the two-dimensional medical images of the target object into the pre-trained neural network, the rotation parameters of its projection direction are determined, and the three-dimensional medical images are projected based on these parameters to generate a two-dimensional digital reconstruction image with the largest field of view, combined with binarization Process and template matching, determine rotation and translation parameters, and realize image registration.

Benefits of technology

It reduces the computational complexity, improves the solution speed, and realizes the rapid completion of image registration, which is suitable for real-time requirements in image-guided treatment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112308764B_ABST
    Figure CN112308764B_ABST
Patent Text Reader

Abstract

The present invention discloses an image registration method and device. The method comprises: inputting a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing the projection direction of the two-dimensional medical image; projecting the three-dimensional medical image of the target object based on the rotation parameter to obtain a two-dimensional digital reconstructed image of the maximum field of view; respectively determining the projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image; determining the translation parameter of the three-dimensional medical image relative to the two-dimensional medical image according to the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, wherein the translation parameter and the rotation parameter characterize the registration information of the three-dimensional medical image to the two-dimensional medical image. The computational complexity is reduced and the solution speed is accelerated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to an image registration method and device. Background Art

[0002] During image guided therapy (IGT), such as spinal surgery, guidance tracking during surgery is achieved by registering the patient's 2D (two-dimensional) X-ray image with the patient's 3D (three-dimensional) medical image.

[0003] At present, 2D / 3D registration algorithms are generally based on search strategies. By changing the image registration parameters of the DRR (Digitally Reconstructed Radiograph) algorithm, DRR images are obtained from 3D medical images, and a similarity metric (such as mutual information, etc.) between the DRR image and the X-ray film is established. The image registration parameters are adjusted according to the similarity metric until a set of optimal parameters is obtained as the final registration result. This method requires frequent use of the DRR algorithm to generate the DRR images to be matched, which consumes a lot of time and is difficult to meet real-time requirements. Summary of the invention

[0004] In view of this, the present invention provides an image registration method and device.

[0005] Specifically, the present invention is achieved through the following technical solutions:

[0006] In a first aspect, an image registration method is provided, comprising:

[0007] Inputting a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing a projection direction of the two-dimensional medical image;

[0008] Performing projection processing on the three-dimensional medical image of the target object based on the rotation parameters to obtain a two-dimensional digitally reconstructed image with a maximum field of view;

[0009] respectively determining a projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image;

[0010] The translation parameters of the three-dimensional medical image relative to the two-dimensional medical image are determined according to the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and the translation parameters and the rotation parameters represent the registration information of the three-dimensional medical image to the two-dimensional medical image.

[0011] Optionally, determining a projection area of ​​the target object in the two-dimensional digital reconstructed image includes:

[0012] Performing a fixed threshold binarization process on the two-dimensional digital reconstructed image to obtain a first binarized image;

[0013] Performing adaptive threshold binarization processing on the two-dimensional digital reconstructed image to obtain a second binarized image;

[0014] The projection area is determined according to the first binarized image and the second binarized image.

[0015] Optionally, determining a projection area of ​​the target object in the two-dimensional medical image includes:

[0016] Performing a fixed threshold binarization process on the two-dimensional medical image to obtain a third binarized image;

[0017] Performing adaptive threshold binarization processing on the two-dimensional medical image to obtain a fourth binarized image;

[0018] The projection area is determined according to the third binarized image and the fourth binarized image.

[0019] Optionally, before binarization processing is performed on the two-dimensional digital reconstructed image or the two-dimensional medical image, the method further includes:

[0020] Performing histogram equalization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image;

[0021] Gaussian filtering is performed on the two-dimensional digital reconstructed image or the two-dimensional medical image that has undergone the histogram equalization processing, so as to perform binarization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image that has undergone the Gaussian filtering processing.

[0022] Optionally, determining a translation parameter of the three-dimensional medical image relative to the two-dimensional medical image includes:

[0023] Performing template matching on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image;

[0024] The translation parameter is determined according to the template matching result.

[0025] Optionally, the two-dimensional digitally reconstructed image and the two-dimensional medical image have the same pitch-to-height ratio.

[0026] In a second aspect, an image registration device is provided, comprising:

[0027] An input module, configured to input a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing a projection direction of the two-dimensional medical image;

[0028] A projection module, used for projecting the three-dimensional medical image of the target object based on the rotation parameter to obtain a two-dimensional digitally reconstructed image with a maximum field of view;

[0029] A determination module, used to determine the projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image respectively;

[0030] The determination module is also used to determine the translation parameters of the three-dimensional medical image relative to the two-dimensional medical image based on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and the translation parameters and the rotation parameters represent the registration information of the three-dimensional medical image to the two-dimensional medical image.

[0031] Optionally, the determining module includes:

[0032] A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional digital reconstructed image to obtain a first binarized image;

[0033] A second binarization unit is used to perform a binarization process of the two-dimensional digital reconstructed image with an adaptive threshold to obtain a second binarized image;

[0034] A determination unit is used to determine a projection area in the two-dimensional digital reconstructed image according to the first binarized image and the second binarized image.

[0035] Optionally, the determining module is used to:

[0036] A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional medical image to obtain a third binarized image;

[0037] A second binarization unit is used to perform an adaptive threshold binarization process on the two-dimensional medical image to obtain a fourth binarized image;

[0038] A determining unit is used to determine a projection area in the two-dimensional medical image according to the third binarized image and the fourth binarized image.

[0039] Optionally, it also includes:

[0040] An equalization processing module, used for performing histogram equalization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image;

[0041] The filtering module is used to perform Gaussian filtering on the two-dimensional digital reconstructed image or the two-dimensional medical image after the histogram equalization processing, so as to perform binarization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image after the Gaussian filtering processing.

[0042] Optionally, the determining module includes:

[0043] A matching unit is used to perform template matching on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and determine the translation parameter according to the template matching result.

[0044] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned image registration methods when executing the computer program.

[0045] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned image registration methods are implemented.

[0046] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0047] During the image registration process performed by the embodiment of the present invention, the parameters are decoupled into two processes of solving rotation parameters and translation parameters. A single DRR projection is used to complete the coarse registration. Compared with the exhaustive search and joint parameter solution methods of the related art, the computational complexity is reduced and the solution speed is accelerated.

[0048] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0050] Figure 1 is a flow chart of an image registration method shown in an exemplary embodiment of the present invention;

[0051] Figure 2 is a flow chart showing a method of determining a projection area of ​​a target object in a two-dimensional digitally reconstructed image according to an exemplary embodiment of the present invention;

[0052] Figure 3 is a flow chart showing a method of determining a projection area of ​​a target object in a two-dimensional medical image according to an exemplary embodiment of the present invention;

[0053] Figure 4 is a module schematic diagram of an image registration device shown in an exemplary embodiment of the present invention;

[0054] Figure 5 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0056] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0057] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0058] 2D / 3D medical image registration is to place an image in a fixed spatial coordinate system, for example, place a 2D medical image in an XY coordinate system, and perform a series of spatial transformations on the 3D medical image to align the points on the 3D medical image, or at least the points in the region of interest, with the 2D medical image. The spatial transformation relationship of transforming the 3D medical image from the X'Y'Z' coordinate system to the XY coordinate system can be represented by three translation parameters (Δx, Δy, Δz) and three rotation parameters (Δθ x , Δθ y , Δθ z )definition.

[0059] Generally speaking, the image registration process is divided into two stages. The first stage is the coarse registration stage, which is to roughly align the parts and determine the registration information (translation parameters and rotation parameters) of the 3D medical image to the 2D medical image. The second stage is the fine registration stage, which optimizes the initial registration parameters to achieve the best registration effect. The effect of the fine registration process depends to a large extent on the accuracy of the coarse registration. If the coarse registration fails to provide a good estimate of the initial registration parameters, the fine registration will fall into a local optimal solution and fail to generate the expected results, resulting in registration failure.

[0060] Figure 1 is a flowchart of an image registration method shown in an exemplary embodiment of the present invention, which is applicable to the coarse registration stage, see Figure 1 , the method may include the following steps:

[0061] Step 101: Input a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter representing a projection direction of the two-dimensional medical image.

[0062] The target object is the diseased part of the patient, for example, the diseased spine. The two-dimensional medical image can be an X-ray image obtained during surgery.

[0063] The neural network is used to determine the rotation parameters of the projection direction of a two-dimensional medical image. The neural network is trained using a large number of training samples, which are X-ray images from different patients. The X-ray images used as training samples are marked with the rotation parameters of their projection direction.

[0064] Regarding the labeling of training samples, since the patient's posture is relatively fixed during the actual operation, it can generally be divided into two categories: lying prone on the operating table or lying on the side on the operating table. According to the imaging posture of the target object in the two-dimensional medical image, at least two of the three rotation parameters can be determined, and the remaining rotation parameter can be selected between 90°, 0°, and -90°.

[0065] In the training process of the neural network, the training samples can be divided into a training set and a validation set according to a 9:1 ratio, but not limited to it. The samples in the training set are input into the neural network in sequence, and the parameters of the neural network are adjusted according to the output results of the neural network and the sample labels of the corresponding samples. Among them, the loss function of the model training can be, but not limited to, the cross entropy loss function. In the process of model training, the validation set can also be used for early stopping, and the trained neural network can be used for the rotation parameters of two-dimensional medical images.

[0066] Step 102: Project the three-dimensional medical image of the target object based on the rotation parameters to obtain a two-dimensional digitally reconstructed image with a maximum field of view.

[0067] The three-dimensional medical image may be, for example, a CT (computed tomography) image, an MRI (magnetic resonance) image, a PET (positron emission tomography) image, or the like.

[0068] In step 102, the 3D medical image is globally projected based on the rotation parameters. Different from the related art that only projects an image with the same window size as the 2D medical image, the DRR algorithm used in this embodiment projects the 3D medical image in the projection direction of the 2D medical image with the maximum field of view. In the projection process, four parameters are actually needed for global projection, including three rotation parameters (Δθ x , Δθ y , Δθ z ) and deltaZ; wherein deltaZ can be directly given by the SHR (Spacing Height Ratio) of the two-dimensional medical image to obtain a global two-dimensional digitally reconstructed image (DRR image).

[0069] In order to find a position in the two-dimensional digital reconstructed image that is most similar to the two-dimensional medical image and realize image registration in the following steps, the SHR of the two-dimensional digital reconstructed image must be the same as that of the two-dimensional medical image.

[0070] Step 103: Determine the projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image respectively.

[0071] Figure 2 This is a flow chart showing a method of determining a projection area of ​​a target object in a two-dimensional digitally reconstructed image according to an exemplary embodiment of the present invention, which may include the following steps:

[0072] Step 103 - 1 a: perform a fixed threshold binarization process on the two-dimensional digital reconstructed image to obtain a first binarized image.

[0073] The setting of the fixed threshold is related to the target object. Taking the target object as a bone as an example, it is necessary to determine the projection area of ​​the bone in the two-dimensional digital reconstructed image. The grayscale value of the bone in the CT image is greater than 150. The fixed threshold can be set to 200. For each pixel in the two-dimensional digital reconstructed image, if the grayscale value of the pixel is less than 200, the pixel value of the pixel is replaced with 0; if the grayscale value of the pixel is greater than or equal to 200, the pixel value of the pixel is replaced with 255, thereby distinguishing the projection area of ​​the target object in the two-dimensional digital reconstructed image from other areas, and preliminarily determining the bone contour in the two-dimensional digital reconstructed image.

[0074] In one embodiment, before step 103 - 1a, the two-dimensional digital reconstructed image may be subjected to histogram equalization processing to make the image color of the two-dimensional digital reconstructed image evenly distributed, thereby ensuring the processing effect of the fixed threshold binarization in step 103 - 1a and making the result more robust.

[0075] In order to prevent the high-frequency noise of the image from having too great an impact on the binarization result, the two-dimensional digital reconstructed image after the histogram equalization processing can also be subjected to Gaussian filtering processing to achieve image smoothing operation. In step 103-1a, the binarization processing is performed based on the two-dimensional digital reconstructed image after the Gaussian filtering processing. The convolution kernel size used in the Gaussian filtering processing can be, but is not limited to, 5*5 pixels.

[0076] Since the originally stored three-dimensional medical image, such as CT image data, is a floating-point type of data, the two-dimensional digital reconstructed image obtained from the floating-point type of CT image data is also a floating-point type, before the two-dimensional digital reconstructed image is subjected to histogram equalization, the pixel value of the two-dimensional digital reconstructed image can be discretized to 0-255, and the discretized two-dimensional digital reconstructed image can be subjected to histogram equalization to improve the speed of subsequent image processing. Of course, before obtaining the two-dimensional digital reconstructed image, the pixel value of the three-dimensional medical image can also be discretized to 0-255, and the discretized three-dimensional medical image can be subjected to projection processing.

[0077] Step 103 - 2a: perform adaptive threshold binarization processing on the two-dimensional digital reconstructed image to obtain a second binarized image.

[0078] The two-dimensional digital reconstructed image in step 103-2a may also be an image that has been processed by histogram equalization and Gaussian filtering. In step 103-2a, the two-dimensional digital reconstructed image that has been processed as above is binarized using an adaptive threshold. The binarization is performed by adaptively setting a threshold based on the local area features of the two-dimensional digital reconstructed image. The size of the local area may be, but is not limited to, 35*35 pixels.

[0079] Step 103 - 3a: determine a projection area of ​​the target object in the two-dimensional digital reconstructed image according to the first binarized image and the second binarized image.

[0080] Step 103 - 3a is to multiply the first binary image by the second binary image to obtain the projection area of ​​the target object in the two-dimensional digital reconstructed image, that is, the contour of the target object.

[0081] In this embodiment, the fixed threshold can use the information of the entire image for binarization, and the adaptive threshold can better perform local binarization. The former ensures the overall binarization effect to remove most of the non-bone (target object) parts, while the latter is used to process the details between bones. Combining the advantages of the two can well extract the bone contours in the two-dimensional digital reconstructed image.

[0082] Figure 3 The flowchart of determining a projection area of ​​a target object in a two-dimensional medical image according to an exemplary embodiment of the present invention may include the following steps:

[0083] Step 103 - 1b: perform binarization processing on the two-dimensional medical image with a fixed threshold to obtain a third binarized image.

[0084] The setting of the fixed threshold is related to the target object. Taking the target object as a bone as an example, it is necessary to determine the projection area of ​​the bone in the two-dimensional medical image. The grayscale value of the bone in the CT image is greater than 150. The fixed threshold can be set to 200. For each pixel in the two-dimensional medical image, if the grayscale value of the pixel is less than 200, the pixel value of the pixel is replaced with 0; if the grayscale value of the pixel is greater than or equal to 200, the pixel value of the pixel is replaced with 255, thereby preliminarily determining the bone contour in the two-dimensional medical image.

[0085] In one embodiment, before step 103 - 1b, histogram equalization processing may be performed on the two-dimensional medical image to make the image color of the two-dimensional medical image evenly distributed, thereby ensuring the processing effect of the fixed threshold binarization in step 103 - 1b and making the result more robust.

[0086] In order to prevent the high-frequency noise of the image from having too great an impact on the binarization result, the two-dimensional medical image processed by histogram equalization can also be subjected to Gaussian filtering to achieve image smoothing. In step 103-1b, binarization is performed based on the two-dimensional medical image processed by Gaussian filtering. The convolution kernel size used in the Gaussian filtering can be, but is not limited to, 5*5 pixels.

[0087] Since the originally stored two-dimensional medical images, such as X-ray image data, are floating-point data, before performing histogram equalization on the X-ray image, the pixel values ​​of the X-ray image can be discretized to 0-255, and the discretized X-ray image can be subjected to histogram equalization to improve the speed of image processing.

[0088] Step 103 - 2b: perform adaptive threshold binarization processing on the two-dimensional medical image to obtain a fourth binarized image.

[0089] The two-dimensional medical image in step 103-2b may also be an image that has been processed by histogram equalization and Gaussian filtering. In step 103-2b, the two-dimensional medical image that has been processed as above is binarized using an adaptive threshold. The binarization is performed by adaptively setting a threshold based on the local area features of the two-dimensional medical image. The size of the local area may be, but is not limited to, 35*35 pixels.

[0090] Step 103 - 3b: determine a projection area of ​​the target object in the two-dimensional medical image according to the third binarized image and the fourth binarized image.

[0091] Step 103 - 3b is to multiply the third binarized image by the fourth binarized image to obtain the projection area of ​​the target object in the two-dimensional medical image, that is, the contour of the target object.

[0092] In this embodiment, the fixed threshold can use the information of the entire image for binarization, and the adaptive threshold can better perform local binarization. The former ensures the overall binarization effect to remove most of the non-bone (target object) parts, while the latter is used to process the details between bones. Combining the advantages of the two can well extract the bone contours in two-dimensional medical images.

[0093] Step 104 : determining a translation parameter of the three-dimensional medical image relative to the two-dimensional medical image according to the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image.

[0094] When determining the translation parameters of a three-dimensional medical image relative to a two-dimensional medical image, a template matching algorithm can be used, and the projection area E1 of the target object in the two-dimensional medical image is used as a template, and the projection area E2 of the target object in the two-dimensional digital reconstructed image is used as the image area to be registered. The projection area E1 is slid on the projection area E2, and the sliding step size can be set according to actual needs, for example, it is set to 1 pixel. Each time the projection area E1 moves on the projection area E2, the similarity between the projection area E1 and the projection area E2 is calculated, and the translation parameter is determined based on the relative position of the projection area E1 and the projection area E2 when the similarity is the maximum value.

[0095] The template matching algorithm can be used to quickly obtain the optimal matching position of the projection area E1 and the projection area E2, and determine at least two of the translation parameters. The remaining translation parameter can be determined based on the above at least two translation parameters while ensuring that the SHR of the two-dimensional digital reconstructed image is always consistent with the SHR of the two-dimensional medical image.

[0096] The similarity measure between the projection area E1 and the projection area E2 may be, but is not limited to, NCC (Normalization cross correlation).

[0097] The rotation parameters determined in step 101 and the translation parameters determined in step 104 may represent the registration information of the three-dimensional medical image to the two-dimensional medical image. The registration information is the registration information of the coarse registration stage and is used to provide a reference for the fine registration stage.

[0098] In the embodiment of the present invention, during the image registration process, the parameters are decoupled into two processes of solving rotation parameters and translation parameters. A single DRR projection is used to complete the coarse registration. Compared with the exhaustive search and joint parameter solution methods of the related art, the computational complexity is reduced and the solution speed is accelerated.

[0099] Corresponding to the above-mentioned image registration method embodiment, the present invention also provides an embodiment of an image registration device.

[0100] Figure 4 is a schematic diagram of a module of an image registration device according to an exemplary embodiment of the present invention, which may include:

[0101] An input module 41, for inputting a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing a projection direction of the two-dimensional medical image;

[0102] A projection module 42, configured to project the three-dimensional medical image of the target object based on the rotation parameter to obtain a two-dimensional digitally reconstructed image with a maximum field of view;

[0103] A determination module 43, used to determine the projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image respectively;

[0104] The determination module 43 is also used to determine the translation parameters of the three-dimensional medical image relative to the two-dimensional medical image based on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and the translation parameters and the rotation parameters represent the registration information of the three-dimensional medical image to the two-dimensional medical image.

[0105] Optionally, the determining module includes:

[0106] A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional digital reconstructed image to obtain a first binarized image;

[0107] A second binarization unit is used to perform a binarization process of the two-dimensional digital reconstructed image with an adaptive threshold to obtain a second binarized image;

[0108] A determination unit is used to determine a projection area in the two-dimensional digital reconstructed image according to the first binarized image and the second binarized image.

[0109] Optionally, the determining module is used to:

[0110] A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional medical image to obtain a third binarized image;

[0111] A second binarization unit is used to perform an adaptive threshold binarization process on the two-dimensional medical image to obtain a fourth binarized image;

[0112] A determining unit is used to determine a projection area in the two-dimensional medical image according to the third binarized image and the fourth binarized image.

[0113] Optionally, it also includes:

[0114] An equalization processing module, used for performing histogram equalization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image;

[0115] The filtering module is used to perform Gaussian filtering on the two-dimensional digital reconstructed image or the two-dimensional medical image after the histogram equalization processing, so as to perform binarization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image after the Gaussian filtering processing.

[0116] Optionally, the determining module includes:

[0117] A matching unit is used to perform template matching on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and determine the translation parameter according to the template matching result.

[0118] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0119] Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present invention, and shows a block diagram of an exemplary electronic device 50 suitable for implementing the embodiments of the present invention. Figure 5 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0120] like Figure 5As shown, the electronic device 50 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 50 may include, but are not limited to: at least one processor 51, at least one memory 52, and a bus 53 connecting different system components (including the memory 52 and the processor 51).

[0121] The bus 53 includes a data bus, an address bus, and a control bus.

[0122] The memory 52 may include a volatile memory, such as a random access memory (RAM) 521 and / or a cache memory 522 , and may further include a read-only memory (ROM) 523 .

[0123] The memory 52 may also include a program tool 525 (or utility) having a set (at least one) of program modules 524, such program modules 524 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0124] The processor 51 executes various functional applications and data processing by running the computer program stored in the memory 52, such as the method provided in any of the above embodiments.

[0125] The electronic device 50 may also communicate with one or more external devices 54 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 55. Furthermore, the model-generated electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 56. As shown, the network adapter 56 communicates with other modules of the model-generated electronic device 50 via a bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated electronic device 50, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0126] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0127] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method provided by any of the above embodiments is implemented.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image registration method, characterized in that: include: Inputting a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing a projection direction of the two-dimensional medical image; Performing projection processing on the three-dimensional medical image of the target object based on the rotation parameters to obtain a two-dimensional digitally reconstructed image with a maximum field of view; respectively determining a projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image; The translation parameters of the three-dimensional medical image relative to the two-dimensional medical image are determined according to the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and the translation parameters and the rotation parameters represent the registration information of the three-dimensional medical image to the two-dimensional medical image.

2. The image registration method according to claim 1, characterized in that: Determining a projection area of ​​the target object in the two-dimensional digital reconstructed image includes: Performing a fixed threshold binarization process on the two-dimensional digital reconstructed image to obtain a first binarized image; Performing adaptive threshold binarization processing on the two-dimensional digital reconstructed image to obtain a second binarized image; The projection area is determined according to the first binarized image and the second binarized image.

3. The image registration method according to claim 1, characterized in that: Determining a projection area of ​​the target object in the two-dimensional medical image includes: Performing a fixed threshold binarization process on the two-dimensional medical image to obtain a third binarized image; Performing adaptive threshold binarization processing on the two-dimensional medical image to obtain a fourth binarized image; The projection area is determined according to the third binarized image and the fourth binarized image.

4. The image registration method according to claim 2 or 3, characterized in that: Before binarizing the two-dimensional digital reconstructed image or the two-dimensional medical image, the method further includes: Performing histogram equalization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image; Gaussian filtering is performed on the two-dimensional digital reconstructed image or the two-dimensional medical image that has undergone the histogram equalization processing, so as to perform binarization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image that has undergone the Gaussian filtering processing.

5. The image registration method according to claim 1, characterized in that: Determining a translation parameter of the three-dimensional medical image relative to the two-dimensional medical image includes: Performing template matching on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image; The translation parameter is determined according to the template matching result.

6. The image registration method according to claim 1, characterized in that: The two-dimensional digitally reconstructed image has the same pitch-to-height ratio as the two-dimensional medical image.

7. An image registration device, characterized in that: include: An input module, configured to input a two-dimensional medical image of a target object into a pre-trained neural network to determine a rotation parameter characterizing a projection direction of the two-dimensional medical image; A projection module, used for projecting the three-dimensional medical image of the target object based on the rotation parameter to obtain a two-dimensional digitally reconstructed image with a maximum field of view; A determination module, used to determine the projection area of ​​the target object in the two-dimensional digital reconstructed image and the two-dimensional medical image respectively; The determination module is also used to determine the translation parameters of the three-dimensional medical image relative to the two-dimensional medical image based on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and the translation parameters and the rotation parameters represent the registration information of the three-dimensional medical image to the two-dimensional medical image.

8. The image registration device according to claim 7, characterized in that: The determination module comprises: A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional digital reconstructed image to obtain a first binarized image; A second binarization unit is used to perform a binarization process of the two-dimensional digital reconstructed image with an adaptive threshold to obtain a second binarized image; A determination unit is used to determine a projection area in the two-dimensional digital reconstructed image according to the first binarized image and the second binarized image.

9. The image registration device according to claim 7, characterized in that: The determination module comprises: A first binarization unit is used to perform a binarization process with a fixed threshold on the two-dimensional medical image to obtain a third binarized image; A second binarization unit is used to perform an adaptive threshold binarization process on the two-dimensional medical image to obtain a fourth binarized image; A determining unit is used to determine a projection area in the two-dimensional medical image according to the third binarized image and the fourth binarized image.

10. The image registration device according to claim 8 or 9, characterized in that: Also includes: An equalization processing module, used for performing histogram equalization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image; The filtering module is used to perform Gaussian filtering on the two-dimensional digital reconstructed image or the two-dimensional medical image after the histogram equalization processing, so as to perform binarization processing on the two-dimensional digital reconstructed image or the two-dimensional medical image after the Gaussian filtering processing.

11. The image registration device according to claim 7, characterized in that: The determining module comprises: A matching unit is used to perform template matching on the projection area in the two-dimensional digital reconstructed image and the projection area in the two-dimensional medical image, and determine the translation parameter according to the template matching result.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image registration method according to any one of claims 1 to 6 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image registration method according to any one of claims 1 to 6 are implemented.