Image registration method and device
By performing wavelet transformation and Laplace pyramid correction registration on ultrasound images and CT/MRI images, calculating the CC function value, the problem of error-prone image registration in the prior art is solved, and the accuracy and reliability of image registration are improved sex.
Patent Information
- Application Number
- CN201911214987.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-12-02
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2039-12-02
AI Technical Summary
Existing image registration technology can easily lead to errors in ultrasonic volume navigation, resulting in registration errors, affecting the accurate positioning of the lesion site.
By performing wavelet transformation on the registered image and the reference image, the low frequency, horizontal high frequency, vertical high frequency and diagonal high frequency components are extracted respectively, and the registration is corrected by the Laplace pyramid, the CC function value of each component is calculated, and the accumulation is accumulated. And the image exceeding the threshold is used as the registration result.
It improves the accuracy, directionality and displacement tolerance of image registration, reduces errors, and enhances the accuracy and reliability of registration.
Smart Images

Figure CN112991403B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to image processing, and in particular to an image registration method and device. Background Art
[0002] In recent years, a new ultrasound volume navigation technology has been developed. Its principle is to fuse the real-time or static sonogram with the H-dimensional data of CT or MR collected before the operation in the process of ultrasound search for lesions. After obtaining the fused image, the electromagnetic field tracking system is used to assist doctors to find the lesion site more quickly in the real-time ultrasound image. This technology is a combination of electromagnetic navigation technology and image fusion technology. It requires the ultrasound system to have an electromagnetic tracking device and image registration and fusion algorithms. The latter can realize multi-mode image registration, fusion, and visual navigation in percutaneous interventional treatment or puncture biopsy. The image registration and fusion algorithm can register and fuse the patient's preoperative CT / MRI image with the ultrasound sonogram during the operation in real time, and the position and trajectory of the needle tip can be displayed in the sonogram in real time. However, the image registration technology currently used is prone to obtaining wrong window pairs, resulting in registration errors. Summary of the invention
[0003] The present application provides an image registration method and device.
[0004] According to a first aspect of the present application, the present application provides an image registration method, comprising:
[0005] Input the image to be registered and the reference image and perform wavelet transform respectively to obtain the corresponding sub-image of the image to be registered and the sub-image of the reference image;
[0006] Registering the sub-image of the image to be registered, and calculating the CC function value of the sub-image of the image to be registered according to the sub-image of the reference image and the registered sub-image of the image to be registered;
[0007] The modified registration sub-image is obtained according to the CC function value.
[0008] Furthermore, the image sub-image to be registered and the reference image sub-image respectively include a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component;
[0009] The registering the sub-images of the image to be registered comprises:
[0010] Perform Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component, and the diagonal high-frequency component of the image sub-image to be registered respectively.
[0011] Further, the performing Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the image sub-image to be registered respectively includes:
[0012] Starting from the bottom layer image, respectively, for the multi-layer images of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the sub-image to be registered, the registration parameters of the layer image are estimated by the translation amount and the rotation angle, and the CC function value of each component is calculated;
[0013] Adjust the components of the next layer of sub-images through the registration parameters of the layer of sub-images until the highest layer of sub-images;
[0014] The CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component, and the diagonal high frequency component of the highest layer sub-image are recorded.
[0015] Furthermore, obtaining a modified registration sub-image according to the CC function value includes:
[0016] Accumulating CC function values of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the highest-level sub-image respectively;
[0017] The image with the maximum cumulative sum and exceeding the predetermined threshold is output as the registered image.
[0018] According to a second aspect of the present application, the present application provides an image registration device, comprising:
[0019] A conversion module, used for inputting the image to be registered and the reference image and performing wavelet transform respectively to obtain a corresponding sub-image of the image to be registered and a sub-image of the reference image;
[0020] A registration module, used for registering the sub-image of the image to be registered, and calculating the CC function value of the sub-image of the image to be registered according to the sub-image of the reference image and the registered sub-image of the image to be registered;
[0021] An output module is used to obtain a corrected registration sub-image according to the CC function value.
[0022] Furthermore, the image sub-image to be registered and the reference image sub-image respectively include a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component;
[0023] The registration module is further used to perform Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the image sub-image to be registered.
[0024] Furthermore, the registration module comprises:
[0025] A processing unit, for respectively estimating the registration parameters of the layer image through the translation amount and the rotation angle for the multi-layer image of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the sub-image to be registered, starting from the bottom layer image, and calculating the CC function value of each component;
[0026] An adjustment unit, used for adjusting the sub-image components of the next layer up to the highest layer sub-image by using the registration parameters of the sub-image of the layer;
[0027] A recording unit is used to record CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component and the diagonal high frequency component of the highest layer sub-image.
[0028] Furthermore, the output module includes:
[0029] An accumulation unit, used for respectively accumulating CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component and the diagonal high frequency component of the highest layer sub-image;
[0030] The output unit is used to output the image with the maximum cumulative sum and exceeding a predetermined threshold as the registration image.
[0031] According to a third aspect of the present application, the present application provides an image registration device, comprising:
[0032] Memory, used to store programs;
[0033] A processor is used to implement the above method by executing the program stored in the memory.
[0034] According to a fourth aspect of the present application, the present application provides a computer-readable storage medium, comprising a program, wherein the program can be executed by a processor to implement the above method.
[0035] The beneficial effects of this application are (or do not write this sentence, for applications to be filed in the United States, the beneficial effects part can be omitted):
[0036] Due to the adoption of the above technical solution, the beneficial effects of this application are:
[0037] In the embodiment of the present application, the wavelet transform is performed on the image to be registered and the reference image respectively to obtain the corresponding sub-image of the image to be registered and the sub-image of the reference image; the sub-image of the image to be registered is registered, and the CC function value of the sub-image of the image to be registered is calculated according to the sub-image of the reference image and the registered sub-image of the image to be registered; and the corrected registered image is obtained according to the CC function value. The present application solves the CC function for the sub-images of multiple scales and multiple angles after wavelet decomposition, and registers multiple multi-scale component sub-images through its multi-level characteristics, which makes appropriate improvements to the error, accuracy, directionality, and displacement tolerance to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 A flowchart of a method of the present application in one embodiment;
[0039] Figure 2 A flow chart of the method of the present application in another embodiment;
[0040] Figure 3 A schematic diagram of a program module of the device of the present application in one embodiment;
[0041] Figure 4 This is a schematic diagram of a program module of the device of the present application in another embodiment. DETAILED DESCRIPTION
[0042] The present invention is further described in detail below through specific implementations in combination with the accompanying drawings. The present application can be implemented in a variety of different forms and is not limited to the implementation described in this embodiment. The purpose of providing the following specific implementations is to facilitate a clearer and more thorough understanding of the disclosure of the present application, wherein words indicating directions such as up, down, left, and right are only for the position of the structure shown in the corresponding drawings.
[0043] However, those skilled in the art may realize that one or more of the specific details may be omitted, or other methods, components or materials may be used. In some examples, some embodiments are not described or are not described in detail.
[0044] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.
[0045] In addition, the technical features and technical solutions described herein may be combined in any suitable manner in one or more embodiments. It is easy for a person skilled in the art to understand that the steps or operation sequence of the methods related to the embodiments provided herein may also be changed. Therefore, any sequence in the drawings and embodiments is only for illustrative purposes and does not imply a requirement to follow a certain sequence unless it is explicitly stated that a certain sequence is required.
[0046] The CC function is solved for sub-images of multiple scales and angles after wavelet decomposition, and through its multi-level characteristics, multiple multi-scale component sub-images are registered using a Laplace pyramid registration method. There is a CC function at the highest level of the sub-image, with a total of four CC function values, including low frequency, horizontal high frequency, vertical high frequency, and diagonal high frequency. Each CC function value carries different registration information. If a single CC function value is used as the criterion, it is easy to introduce errors. The four CC function values are accumulated, and the one that exceeds the threshold and is the largest is the registration result. To a certain extent, the error, accuracy, directionality, and displacement tolerance are appropriately improved.
[0047] Embodiment 1:
[0048] like Figure 1 As shown, the image registration method of the present application, in one embodiment, comprises the following steps:
[0049] Step 102: input the image to be registered and the reference image and perform wavelet transform respectively to obtain corresponding sub-images of the image to be registered and sub-images of the reference image.
[0050] Step 104: registering the sub-image of the image to be registered, and calculating the CC function value of the sub-image of the image to be registered according to the sub-image of the reference image and the registered sub-image of the image to be registered.
[0051] The CC function value can be calculated by the following formula:
[0052]
[0053] Where W is the original image window pair image, E represents the mean, I is the window selected for the image to be registered, and (i, j) are the coordinates of the points that make up the image within the window.
[0054] Step 106: Obtain a corrected registration image according to the CC function value.
[0055] like Figure 2 As shown, the image registration method of the present application, in another embodiment, comprises the following steps:
[0056] Step 202: input the image to be registered and the reference image respectively.
[0057] Step 204: performing wavelet decomposition on the image to be registered and the reference image respectively to obtain sub-images of multiple scales, multiple directions and multiple levels.
[0058] After performing wavelet decomposition on the image to be registered, a sub-image of the image to be registered is obtained, and the sub-image of the image to be registered includes low-frequency components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components. After performing wavelet decomposition on the reference image, a sub-image of the reference image is obtained, and the sub-image of the reference image also includes low-frequency components, horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components.
[0059] Step 206: respectively obtain the bottom layer sub-images of the low frequency component, horizontal high frequency component, vertical high frequency component and diagonal high frequency component of the sub-image of the image to be registered. At this time, the bottom layer sub-image is the current sub-image.
[0060] Step 208: Calculate the registration parameters of the sub-image of this layer. The registration parameters include calculating the translation Δx and the rotation angle And calculate the CC function value of the sub-image of this layer.
[0061] Step 210: Determine whether the layer sub-image is the highest layer sub-image; if so, go to step 214; otherwise, go to step 212.
[0062] Step 212 : Use the registration parameters of the sub-image of this layer to adjust the sub-image of the lower layer, increase the registration level by 1, convert the sub-image of the lower layer into the current sub-image, and go to step 208 .
[0063] Step 214: Obtain the CC function value of the highest level sub-image of each component of the sub-image of the reference image.
[0064] Furthermore, the method of the present application can perform Laplacian pyramid correction registration on one or more of the low-frequency component, horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component of the image sub-image to be registered. In this embodiment, the four components of the low-frequency component, horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component of the image sub-image to be registered can be Laplacian pyramid correction registration. Specifically, one of the four components can be registered first to obtain the CC function value of the highest-level sub-image of the component, and then another component can be registered to obtain the CC function value of its highest-level sub-image, until the CC function values of the highest-level sub-images of all components are obtained. Alternatively, the bottom-level sub-images of the four components can be registered at the same time, and then the next-level sub-images of the four components can be registered, until the CC function values of the highest-level sub-images of all components are obtained.
[0065] Step 216: Accumulate the CC function values of the low-frequency component, horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component of the highest-level sub-image.
[0066] Step 218: Output the image with the maximum cumulative sum and exceeding a predetermined threshold as the registered image.
[0067] Embodiment 2:
[0068] like Figure 3 , Figure 4 As shown, the image registration device 300 of the present application, in one implementation manner, includes a conversion module 310 , a registration module 320 and an output module 330 .
[0069] The conversion module 310 is used to input the image to be registered and the reference image and perform wavelet transform respectively to obtain the corresponding sub-image of the image to be registered and the sub-image of the reference image;
[0070] A registration module 320, used to register the sub-image of the image to be registered, and calculate the CC function value of the sub-image of the image to be registered according to the reference image sub-image and the registered sub-image of the image to be registered;
[0071] The output module 330 is used to obtain a corrected registration image according to the CC function value.
[0072] Further, the image sub-image to be registered and the reference image sub-image respectively include a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component;
[0073] The registration module 320 is further used to perform Laplacian pyramid correction registration on one or more of the low frequency component, horizontal high frequency component, vertical high frequency component and diagonal high frequency component of the sub-image to be registered.
[0074] Furthermore, the registration module 320 may include a processing unit 321 , an adjustment unit 322 , and a recording unit 323 .
[0075] The processing unit 321 is used to estimate the registration parameters of the layer image through the translation amount and the rotation angle for the low frequency component, horizontal high frequency component, vertical high frequency component and diagonal high frequency component of the multi-layer sub-image of the image sub-image to be registered, starting from the bottom layer sub-image, and calculate the CC function value of each component;
[0076] An adjustment unit 322, configured to adjust the sub-image components of the next layer up to the highest layer sub-image by using the registration parameters of the sub-image of the layer;
[0077] The recording unit 323 is used to record the CC function values of the low frequency component, horizontal high frequency component, vertical high frequency component and diagonal high frequency component of the highest layer sub-image.
[0078] Further, the output module 330 may include an accumulation unit 331 and an output unit 332 .
[0079] An accumulation unit, used for respectively accumulating CC function values of a low frequency component, a horizontal high frequency component, a vertical high frequency component and a diagonal high frequency component of a top layer sub-image;
[0080] The output unit is used to output the image with the maximum cumulative sum and exceeding a predetermined threshold as the registration image.
[0081] Embodiment three:
[0082] An image registration device of the present application, in one implementation manner, includes a memory and a processor.
[0083] Memory, used to store programs;
[0084] The processor is used to implement the method in the first embodiment by executing the program stored in the memory.
[0085] Embodiment 4:
[0086] A computer-readable storage medium includes a program, and the program can be executed by a processor to implement the method in embodiment 1.
[0087] Those skilled in the art will appreciate that all or part of the steps of the various methods in the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which may include: a read-only memory, a random access memory, a disk or an optical disk, etc.
[0088] The above contents are further detailed descriptions of the present application in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application.
Claims
1. An image registration method, characterized in that: include: Input the image to be registered and the reference image and perform wavelet transform respectively to obtain the corresponding sub-image of the image to be registered and the sub-image of the reference image; Registering the sub-image of the image to be registered, and calculating the CC function value of the sub-image of the image to be registered according to the sub-image of the reference image and the registered sub-image of the image to be registered; Acquire a corrected registration image according to the CC function value; The image sub-image to be registered and the reference image sub-image respectively include a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component; Wherein, obtaining the corrected registration image according to the CC function value includes: Accumulating CC function values of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the highest-level sub-image respectively; Output the image with the maximum cumulative sum and exceeding a predetermined threshold as the registration image; The CC function value is specifically calculated by the following formula: Where W is the original image window pair image, E represents the mean, I is the window selected for the image to be registered, and (i, j) are the coordinates of the points that make up the image within the window.
2. The method according to claim 1, characterized in that The registering the sub-images of the image to be registered comprises: Perform Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component, and the diagonal high-frequency component of the image sub-image to be registered respectively.
3. The method according to claim 2, characterized in that The performing Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the sub-image of the image to be registered respectively comprises: Starting from the bottom layer image, respectively, for the multi-layer images of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the sub-image to be registered, the registration parameters of the layer image are estimated by the translation amount and the rotation angle, and the CC function value of each component is calculated; Adjust the components of the next layer of sub-images through the registration parameters of the layer of sub-images until the highest layer of sub-images; The CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component, and the diagonal high frequency component of the highest layer sub-image are recorded.
4. An image registration device, characterized in that: include: A conversion module, used for inputting the image to be registered and the reference image and performing wavelet transform respectively to obtain a corresponding sub-image of the image to be registered and a sub-image of the reference image; A registration module, used for registering the sub-image of the image to be registered, and calculating the CC function value of the sub-image of the image to be registered according to the sub-image of the reference image and the registered sub-image of the image to be registered; An output module, used for obtaining a corrected registration image according to the CC function value; The image sub-image to be registered and the reference image sub-image respectively include a low-frequency component, a horizontal high-frequency component, a vertical high-frequency component and a diagonal high-frequency component; The output module comprises: An accumulation unit, used for respectively accumulating CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component and the diagonal high frequency component of the highest layer sub-image; An output unit, used for outputting an image with a maximum cumulative sum and exceeding a predetermined threshold as a registration image; The CC function value is specifically calculated by the following formula: Where W is the original image window pair image, E represents the mean, I is the window selected for the image to be registered, and (i, j) are the coordinates of the points that make up the image within the window.
5. The device according to claim 4, characterized in that The registration module is further used to perform Laplacian pyramid correction registration on one or more of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the image sub-image to be registered.
6. The device according to claim 5, characterized in that The registration module comprises: A processing unit, for respectively estimating the registration parameters of the layer image through the translation amount and the rotation angle for the multi-layer image of the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component and the diagonal high-frequency component of the sub-image to be registered, starting from the bottom layer image, and calculating the CC function value of each component; An adjustment unit, used for adjusting the sub-image components of the next layer up to the highest layer sub-image by using the registration parameters of the sub-image of the layer; A recording unit is used to record CC function values of the low frequency component, the horizontal high frequency component, the vertical high frequency component and the diagonal high frequency component of the highest layer sub-image.
7. An image registration device, characterized in that: include: Memory, used to store programs; A processor, configured to implement the method according to any one of claims 1 to 3 by executing the program stored in the memory.
8. A computer-readable storage medium, characterized in that: The method comprises a program which can be executed by a processor to implement the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Infrared and visual pseudo-color image fusion and enhancement method
CN101714251A
Neighborhood normalized gradient and neighborhood standard deviation-based multi-focus image fusion method
CN102063713A
Method for fusing typhoon cloud pictures based on NSCT (Nonsubsampled Controurlet Transformation) and particle swarm optimization algorithm
CN102063715A
Image registration method and device, equipment and storage medium
CN108830886A
Infrared and visible light image fusion method based on regional property fuzzy
CN1822046A
