Fundus Image Registration Method and System Based on Two-Stage Parameterized Deformation Modeling

Abstract

The present disclosure provides a fundus image registration method and system based on two-stage parametric deformation modeling, which relates to the technical field of fundus image processing. The method includes obtaining a target image and a floating image, and respectively extracting their G channels; based on the G channels of the target image and the floating image, a coarse-fine two-stage parametric method is used for registration to obtain a registered image. Among them, in the coarse registration stage, by detecting the centroids of the optic disc and fovea and the blood vessel bifurcation points, an affine transformation is quickly estimated using three pairs of feature points to obtain an initial deformation field; in the fine registration stage, a multi-scale iterative optimization strategy is adopted to parametrically model the gray-scale transformation relationship between the initial deformation field and the target image at each scale, and the complete parametric deformation field is iteratively parametrically fitted and inferred; the parametric deformation field obtained by fitting and inference is upsampled, and the original input floating image is deformed using the upsampled parametric deformation field to obtain the final registered image.

Description

Technical Field

[0001] The present disclosure relates to the technical field of fundus image processing, and particularly to a fundus image registration method and system based on two-stage parametric deformation modeling. Background Art

[0002] The statements in this part merely provide background technical information related to the present disclosure and do not necessarily constitute prior art.

[0003] In the early stage of fundus diseases, there are usually no obvious clinical symptoms and they are extremely easy to be ignored. Therefore, it is necessary to rely on professional fundus examination equipment for accurate diagnosis. As a key basis for ophthalmic diagnosis, fundus images contain rich medical information, including key pathological indicators such as blood vessel distribution characteristics, positions of bleeding points, pigment deposition areas, and cyst morphologies. These information provide important references for the diagnosis of various ophthalmic diseases. However, it is necessary to perform registration processing on fundus images. As an efficient processing strategy, fundus image registration technology can not only assist doctors in comprehensively and quickly analyzing lesions and helping to improve the diagnosis and treatment effect, but also is the core technology of an automatic auxiliary diagnosis system for fundus diseases based on artificial intelligence.

[0004] The challenges faced by fundus image registration mainly include poor image quality, modality differences, eye movement and projection distortion, pathological changes, large displacements and large deformations, high computational complexity, lack of accurate evaluation criteria, multi-scale features, and illumination and shadow interference, etc. These challenges make fundus image registration a complex and challenging task.

[0005] In some existing fundus registration schemes, feature-based methods, intensity-based methods, and deep learning-based methods are mostly used. However, the existing registration methods still have the following problems:

[0006] 1) Feature-based methods have poor robustness in low-quality images, small overlapping regions, or textureless scenarios;

[0007] 2) Intensity-based methods are sensitive to non-uniform illumination and random intensity changes, and have high computational complexity;

[0008] 3) Existing evaluation methods rely on subjective judgment or manual annotation, and the objective quantitative indicators are not robust;

[0009] 4) Elastic registration methods (such as optical flow methods) have inaccurate estimation of large displacements and rely on the assumption of gray constancy, and there are certain limitations for large-scale deformation registration. Summary of the Invention

[0010] To solve the above problems, the present disclosure proposes a fundus image registration method and system based on two-stage parametric deformation modeling, and proposes a parametric modeling method for adaptively adjusting the complexity of the deformation field, performing coarse-fine two-stage parametric deformation modeling registration. In the first stage, an affine transformation is calculated by extracting three pairs of feature points, and in the second stage, multi-scale LAP iteration is performed. The local all-pass filter (LAP) is used to iteratively optimize in combination with the polynomial displacement field model, systematically solving the core defects of the existing retinal image registration technology in terms of robustness, efficiency, accuracy, and applicability, and providing an efficient and reliable automated tool for clinical pathological analysis.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions:

[0012] A fundus image registration method based on two-stage parametric deformation modeling, comprising:

[0013] Obtain a target image and a floating image, downsample them, and extract the G channels of the target image and the floating image respectively;

[0014] Based on the extracted G channels of the target image and the floating image, perform registration using a coarse-fine two-stage parametric method to obtain a final registered image;

[0015] Among them, in the coarse registration stage, by detecting the centroids of the optic disc and the fovea and the vascular bifurcation points, an affine transformation is quickly estimated using three pairs of feature points to obtain an initial deformation field; in the fine registration stage, a multi-scale iterative optimization strategy is adopted to construct an inverted pyramid model. Based on the inverted pyramid model, the gray-scale transformation between the initial deformation field and the target image is parametrically modeled at each scale, and the deformation field and the gray-scale transformation are iteratively calculated, and the complete parametric deformation field is inferred by parametric fitting.

[0016] Perform upsampling processing on the inferred parametric deformation field by fitting, and deform the original input floating image using the upsampled parametric deformation field to obtain a final registered image.

[0017] According to some embodiments, the present disclosure adopts the following technical solutions:

[0018] A fundus image registration system based on two-stage parametric deformation modeling, comprising:

[0019] An image acquisition module, configured to obtain a target image and a floating image, downsample them, and extract the G channels of the target image and the floating image respectively;

[0020] A registration module, configured to perform registration using a coarse-fine two-stage parametric method based on the extracted G channels of the target image and the floating image to obtain a final registered image;

[0021] Among them, in the coarse registration stage, by detecting the centroids of the optic disc and fovea, as well as the vascular bifurcation points, an affine transformation is quickly estimated using three pairs of feature points to obtain an initial deformation field; in the fine registration stage, a multi-scale iterative optimization strategy is adopted to construct an inverted pyramid model. Based on the inverted pyramid model, the gray-scale transformation between the initial deformation field and the target image is parametrically modeled at each scale, and the deformation field and gray-scale transformation are iteratively calculated. The complete parametric deformation field is inferred through parametric fitting;

[0022] The parametric deformation field obtained by fitting speculation is upsampled, and the original input floating image is deformed using the upsampled parametric deformation field to obtain the final registered image.

[0023] According to some embodiments, the present disclosure adopts the following technical solutions:

[0024] A computer program product includes a computer program, and when the computer program is executed by a processor, it implements the fundus image registration method based on two-stage parametric deformation modeling described above.

[0025] According to some embodiments, the present disclosure adopts the following technical solutions:

[0026] A non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the fundus image registration method based on two-stage parametric deformation modeling described above is implemented.

[0027] According to some embodiments, the present disclosure adopts the following technical solutions:

[0028] An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes and implements the fundus image registration method based on two-stage parametric deformation modeling described above.

[0029] Compared with the prior art, the beneficial effects of the present disclosure are:

[0030] The fundus image registration method based on two-stage parametric deformation modeling of the present disclosure proposes a coarse-fine two-stage registration method. In the first stage, by detecting the centroids of the optic disc and fovea, as well as the vascular bifurcation points, the affine transformation is quickly initialized using three pairs of feature points; in the second stage, multi-scale LAP iteration is performed, and the local all-pass filter (LAP) is combined with the polynomial displacement field model for iterative optimization. To address the problem of gray-scale differences between images, quadratic polynomial fitting is used to fit the illumination change. To address the balance problem between registration efficiency and accuracy during the iterative process, a method for adaptively adjusting the number of iterations is proposed, which can improve the reliable measure of alignment quality and has an index of low computational complexity.

[0031] The fundus image registration method based on two-stage parametric deformation modeling of the present disclosure proposes a parametric modeling method for adaptively adjusting the complexity of the deformation field in view of the fact that a fixed 3D eyeball model cannot flexibly cope with morphological changes and there are projection distortion problems. The experimental results show that on the FIRE and HRF data sets, the registration quality evaluation index SalC reaches more than 60%, the calculation time is shortened to the second level, and it is robust to images of different qualities, providing efficient and reliable technical support for applications such as clinical diabetic retinopathy analysis and image fusion.

[0032] In the ideal case where there is no gray-scale difference between the target image and the floating image, the fundus image registration method based on two-stage parametric deformation modeling of the present disclosure correlates the two through the gray-scale consistency equation, and introduces a gray-scale transformation function to describe the gray-scale transformation between the target image and the floating image, solving the problem that it is often difficult for the target image and the floating image to meet the gray-scale consistency in the actual application of fundus image registration due to the complexity of fundus diseases and the differences in shooting conditions.

[0033] The fundus image registration method based on two-stage parametric deformation modeling of the present disclosure solves the local misregistration problem through explicit constraints, eliminates the preprocessing and postprocessing steps, improves the algorithm efficiency, and makes it more suitable for the fundus image registration scenarios with gray-scale changes, pathological changes and large deformations. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The specification drawings forming a part of the present disclosure are used to provide a further understanding of the present disclosure. The schematic embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure.

[0035] Figure 1 It is the deformation field obtained by the LAP algorithm of the embodiment of the present disclosure;

[0036] Figure 2 It is the deformation field after fitting with a second-order polynomial in the embodiment of the present disclosure;

[0037] Figure 3 It is a comparison chart of the results of the method proposed in the present disclosure and other methods in the large-scale deformation subset of the FIRE data set in the embodiment of the present disclosure;

[0038] Among them, Figure 3 in (a) is the target image, Figure 3 in (b) is the floating image, Figure 3 in (c) is the result of fusing the target image and the floating image, where the target image is marked in orange and the floating image is marked in blue; Figure 3 in (d) is the result of fusing the registered image of the REMPE method and the target image;Figure 3 In (e) of [reference], it is the result of fusing the registered image and the target image by the Harris - PIIFD method; Figure 3 In (f) of [reference], it is the result of fusing the registered image and the target image by the SURF - PIIFD - RPM method; Figure 3 In (g) of [reference], it is the result of fusing the registered image and the target image by the method of the present disclosure;

[0039] Figure 4 It is the flow block diagram of the fundus image registration method based on two - stage parametric deformation modeling according to the embodiment of the present disclosure;

[0040] Figure 5 It is the method for adaptively adjusting the number of iterations according to the embodiment of the present disclosure;

[0041] Figure 6 It is the schematic diagram of the inverted pyramid model structure according to the embodiment of the present disclosure. Detailed implementation manners

[0042] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanations for the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0044] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] Embodiment 1

[0046] In one embodiment of the present disclosure, a fundus image registration method based on two - stage parametric deformation modeling is provided. The method steps include:

[0047] Step 1: Obtain the target image and the floating image, perform downsampling on them, and respectively extract the G channels of the target image and the floating image;

[0048] Step 2: Based on the G channels of the extracted target image and floating image, perform registration using a coarse-fine two-stage parameterization method to obtain the final registered image. Specifically, in the coarse registration stage, by detecting the centroids of the optic disc and fovea and the vascular bifurcation points, use three pairs of feature points to quickly estimate the affine transformation and obtain the initial deformation field. In the fine registration stage, adopt a multi-scale iterative optimization strategy to construct a Gaussian pyramid model. Based on the Gaussian pyramid model, parameterize the gray-level transformation between the initial deformation field and the target image at each scale, iteratively calculate the deformation field and the gray-level transformation, and parameterize the fitting to infer the complete parameterized deformation field.

[0049] Perform upsampling processing on the parameterized deformation field inferred by fitting, and use the upsampled parameterized deformation field to deform the original input floating image to obtain the final registered image.

[0050] As an embodiment, for a fundus image registration method based on two-stage parameterized deformation modeling in the present disclosure, due to differences in shooting perspectives, there are significant deformations between many fundus images, and the overlapping areas are small, which significantly increases the difficulty of registration. To achieve fast deformation field estimation with pixel-level accuracy in the case of large-scale deformations, the present disclosure proposes a coarse registration - fine registration two-stage image registration method, as Figure 4 shown. The specific implementation process is as follows:

[0051] Step 1: Obtain the target image and the floating image, perform downsampling on them, and respectively extract the G channels of the target image and the floating image.

[0052] Specifically, with the development of fundus imaging technology, the image size usually exceeds 2000×2000 pixels, but in some cases, the image contrast is low and there is a lot of noise. To reduce the computational cost, the present disclosure downsamples the image to a size within 1000×1000 pixels.

[0053] Since the G channel of the fundus image has the strongest contrast, it can clearly show the blood vessel distribution and background differences, and at the same time avoids the problems of the R channel being too bright and the B channel having low brightness and high noise. Therefore, in the present disclosure, the G channels of the target image and the floating image are respectively extracted. Among them, the G channel is one of the three channels of the RGB color image.

[0054] Step 2: Based on the G channels of the extracted target image and floating image, perform registration using a coarse-fine two-stage parameterization method to obtain the final registered image.

[0055] Specifically, in the coarse registration stage, by detecting the centroids of the optic disc and fovea and the vascular bifurcation points, use three pairs of feature points to quickly estimate the affine transformation and obtain the initial deformation field.

[0056] In the fine registration stage, a multi-scale iterative optimization strategy is adopted to construct an inverted pyramid model, and the gray-scale transformation between the deformation field and the images to be registered (floating image and target image) is parametrically modeled at each scale, and the deformation field and the gray-scale transformation are calculated iteratively. Figure 4 The process of a single (the j -th) iteration is described within the blue dashed box. Different from the current mainstream methods based on the image resolution from coarse to fine, the present disclosure adopts a strategy of not changing the image resolution. As Figure 6 shown, by using the inverted pyramid model and dynamically adjusting the window size of the all-pass filter in the LAP algorithm, the step-by-step optimization from a large window to a small window is realized, thus completing the progressive process of the spatial deformation from a rough estimate to a fine estimate. It can quickly capture the approximate alignment, gradually refine to the high-resolution layer, gradually correct the small displacements, and improve the registration accuracy and efficiency.

[0057] First, in the coarse registration stage, by detecting the centroids of the optic disc and the fovea and the vascular bifurcation points, the affine transformation is quickly estimated using three pairs of feature points to obtain the initial deformation field;

[0058] Specifically, in the coarse registration stage, the centroids of the optic disc and the fovea are detected by a band-pass filter, the vascular bifurcation points are obtained by SIFT feature extraction, and the features are screened by the RANSAC method. Finally, one pair is randomly selected from the centroids of the optic disc and the fovea, and two pairs of points are randomly selected from the vascular bifurcation points as the final feature points to estimate the affine transformation.

[0059] In the fine registration stage, based on the LAP (Local All-Pass) algorithm, an explicit constraint is imposed on the deformation field. A linear parametric model is used to represent the deformation field, and the complexity of the deformation field is adaptively adjusted according to the image characteristics. The LAP algorithm regards the image displacement in the spatial domain as a phase change in the frequency domain through an all-pass filter, assumes that the local deformation field changes slowly, approximates it as a constant displacement, and efficiently estimates the dense elastic deformation field through a linear system. The LAP algorithm has the advantages of high speed and high accuracy when satisfying the gray-scale consistency assumption. However, its limitations are that it is sensitive to gray-scale differences, relies on preprocessing and postprocessing steps, and performs poorly in the case of large displacements, which may lead to too high complexity of the deformation field and topological structure changes. To address these problems, the present disclosure solves the local misregistration problem through explicit constraints, eliminates the preprocessing and postprocessing steps, improves the algorithm efficiency, and makes it more suitable for the fundus image registration scenarios with gray-scale changes, pathological changes, and large deformations.

[0060] Furthermore, the geometric deformation between images can be expressed as a linear combination of a series of basis functions, as follows:

[0061] (1)

[0062] Among them, are known basis functions, is the number of basis functions, are the coefficients of the basis functions. Therefore, calculating the two-dimensional deformation field is equivalent to solving coefficients . The type and number of basis functions determine the complexity of the deformation field. Taking the spatial transformation between two fundus images represented by polynomial functions as an example, as the polynomial order increases, the number of model parameters increases accordingly, and the deformation field that can be characterized is also more complex.

[0063] Specifically, the constant term (0th-order polynomial) can describe the translational transformation between images, the first-order polynomial can characterize the affine transformation, the second-order polynomial can approximately simulate the projective transformation, and higher-order polynomials can characterize more complex deformations. However, in the fundus image registration task, it is not that the higher the model order (i.e., the more parameters) the better. An overly high order will not only significantly increase the computational amount, which may lead to overly fine local registration, thus changing the topological structure characteristics of the fundus image, such as the curvature of blood vessels, the deformation of anatomical structures such as the optic disc and macula, but may also cause excessive distortion in non-overlapping regions, changing their anatomical structures, and thus having a significant impact on clinical diagnosis.

[0064] This disclosure adopts a polynomial model, but the proposed framework has wide applicability and supports various "elastic" models based on global representations (such as higher-order polynomials, Fourier series) or local representations (such as B-spline bases, radial basis functions), thereby providing flexible solutions for different application scenarios.

[0065] These coefficients are determined by minimizing the difference between the parameterized model and the dense deformation field obtained by the LAP algorithm in the region :

[0066] (2)

[0067] Accurate deformation field estimation depends on a reliable fitting region , and an accurate fitting region in turn depends on an accurate deformation field. To solve this interdependent problem, this disclosure proposes to dynamically update the fitting region during the iteration process. In the initial iteration, the estimated fitting region usually deviates from the actual overlapping region, but as the number of iterations increases, these deviations will gradually decrease.

[0068] This disclosure only needs to utilize the accurate deformation of part of the region and infer the complete parameterized deformation field through parametric fitting. This fitting strategy has the same effect as the deformation field repair and mean filtering in the LAP algorithm.

[0069] Traditional iterative registration methods adopt a strategy of fixed iteration times, which easily leads to insufficient registration accuracy or waste of computing resources. To solve this problem, the present disclosure proposes a method for adaptively adjusting the iteration times. The core lies in finding an index that can not only reliably measure the alignment quality but also has low computational complexity. As Figure 5 shown, starting from the stable blood vessel structure in fundus images, the present disclosure extracts the reverse green channel, innovatively uses the Fourier-Argand filter and Gaussian filter to filter the reverse green channel image, and extracts the response value (filtering difference) of the blood vessel area as the significant feature for calculating SalC, thereby improving the robustness of feature expression. In the coarse-to-fine registration framework, in addition to the preset maximum iteration times at each scale, the iteration times are also dynamically controlled by SalC: when SalC starts to decline, stop the iteration and return the previous result, so as to balance accuracy and efficiency.

[0070] To ensure the real-time performance of fundus image registration, the present disclosure uses a low-complexity fast shifted linear interpolation method to interpolate the floating image during each iteration. This method can not only ensure the computational efficiency but also effectively maintain the registration accuracy.

[0071] In the ideal case where there is no gray-scale difference between the target image and the floating image , the two are related by the gray-scale consistency equation

[0072] (3)

[0073] where is the two-dimensional deformation field. However, in the actual application of fundus image registration, due to the complexity of fundus diseases and the differences in shooting conditions, it is often difficult for the target image and the floating image to meet the gray-scale consistency assumption. To solve this problem, the present disclosure introduces the gray-scale transformation relation function to describe the gray-scale transformation between the target image and the floating image :

[0074] (4)

[0075] This function may cover combinations of various complex factors such as convolution, non-uniform illumination changes, and non-linear pixel-level transformations. Therefore, in order to accurately solve the deformation field , it is necessary to simultaneously estimate the gray-scale mapping function , which is the key step to achieve high-precision fundus image registration.

[0076] Similar to the parametric model of the deformation field, the gray-level relationship between images can be represented by a linear combination of a series of basis functions:

[0077] (5)

[0078] The basis functions are represented by By minimizing the least-squares difference between the gray levels of the target image and the floating image within the field of view, the coefficients can be calculated, thereby estimating the gray-level relationship between the images.

[0079] For global gray-level changes, the following parametric model can be used:

[0080] (6)

[0081] where represents the gray-level relationship between the target image and the floating image. It has been verified in previous work that global illumination changes can be effectively represented by a second-order polynomial. For locally non-uniform gray-level changes, a linear combination of basis functions such as higher-order polynomial functions, uniform B-spline bases, radial basis functions, or Fourier series can be used to represent them. The selection of these basis functions can more flexibly adapt to the complexity of local gray-level changes, thereby improving the accuracy and robustness of gray-level relationship modeling.

[0082] Aiming at the non-linear gray-level transformation between the to-be-registered images caused by fundus diseases and artifacts, the present disclosure proposes a non-linear gray-level transformation model based on polynomial expansion, constructs the non-linear gray-level transformation between the target image and the floating image , and combines local filtering to reduce the influence of local gray-level differences. The specific model is as follows:

[0083] (7)

[0084] where are the model coefficients, and is the model order. By introducing the polynomial expansion form, this model can better describe the non-linear gray-level transformation between images, thereby effectively alleviating the influence of local gray-level differences on the registration accuracy.

[0085] After completing the deformation field in the two-stage coarse registration - fine registration , the deformation field is upsampled, and then the original input floating image is deformed using the upsampled deformation field to obtain the final registered image.

[0086] Simulation experiments

[0087] Figure 1 ,Figure 2 The comparison shows the dense elastic deformation field estimated by the LAP algorithm and the optimized deformation field after second-order polynomial fitting. Among them, Figure 1 and Figure 2 the different colors in the compass diagrams in the upper right corners of indicate different directions of the displacement field, and the arrows also mark the directions. It can be clearly seen that the polynomial fitting strategy for the dense elastic deformation field effectively replaces the post-processing step of the deformation field in LAP and can correct the local misregistration problem of the LAP algorithm. Since no additional regularization term is introduced in the objective function (Equation 2), the solution process is simplified to a calculation problem of a system of linear equations, which not only greatly reduces the computational complexity but also ensures the uniqueness and optimality of the solution results.

[0088] Figure 3 The comparison of the results of the method proposed in this disclosure with other methods in the large-scale deformation subset of the FIRE dataset is shown. In the figure, the registered image and the target image are fused and displayed, and are marked with orange and blue respectively. In addition, the control points are marked in the figure. The closer the control points are, the higher the registration accuracy. As shown in Table 1, the comparison data of the registration results of different methods in the FIRE dataset.

[0089] Table 1 Comparison data of the registration results of different methods in the FIRE dataset

[0090]

[0091] Example 2

[0092] In one embodiment of the present disclosure, a fundus image registration system based on two-stage parametric deformation modeling is provided, including:

[0093] An image acquisition module, configured to acquire a target image and a floating image, downsample them, and respectively extract the G channels of the target image and the floating image;

[0094] A registration module, configured to perform registration on the basis of the extracted G channels of the target image and the floating image by using a coarse-fine two-stage parametric method to obtain a final registered image;

[0095] Among them, in the coarse registration stage, by detecting the centroids of the optic disc and the fovea and the vascular bifurcation points, a pair of three feature points are used to quickly estimate the affine transformation to obtain an initial deformation field; in the fine registration stage, a multi-scale iterative optimization strategy is adopted to construct an inverted pyramid model, and the gray-scale transformation between the initial deformation field and the target image is parametrically modeled at each scale based on the inverted pyramid model, and the deformation field and the gray-scale transformation are iteratively calculated, and the complete parametric deformation field is inferred by parametric fitting;

[0096] Upsample the parameterized deformation field obtained by fitting and speculation, and use the upsampled parameterized deformation field to deform the floating image of the original input to obtain the final registered image.

[0097] Example 3

[0098] In one embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the fundus image registration method based on two-stage parameterized deformation modeling is implemented.

[0099] Example 4

[0100] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, and the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the fundus image registration method based on two-stage parameterized deformation modeling is implemented.

[0101] Example 5

[0102] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes to implement the fundus image registration method based on two-stage parameterized deformation modeling.

[0103] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer-implemented processing, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0105] Although the specific implementation manners of the present disclosure have been described above in conjunction with the accompanying drawings, they do not limit the protection scope of the present disclosure. Those skilled in the art should understand that, based on the technical solutions of the present disclosure, various modifications or deformations that can be made by those skilled in the art without creative efforts still fall within the protection scope of the present disclosure.

Claims

1. A fundus image registration method based on two-stage parametric deformation modeling, characterized in that: include: Obtain the target image and the floating image, downsample them, and extract the G channels of the target image and the floating image respectively; Based on the extracted target image and the G channel of the floating image, a coarse-fine two-stage parameterization method is used for registration to obtain the final registered image. In the coarse registration stage, the centroid of the optic disc and fovea and the vascular bifurcation point are detected, and the affine transformation is quickly estimated using three pairs of feature points to obtain the initial deformation field. In the fine registration stage, a multi-scale iterative optimization strategy is used to construct an inverted pyramid model. Based on the inverted pyramid model, the grayscale transformation between the initial deformation field and the target image is parameterized at each scale. The deformation field and grayscale transformation are iteratively calculated, and the complete parameterized deformation field is inferred by parametric fitting. The fitted and inferred parameterized deformation field is upsampled, and the original input floating image is deformed using the upsampled parameterized deformation field to obtain the final registered image.

2. The fundus image registration method based on two-stage parametric deformation modeling as claimed in claim 1, characterized in that: The target image and floating image are acquired and downsampled to a size of 1000×1000 pixels or less. The G channel of the fundus image has the strongest contrast and can clearly show the vascular distribution and background differences, while avoiding excessive brightness of the R channel and low brightness and high noise of the B channel. The G channels of the target image and floating image are extracted respectively.

3. The fundus image registration method based on two-stage parametric deformation modeling as claimed in claim 1, characterized in that: The coarse-fine two-stage parameterization method explicitly constrains the deformation field in both stages, constructs an objective function without additional constraints, transforms the image registration problem into a linear optimization problem, obtains a closed-form solution, and adopts a strategy that does not change the image resolution, that is, by dynamically adjusting the window size of the all-pass filter in the multi-scale iterative optimization algorithm, a gradual optimization from large windows to small windows is achieved, thereby completing the progressive process from coarse estimation to fine estimation of spatial deformation.

4. The fundus image registration method based on two-stage parametric deformation modeling as claimed in claim 1, characterized in that: In the fine registration stage, based on the multi-scale iterative optimization algorithm, explicit constraints are imposed on the deformation field, a linear parameterized model is used to represent the deformation field, and the complexity of the deformation field is adaptively adjusted according to the image characteristics. The multi-scale iterative optimization algorithm uses an all-pass filter to regard the image displacement in the spatial domain as a phase change in the frequency domain. It is assumed that the local deformation field changes slowly and is approximately a constant displacement, and the dense elastic deformation field is efficiently estimated through a linear system.

5. The fundus image registration method based on two-stage parametric deformation modeling as claimed in claim 1, characterized in that: In the ideal case where there is no grayscale difference between the target image and the floating image, the target image and the floating image are related by the grayscale consistency equation, and a grayscale transformation function is introduced to describe the target image. With floating images Grayscale transformation between: Among them, the function It involves a combination of complex factors such as convolution, non-uniform illumination changes, or non-linear pixel-level transformations. is a two-dimensional deformation field.

6. The fundus image registration method based on two-stage parametric deformation modeling as claimed in claim 5, characterized in that: Aiming at the nonlinear grayscale transformation between target images caused by fundus diseases and artifacts, a nonlinear grayscale mapping model based on polynomial expansion is constructed. With floating images The nonlinear grayscale mapping relationship between them is combined with local filtering to reduce the impact of local grayscale differences. The specific model is as follows: in, is the model coefficient, is the model order.

7. A fundus image registration system based on two-stage parametric deformation modeling, characterized in that: include: An image acquisition module is used to acquire a target image and a floating image, and downsample them to extract the G channels of the target image and the floating image respectively; A registration module is used to perform registration based on the extracted target image and the G channel of the floating image using a coarse-fine two-stage parameterization method to obtain the final registered image; In the coarse registration stage, the centroid of the optic disc and fovea and the vascular bifurcation point are detected, and the affine transformation is quickly estimated using three pairs of feature points to obtain the initial deformation field. In the fine registration stage, a multi-scale iterative optimization strategy is used to construct an inverted pyramid model. Based on the inverted pyramid model, the grayscale transformation relationship between the initial deformation field and the target image is parameterized at each scale. The deformation field and grayscale transformation are iteratively calculated, and the complete parameterized deformation field is inferred by parametric fitting. The fitted and inferred parameterized deformation field is upsampled, and the original input floating image is deformed using the upsampled parameterized deformation field to obtain the final registered image.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the fundus image registration method based on two-stage parameterized deformation modeling described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, the fundus image registration method based on two-stage parametric deformation modeling as described in any one of claims 1-6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the fundus image registration method based on two-stage parametric deformation modeling as described in any one of claims 1-6.

Images