A fundus image registration method based on multi-disc mask
The information around the blood vessel bifurcation point of the fundus image was extracted through a multi-disk mask, and the coarse registration and precise registration were carried out, which solved the accuracy and real-time problems of fundus image registration in the prior art, and achieved efficient image registration effect.
Patent Information
- Application Number
- CN202210230937.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-09
- Publication Date
- 2025-08-15
- Estimated Expiration
- 2042-03-09
AI Technical Summary
The prior art is difficult to take into account both accuracy and real-time when registering fundus images, and is easily affected by the difference in image rotation angles, so it is impossible to achieve accurate registration.
Multi-disc mask is used to extract the intersection information around the bifurcation point of the blood vessel to form feature vectors, including distance and angle features. Image registration is performed through the steps of coarse registration and precise registration, and image rotation calibration is performed using the characteristics of rotation invariance.
It improves the efficiency and accuracy of image registration, reduces the amount of calculation, and meets the requirements of modern medicine for real-time.
Smart Images
Figure CN114565654B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image analysis, and in particular to a fundus image registration method based on a multi-disc mask. Background Art
[0002] In the prior art, fundus image registration requires first obtaining feature points in the image. Typically, the locations of the points are obtained. Subsequent registration using only the locations will result in significant distortion. Therefore, a feature description operator is needed to describe the information surrounding the feature points, expanding the information about the registration points. This generates a corresponding feature vector. The most common description of registration point information in fundus image processing is the feature vector described by the FREAK algorithm, which has a total of 512 dimensions. Subsequent calculation of the image transformation matrix using this vector for image stitching is time-consuming, especially for high-pixel images, which does not meet the real-time requirements of modern medicine. Furthermore, due to the difference in rotation angles between the registered fundus images during the filming process, even if information surrounding the feature points is included, accurate registration may still be impossible due to the difference in rotation angles between the two images. Therefore, the technical challenge addressed by this solution is how to achieve accurate fundus image registration with minimal computational effort, overcome image rotation angle differences, and meet the real-time requirements of modern medicine. Summary of the Invention
[0003] The present invention provides a fundus image registration method based on a multi-disc mask, which is used to solve the technical problems that the existing technology cannot take into account both accuracy and real-time performance, and is easily affected by differences in image rotation angles and thus the registration accuracy is affected.
[0004] The fundus image registration method based on a multi-disc mask comprises the following steps:
[0005] Step 1: Complete the segmentation and recognition of the vascular tree in the fundus image and obtain the location of the vascular bifurcation point;
[0006] Step 2: Using a multi-disc mask structure, extract information surrounding the vascular bifurcation point to form a corresponding feature vector. The surrounding information includes intersection information generated by the intersection of the disc mask and the vascular tree surrounding the vascular bifurcation point. The intersection information includes a distance feature and an angle feature. The distance feature is the distance between the intersection point and the corresponding vascular bifurcation point, and the angle feature is the clockwise angle of the intersection point to the horizontal coordinate in a coordinate system with the vascular bifurcation point as the origin.
[0007] Step 3: perform coarse feature point registration based on the blood vessel length in the feature vector, and compare the distance features of the feature vector to determine whether the coarse registration of the two fundus images is successful;
[0008] Step 4: Based on the results of the coarse registration, the coarsely registered fundus image is rotated around the corresponding vascular bifurcation point, and the angle features in the feature vector are updated to maintain the image rotation invariance. The updated angle features are then compared to complete the precise registration of the fundus image.
[0009] Preferably, in step 2, this article uses a structure composed of multiple disk masks to extract feature information. The radius of each disk mask increases from small to large, simulating the principle of the human eye retina receiving image information. A two-dimensional coordinate system is established with the feature point as the origin. The center of each disk mask coincides with the feature point, and the collected intersection information is rotationally invariant.
[0010] Preferably, in step 2, if there is an intersection between the disk mask and the vascular tree around the vascular bifurcation point in each quadrant of the two-dimensional coordinate system, the intersection information is collected; if there are two or more intersections between a disk mask and the vascular tree in a quadrant, only the intersection with the longest length is collected.
[0011] Preferably, in step 4, the rotation angle θ of the image to be registered is calculated based on the intersection information of the successful rough matching:
[0012]
[0013] Among them, θ an Indicates the angle characteristic value of each intersection point in the figure obtained by completing the intersection information collection of a fundus image to be registered in step 2, θ bn It represents the angle feature value of each intersection in the figure obtained by completing the intersection information collection of another fundus image to be registered in step 2; vector calibration is performed according to the rotation angle θ of the image to be registered to update the angle feature.
[0014] Preferably, in step 2, there are 4 disk masks from small to large, and the two-dimensional coordinate system has four quadrants. Each intersection information includes two eigenvalues: distance feature and angle feature. Therefore, the feature descriptor corresponding to the feature point has 4×4×2=32 dimensions, and the expression of the corresponding feature vector is as follows:
[0015] V=[lengths angles]
[0016] lengths=[l1,l2,l3,...,l 16 ]
[0017] angles=[θ1,θ2,θ3,...,θ 16 ]
[0018] Among them, V represents the feature vector, lengths represents the distance feature, angles represents the angle feature, l i(i=1,2.3…16) represents the specific distance characteristic value of each intersection point, θ i (i=1, 2.3…16) represents the specific angle characteristic value of each intersection point.
[0019] Preferably, in step 1, a semantic segmentation method based on deep learning technology is used to segment and identify blood vessels in the fundus image, and the center lines of the blood vessels are extracted. The broken blood vessels are connected using mathematical morphology to obtain a final vascular tree that meets the requirements; then, the vascular bifurcation points in the vascular tree are extracted as feature points, the feature point positions are accurately calculated at the sub-pixel level using the Zernike moment, and the eight-neighborhood lookup table method is used to perform rough bifurcation point detection.
[0020] The present invention has the following advantages: Because this method uses a circular disc mask for point sampling, the intersection points between the disc mask and the vascular tree are rotationally invariant. One characteristic of rotational invariance is that changes in the angular characteristics do not affect the distance characteristics of the intersection points. Therefore, the intersection points in two fundus images can be roughly matched based on the distance characteristics. Another advantage is that even if the images are rotated, when sampling points using a circular disc mask centered on the feature point, precise matching can be achieved by simply rotating the image around the feature point. This method is essentially unaffected by differences in the rotation angle of the fundus images to be registered.
[0021] At the same time, when this method performs feature description on the surrounding information of the vascular bifurcation point, the final feature descriptor obtained has 4×4×2=32 dimensions. Compared with the 512-dimensional data of the FREAK descriptor commonly used in the existing technology, the amount of data generated is greatly reduced, the registration efficiency is effectively improved, and it has great advantages in time calculation, meeting the real-time requirements of modern medicine for fundus image processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 This is a flow chart of a fundus image registration method based on a multi-disc mask according to the present invention;
[0023] Figure 2 A diagram of a vascular tree obtained by segmenting and identifying the fundus image in step 1 of the present invention;
[0024] Figure 3 The present invention obtains a diagram of a blood vessel bifurcation point for step 1;
[0025] Figure 4 The topological structure diagram of the multi-disc mask of a blood vessel bifurcation point in step 2 of the present invention;
[0026] FIG5( a ) is a diagram obtained by completing intersection information collection for a fundus image to be registered in step 2 of the present invention;
[0027] FIG5( b ) is a diagram obtained by completing the intersection information collection of another fundus image to be registered in step 2 according to the present invention. DETAILED DESCRIPTION
[0028] The following is a further detailed description of the specific implementation methods of the present invention through the description of the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.
[0029] like Figure 1 As shown in FIG5 , the present invention provides a fundus image registration method based on a multi-disc mask, comprising the following steps:
[0030] Step 1: Complete the segmentation and identification of the vascular tree in the fundus image and obtain the location of the vascular bifurcation point.
[0031] This method can use the semantic segmentation method based on deep learning technology to segment and identify blood vessels in fundus images, and can further process the segmented blood vessel information, such as extracting the center line of the blood vessels and connecting the broken blood vessels using mathematical morphology, so as to obtain the final blood vessel tree that meets the requirements. The blood vessel bifurcation points in the blood vessel tree are then extracted as feature points, and the sub-pixel level accuracy of the feature point positions is calculated using the Zernike moment (such as Figure 2 、 Figure 3 As shown in the figure, the centerlines of the same part of the image may differ slightly from one image to another. To avoid extraction errors, an eight-neighborhood lookup table method is used for rough bifurcation point detection. Compared to the original pixel-level method, the feature point position calculated using Zernike moments is more accurate, which is more conducive to improving the accuracy of image registration.
[0032] Step 2: Use the structure composed of multiple disk masks to extract the surrounding information of the vascular bifurcation point to form a corresponding feature vector. The surrounding information includes the intersection information generated by the intersection of the disk mask and the vascular tree around the vascular bifurcation point.
[0033] Vascular feature extraction can determine the location of vascular bifurcations, but using only positional features during image registration can lead to inaccurate registration. Information surrounding the feature points must also be extracted. Using the FREAK descriptor, commonly used in fundus image processing, to describe the surrounding information generates a large amount of data, resulting in a time-consuming calculation of the image transformation matrix for image stitching, which does not meet the real-time image processing requirements of modern medicine.
[0034] This paper uses a structure composed of multiple disk masks to extract feature information. The topological structure is as follows Figure 4As shown. Four disk masks are used, and the radius of each disk mask is 3, 5, 7, and 9 respectively, from coarse to fine, to simulate the principle of the human eye retina receiving image information. A two-dimensional coordinate system is established with the feature point as the origin, and the center of each disk mask coincides with the feature point. The two-dimensional coordinate system has four quadrants. In each quadrant, if there is an intersection between the disk mask and the vascular tree around the vascular bifurcation point, the intersection information is collected. The intersection information includes a distance feature and an angle feature. The distance feature is the distance between the intersection and the corresponding vascular bifurcation point, and the angle feature is the clockwise angle of the intersection to the horizontal coordinate in the coordinate system with the vascular bifurcation point as the origin (that is, the clockwise angle of the line connecting the intersection to the origin of the coordinate system to the horizontal coordinate). The surrounding information of the vascular bifurcation point is described in the above manner. Since multiple concentric disk masks are used, the collected intersection information is rotationally invariant.
[0035] During the acquisition process, if a disk mask in a quadrant intersects with the vascular tree at two or more points, only the longest intersection is collected. This allows for a maximum of four intersections per quadrant. If there are insufficient intersections, the missing intersections are assigned a value of (0,0). The resulting feature descriptor has 4 × 4 × 2 = 32 dimensions, significantly reducing the amount of data generated compared to the 512-dimensional data of the FREAK descriptor, significantly reducing computational time. The corresponding feature vector is expressed as follows:
[0036] V=[lengths angles]
[0037] lengths=[l1,l2,l3,...,l 16 ]
[0038] angles=[θ1,θ2,θ3,...,θ 16 ]
[0039] Among them, V represents the feature vector, lengths represents the distance feature, angles represents the angle feature, l i (i=1,2.3…16) represents the specific distance characteristic value of each intersection point, θ i (i=1, 2.3…16) represents the specific angle characteristic value of each intersection point.
[0040] Step 3: Perform coarse feature point registration based on the blood vessel length in the feature vector, and compare the distance features between each intersection point and the corresponding blood vessel bifurcation point to determine whether the two fundus images are coarsely registered successfully.
[0041] Since the images have a certain rotation angle during the shooting process, as shown in Figure 5(a) and Figure 5(b), Figure 5(a) and Figure 5(b) are the same bifurcation point in the two images to be registered, but Figure 5(b) has been rotated relative to Figure 5(a). Directly calculating the eigenvalue at the intersection will obtain the information V of the intersection a in Figure 5(a). a = [l1, θ1], the information V of the intersection b in Figure 5(b) b =[l2,θ2], the two intersection points will be considered unmatched during feature point matching, resulting in errors. To avoid errors caused by rotation, a coarse matching is performed before feature point matching, using only the distance features of the feature points for comparison. If the distance features of point a and point b are the same, the coarse registration of the two points is considered successful.
[0042] Step 4: Based on the results of the coarse registration, the coarsely registered fundus image is rotated around the feature point (i.e., the corresponding vascular bifurcation point), and the angular features in the feature vector are updated to maintain the image rotation invariance. The updated angular features are then compared to complete the precise registration of the fundus image.
[0043] Calculate the rotation angle θ of the image to be registered based on the intersection information of the successful rough matching:
[0044]
[0045] Among them, θ an Indicates the angle characteristic value of each intersection point in the figure obtained by completing the intersection information collection of a fundus image to be registered in step 2, θ bn The angular feature values of each intersection in the image are obtained by collecting intersection information from the other fundus image to be registered in step 2. Vector calibration is performed based on the rotation angle θ of the image to be registered, and the angular features are updated. The feature vectors of the two fundus images are accurately compared based on the updated angular features. Registration is successful if these values are met.
[0046] In this approach, since the disc mask used for point sampling exhibits rotational invariance, the intersections between it and the vascular tree are rotationally invariant. One characteristic of rotational invariance is that changes in the angular characteristics do not affect the distance characteristics of the intersections. Therefore, the intersections in the two fundus images can be roughly matched based on the distance characteristics. Another characteristic is that even if the images are rotated, when sampling points using the disc mask centered on the feature point, precise matching can be achieved by simply rotating the image around the feature point. The specific rotation angle is the same as the rotation angle θ of the image to be registered described in the previous step.
[0047] Finally, experiments verified the improved registration efficiency of this method. Taking any five pairs of images, we used the FREAK algorithm and our proposed algorithm for feature description, and finally performed image registration. The time comparison is shown in Table 1. Compared to the 512-dimensional FREAK descriptor, this method has a significant advantage in computational time.
[0048] Table 1 Registration time of FREAK algorithm and the method in this paper
[0049]
[0050] Here, the improvement ratio = (FREAK algorithm time - this algorithm time) / FREAK algorithm time * 100%.
[0051] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.
Claims
1. A fundus image registration method based on a multi-disc mask, characterized by: The following steps are involved: Step 1: Complete the segmentation and recognition of the vascular tree in the fundus image and obtain the location of the vascular bifurcation point; Step 2: Using a multi-disc mask structure, extract information surrounding the vascular bifurcation point to form a corresponding feature vector. The surrounding information includes intersection information generated by the intersection of the disc mask and the vascular tree surrounding the vascular bifurcation point. The intersection information includes a distance feature and an angle feature. The distance feature is the distance between the intersection point and the corresponding vascular bifurcation point, and the angle feature is the clockwise angle of the intersection point to the horizontal coordinate in a coordinate system with the vascular bifurcation point as the origin. Step 3: Perform coarse registration based on the distance features in the feature vectors, and compare the distance features of the feature vectors to determine whether the coarse registration of the two fundus images is successful; Step 4: Based on the results of the coarse registration, the coarsely registered fundus image is rotated around the corresponding vascular bifurcation point, and the angle features in the feature vector are updated to maintain the image rotation invariance. The updated angle features are then compared to complete the precise registration of the fundus image.
2. The fundus image registration method based on a multi-disc mask according to claim 1, characterized in that: In step 2, feature information is extracted using a structure composed of multiple disk masks, and vascular bifurcation points in the vascular tree are extracted as feature points. The radii of each disk mask are different, simulating the principle of the human eye retina receiving image information. A two-dimensional coordinate system is established with the feature point as the origin, and the center of each disk mask coincides with the feature point. The collected intersection information is rotationally invariant.
3. The fundus image registration method based on a multi-disc mask according to claim 2, characterized in that: In step 2, if there is an intersection point between the disk mask and the vascular tree around the vascular bifurcation point in each quadrant of the two-dimensional coordinate system, the intersection point information is collected. If a disk mask in a quadrant has two or more intersection points with the vascular tree, only the intersection point with the longest distance characteristic value is collected.
4. The fundus image registration method based on a multi-disc mask according to any one of claims 1 to 3, characterized in that: In step 4, the rotation angle θ of the fundus image to be registered is calculated based on the intersection information of the successfully coarsely registered image: Among them, θ an Indicates the angle characteristic value of each intersection point in the figure obtained by completing the intersection information collection of a fundus image to be registered in step 2, θ bn It represents the angle feature value of each intersection in the figure obtained by completing the intersection information collection of another fundus image to be registered in step 2; vector calibration is performed according to the rotation angle θ of the fundus image to be registered to update the angle feature.
5. The fundus image registration method based on a multi-disc mask according to claim 3, characterized in that: In step 2, there are four disk masks with different radii. Concentrically arranged multi-disk masks are used to extract feature information. The two-dimensional coordinate system has four quadrants. Each intersection information includes two eigenvalues: distance feature and angle feature. Therefore, the feature descriptor corresponding to the feature point has 4×4×2=32 dimensions. The expression of the corresponding feature vector is as follows: V=[lengths angles] lengths=[l1,l2,l3,...,l 16 ] angles=[θ1,θ2,θ3,...,θ 16 ] Among them, V represents the feature vector, lengths represents the distance feature, angles represents the angle feature, l i Represents the specific distance characteristic value of each intersection, θ i Indicates the specific angle characteristic value of each intersection point, i = 1, 2.3…16.
6. The fundus image registration method based on a multi-disc mask according to claim 1, characterized in that: In step 1, a semantic segmentation method based on deep learning technology is used to segment and identify blood vessels in the fundus image, and the center lines of the blood vessels are extracted. The broken blood vessels are connected using mathematical morphology to obtain a vascular tree. The vascular bifurcation points in the vascular tree are then extracted as feature points, and the positions of the feature points are accurately calculated at the sub-pixel level using the Zernike moment. The eight-neighborhood lookup table method is used to perform rough bifurcation point detection.