Calculation method of retinal blood vessel diameter in fundus imaging based on region segmentation

Abstract

This invention relates to a method for calculating retinal vessel diameters in fundus images based on region segmentation. The method includes: acquiring a fundus image to be processed; locating the center point of the fundus image to obtain a fundus image with a center point marker; performing region segmentation on the marked fundus image based on an automatic segmentation model to obtain segmented vessel regions; extracting the initial centerline of each vessel region to obtain a fundus image with vessel centerlines; performing secondary segmentation on the fundus image with vessel centerlines to obtain a fundus image with a tree structure and single-branch vessels; and then performing interpolation and smoothing operations pixel by pixel to obtain the average, maximum, and minimum vessel diameters of each vessel region. This invention aims to divide the vessels into different regions based on an automatic segmentation model and simultaneously calculate various vessel parameters within each region, exploring its segmentation capabilities and potential application value.

Description

Technical Field

[0001] This invention relates to the field of retinal fundus image analysis, and more specifically, to a method, device, system, computer-readable storage medium, and application of calculating the diameter of retinal vessels in fundus images based on region segmentation. Background Technology

[0002] In fundus image analysis, accurate localization of the macular center is crucial for computer-aided diagnostic systems of diabetic retinopathy. Furthermore, changes in retinal vascular morphology have significant clinical implications for cardiovascular and systemic diseases. For example, increased venule diameter is associated with the risk of diabetic retinopathy and stroke; hypertensive retinopathy can lead to arterial stenosis and a decreased arterial-venous ratio; and atherosclerosis can cause arterial stenosis. Given that different vascular regions have varying predictive value for different diseases, clinicians currently rely entirely on experience to assess changes in arteriovenous morphology in different regions. Existing technologies only calculate the average vessel diameter and cannot automatically segment different regions and simultaneously calculate vascular parameters within each region.

[0003] In recent years, due to the successful application of deep learning in vision, a large amount of work has been dedicated to developing image segmentation methods using deep learning models, achieving good results. Therefore, this study utilizes deep learning methods to automatically locate the optic disc center point and the fovea of ​​the macula and automatically segment each region. Simultaneously, it combines computer vision technology to calculate the diameter parameters of various blood vessels in each region, exploring its significant clinical implications and potential application value in the predictive analysis of cardiovascular and systemic diseases. Summary of the Invention

[0004] The method of this invention is based on deep learning algorithms to automatically locate the center point of the optic disc and the fovea of ​​the macula and segment each region. At the same time, it extracts the tree structure and branch vessels of the fundus and then automatically calculates the diameter parameters of each branch vessel in each region through normal vectors, interpolation operations, etc., to explore its computing power and potential application value.

[0005] This application discloses a method for calculating the diameter of retinal vessels in fundus photography based on region segmentation, including:

[0006] Obtain fundus images to be processed;

[0007] Based on the automatic positioning model, the center point of the optic cup and optic disc and the fovea of ​​the macula are located in the fundus image to be processed, and fundus images with center point markings are obtained.

[0008] Based on the automatic segmentation model, the fundus image marked with the center point is segmented to obtain the segmented vascular regions.

[0009] The initial centerline of the blood vessels in each segmented vascular region is extracted to obtain a fundus image with the centerline of the blood vessels.

[0010] The fundus image with the vascular centerline is segmented twice to obtain a fundus image with a tree structure and single-branched vessels;

[0011] Interpolation calculations were performed on the fundus images with the tree-like structure and single-branched vessels, and the average, maximum, and minimum diameters of each vascular region were obtained through calculation.

[0012] Furthermore, preferably, the method for constructing the automatic localization model is as follows: by using labeled fundus images as the training set, the Swin-Unet automatic segmentation model with boundary loss and Focal loss is used for training, the obtained center point markers are compared with the labeled center points to generate loss values, and the model is optimized through backpropagation to obtain the trained automatic localization model.

[0013] Furthermore, the automatic segmentation model employs any one or more of the following algorithms: FCN, U-Net, Unet++, Unet+++, DeepLab v3+, SegNet, RefineNet, CSF-Net, and Swin-Unet; preferably, the automatic segmentation model is Swin-Unet.

[0014] Specifically, the preferred method for constructing the automatic segmentation model is based on the Drishti dataset, using the Swin-Unet model with Boundary loss and Focal loss as loss functions for training, and optimizing the model through the generated loss values ​​and backpropagation to obtain the trained automatic segmentation model.

[0015] Furthermore, the secondary segmentation automatically divides the vascular tree with the vascular centerline into vascular segments using a segmentation algorithm, and simultaneously calculates the bifurcation points of the skeleton line and the endpoints of the vascular vessels to obtain fundus images with a tree structure and single-branched vessels.

[0016] Furthermore, the calculation of the bifurcation point of the skeleton line is obtained by using the intersection operator to calculate each pixel and its surrounding pixels in the fundus image with the vascular center line, and judging the calculated value against a set threshold; the calculation of the vascular endpoint is obtained by performing endpoint calculation on the edge pixels of each vascular segment, and judging the calculated value against a set endpoint operator threshold.

[0017] Furthermore, the interpolation operation on the fundus image with tree structure and single-branch vessels also includes drawing the normal vector of the branch by traversing the single-branch vessels according to the tree structure, and then performing the interpolation operation.

[0018] Furthermore, the interpolation operation employs one or more of the following methods: nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, interpolation, B-spline curve interpolation, nonlinear interpolation, and nearest neighbor interpolation. Preferably, the specific process of the interpolation operation is as follows: nonlinear interpolation is performed along the normal direction of the branch; non-integer contour points are found to generate pixel intensity contours; clustering is then performed along the normal direction of the branch; the pixel intensity contours are connected to form contour lines, and the edges of the contour lines are smoothed; the contour lines are arranged side by side and straightened to generate contours with each blood vessel; then nearest neighbor interpolation is used to process the contours with each blood vessel; the length of the contours with each blood vessel is calculated pixel by pixel along the center line to obtain the average, maximum, and minimum diameter of each blood vessel.

[0019] Furthermore, the segmented vascular regions include the optic disc region, macular region, temporal side of the macula, superior temporal side, inferior temporal side, superior nasal side, and inferior nasal side.

[0020] This application also discloses a system for calculating the diameter of retinal vessels in fundus photography based on region segmentation, the system comprising:

[0021] The acquisition module is used to acquire fundus images to be processed;

[0022] An automatic positioning module is used to locate the center point of the optic cup and optic disc and the fovea of ​​the macula in the fundus photograph to be processed, so as to obtain a fundus photograph with the center point marked.

[0023] The region segmentation module automatically locates and segments the fundus image with the center point marked based on the automatic segmentation model, and obtains the segmented vascular regions.

[0024] The centerline extraction module is used to extract the initial centerline of the blood vessels from each segmented blood vessel region to obtain a fundus image with the centerline of the blood vessels.

[0025] The secondary segmentation module is used to perform secondary segmentation on the fundus image with the vascular centerline to obtain a fundus image with a tree structure and single-branched vessels.

[0026] The calculation module is used to perform interpolation calculations on the fundus images with tree structure and single-branch vessels, and to obtain the average, maximum and minimum diameters of each vascular region through calculation.

[0027] This application also discloses a calculation device for the diameter of retinal vessels in fundus photography based on region segmentation. The device includes a memory and a processor. The memory is used to store program instructions. The processor is used to call the program instructions. When the program instructions are executed, they are used to implement the calculation method for the diameter of retinal vessels in fundus photography based on region segmentation.

[0028] This application also discloses a computer-readable storage medium storing a computer program for calculating the diameter of retinal vessels in fundus photography based on region segmentation. When the computer program is executed by a processor, it implements the method for calculating the diameter of retinal vessels in fundus photography based on region segmentation.

[0029] The aforementioned devices or systems are used in the selection of analytical schemes to assist in the analysis of retinal vascular morphological changes in cardiovascular and systemic diseases; optionally, the analysis includes the correlation between increased venule diameter and the risk of diabetic retinopathy and stroke, hypertensive retinopathy leading to arterial stenosis and decreased arterial-venous ratio, and atherosclerosis leading to arterial stenosis analysis, which has a positive impact on the study of cardiovascular and systemic diseases.

[0030] The above-mentioned equipment or systems are used in the intelligent calculation of various blood vessel diameter parameters;

[0031] The above-mentioned equipment or system is used in the automatic segmentation of the optic cup and optic disc region; optionally, the segmentation includes the tree structure and single-branching of blood vessels in the fundus photograph to obtain the segmented fundus photograph;

[0032] The aforementioned equipment or system is used in predicting the occurrence and development of eye diseases. Optionally, the prediction includes the predictive analysis of various vascular parameters in the segmented fundus images. These various vascular parameters include the diameters of the infratemporal and superior temporal arteries, the diameter of the macular vessels, the diameter of the infratemporal artery, and the diameter of the superior temporal artery, etc., which have a positive impact on and promote the research of optic disc glaucoma, peripheral glaucoma, and macular diseases.

[0033] This invention utilizes deep learning to train an automatic localization model and an automatic segmentation model based on a dataset containing the coordinates of the macula's center point and the optic cup and optic disc. It automatically locates the optic disc center and the macula's center point, simultaneously segmenting each target region. Then, it obtains various vascular parameters within the target region pixel by pixel through interpolation, normal vector processing, and smoothing. This overcomes the limitations of methods that rely solely on experience to calculate arteriovenous morphological changes and methods that only calculate the average diameter of blood vessels. It is highly innovative and will significantly advance research on fundus photography data analysis.

[0034] Advantages of this application:

[0035] 1. This application innovatively discloses a method for calculating the diameter of retinal vessels in fundus photography based on region segmentation. This method automatically divides the target regions in fundus photography based on a deep learning segmentation model, and simultaneously calculates various vascular parameters in each target region (e.g., optic disc area, macular area, temporal side of the macula, superior temporal side, inferior temporal side, superior nasal side, and inferior nasal side), objectively improving the accuracy and depth of data analysis.

[0036] 2. This application innovatively extracts the centerline of each target region after automatic segmentation to obtain a vascular tree with the vascular centerline. Then, through secondary segmentation, fundus images with tree structure and single-branch vessels are obtained. At the same time, the intersection point operator is used to calculate the bifurcation point of the skeleton line and the endpoint operator is used to calculate the endpoint of the vessel. Thus, the average diameter and the maximum and minimum diameter of each vascular region are obtained, which significantly improves efficiency.

[0037] 3. This application creatively discloses fundus images that are automatically segmented based on a deep learning segmentation model. The normal vector of each branch is calculated by traversing the single-branch vessels according to the tree structure. Through interpolation and smoothing, parameters of each vascular region, including the diameter of the inferotemporal artery, the diameter of the superior temporal artery, and the diameter of the macular vessels, are obtained. This can better reflect the characteristics of each vascular region parameter. Given that the vessels in different regions have predictive significance for different diseases, this application can be more accurately applied to the auxiliary analysis of the occurrence and development of diseases related to fundus image data. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic flowchart of the method for calculating the diameter of retinal vessels in fundus photography based on region segmentation, provided in an embodiment of the present invention.

[0040] Figure 2 This is a single-branch vascular diagram with marked serial numbers provided in the embodiments of the present invention; wherein, zone 1 - optic disc area, zone 2 - macular area, zone 3 - temporal side of the macular area, zone 4 - superior temporal side, zone 5 - inferior temporal side, zone 6 - superior nasal side, zone 7 - inferior nasal side;

[0041] Figure 3 This is a fundus image with a vascular centerline provided in an embodiment of the present invention;

[0042] Figure 4 This is a process diagram of calculating the diameter of retinal vessels in fundus photography based on region segmentation, provided in an embodiment of the present invention.

[0043] Figure 5 This is an effect analysis diagram of retinal vessel diameter based on region segmentation provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of a calculation device for retinal vessel diameter based on region segmentation provided in an embodiment of the present invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0046] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be performed in the order they appear herein, or may be performed in parallel. The operation numbers, such as S101, S102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel.

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Figure 1 This is a schematic flowchart of a method for calculating the diameter of retinal vessels in fundus photography based on region segmentation, provided in an embodiment of the present invention. Specifically, it includes the following steps:

[0049] S101: Obtain the fundus image to be processed.

[0050] In one embodiment, the fundus images to be processed include a dataset containing the macular center point, optic disc center coordinates, and optic cup-optic disc. Optionally, the dataset includes publicly available datasets such as Drive fundus images and Drishti fundus images with optic cup-optic disc and fovea localization.

[0051] In one specific embodiment, the data used in the model involved in this invention includes 40 fundus images from the public dataset Drive as a training and test set for vascular segmentation, and 100 images from the Drishti dataset as a training and test set for optic cup, optic disc, and fovea localization.

[0052] S102: Based on the automatic localization model, mark the center point of the fundus image to be processed to obtain a fundus image with the center point marked.

[0053] In one embodiment, the method for center point marking involves automatically locating the optic disc center point and macula center point in the fundus image to be processed using an automatic localization model, thereby obtaining a fundus image with center point markings. Preferably, the automatic localization model is constructed as follows: using the labeled fundus images as a training set, a Swin-Unet automatic segmentation model with boundary loss and Focal loss is trained; the obtained center point markings are compared with the labeled center points to generate loss values; and the model is optimized through backpropagation to obtain a trained automatic localization model.

[0054] In one specific embodiment, the automatic localization model is constructed as follows: The training dataset is the Drishti dataset, which contains the coordinates of the center point of the optic disc and the center point of the macula (including 50 training images and 50 test images). The model is trained using Swin-Unet with Boundary loss and Focal loss. The obtained center point labels are compared with the labeled center points to generate loss values. The model is optimized through backpropagation to obtain the trained automatic localization model.

[0055] S103: Based on the automatic segmentation model, the fundus image with center point marking is segmented to obtain the segmented vascular regions.

[0056] In one embodiment, the automatic segmentation model employs one or more of the following algorithms: FCN, U-Net, Unet++, Unet+++, DeepLab v3+, SegNet, RefineNet, CSF-Net, and Swin-Unet; preferably, the automatic segmentation model is Swin-Unet.

[0057] Each decoder layer in Unet+++ fuses small-scale and same-scale feature maps from the encoder, as well as large-scale feature maps from the decoder, which capture both fine-grained and coarse-grained semantics across the entire scale.

[0058] Deeplab v3+ is a network architecture in the Deeplab series. It is a culmination of dilated convolution, encoding / decoding, and multi-scale models, and is also one of the most commonly used semantic segmentation network models.

[0059] The RefineNet model downsamples using ResNet and achieves deeper fusion of multi-scale features through a series of convolutions, fusions, and pooling. It employs chained residual pooling, uses pooling with windows of different sizes, and fuses features using residual connections and learnable weights.

[0060] CSF-Net is a color spectral fusion network for semantic labeling of airborne laser scanning point clouds. It is trained in an end-to-end manner using weighted cross-entropy loss to build a multimodal fusion network.

[0061] Swin-Unet, based on the Swin Transformer block, constructs a symmetric encoder-decoder architecture with skip connections. It develops a patch extension layer that can achieve upsampling and feature dimension increase without convolution or interpolation operations, and finally constructs a U-shaped encoder-decoder structure based purely on Transformer.

[0062] In one specific embodiment, the preferred process for constructing the automatic segmentation model is as follows: the training dataset consists of 40 images from the Drive fundus photographs and 50 images from the Drishti optic cup and optic disc dataset as the segmentation training set. The Swin-Unet model is used for training, and the model is adjusted by using other loss functions, namely boundary loss and Focal loss. The final model achieves an ACC of 99.6% on the validation set.

[0063] In a more specific embodiment, the segmented vascular regions are as follows: Figure 2 The areas occupied by the single-branch vessels marked with serial numbers shown mainly include the optic disc area, macula, temporal side of the macula, superior temporal side, inferior temporal side, superior nasal side, and inferior nasal side. Figure 2 The criteria for dividing each region are as follows:

[0064] Zone 1 - Optic Disc Zone: The image is centered on the optic disc and bounded by the segmented image of the optic disc (specifically, a circle is drawn with the center of the optic disc as the center point and the diameter of the optic disc as the diameter).

[0065] Area 2 - Macular area: The image is drawn with the macula as the center and the diameter of the optic disc as the diameter, which is the macular area.

[0066] Area 3 - Temporal side of the macula: The image is located on the temporal side of the macula, within the horizontal tangent lines above and below the macula.

[0067] Zone 4 - Superior temporal: The lower edge of the image is tangent to the horizontal line of the upper edge of the optic disc region, and the nasal edge of the image is tangent to the central line of the optic disc.

[0068] Zone 5 - Inferotemporal: The upper edge of the image is tangent to the horizontal line of the lower edge of the optic disc region, and the nasal edge of the image is tangent to the central line of the optic disc.

[0069] Zone 6 - Superior Nasal Side: The lower edge of the image is tangent to the horizontal line of the upper edge of the optic disc region, and the temporal edge of the image is tangent to the central line of the optic disc.

[0070] Zone 7 - Nasal Inferior Side: The lower edge of the image is tangent to the horizontal line of the lower edge of the optic disc region, and the temporal edge of the image is tangent to the central line of the optic disc.

[0071] Specifically, the steps for dividing the target area for fundus imaging are as follows: a, b, c, d, and e.

[0072] a. Connect the coordinates of the fovea center and the center of the test disc, and draw a straight line to divide the fundus imaging area into the upper and lower sides;

[0073] b. On the original image, with the optic disc and the fovea of ​​the macula as the center, and the diameter of the optic disc as the diameter, draw circles at the optic disc and the fovea of ​​the macula respectively. The circular area at the macula is the macular area, and the circular area at the optic disc is the optic disc area.

[0074] c. At the highest and lowest points of the viewing disk boundary, draw lines parallel to the straight line drawn in a;

[0075] d. Draw a perpendicular line from the straight line in step a at the center of the macula to divide the fundus imaging area into the temporal and nasal sides;

[0076] e. The temporal side of the macula is located within the boundary of the parallel lines drawn in step c, outside the circular boundary of the macula drawn in step b.

[0077] S104: Extract the initial centerline of the blood vessels from each segmented vascular region to obtain a fundus image with the centerline of the blood vessels.

[0078] In one embodiment, the segmentation of the fundus image to be processed includes automatically detecting and removing vascular pixels in the fundus image through a morphological refinement process, resulting in, for example, Figure 3 The image shown is a fundus photograph with the central line of the blood vessels.

[0079] S105: Perform secondary segmentation on fundus images with vascular centerlines to obtain fundus images with tree-like structures and single-branched vessels.

[0080] In one embodiment, the intersection points of the skeleton lines and the endpoints of the blood vessels in the fundus image with the vascular centerline obtained after the first segmentation are calculated. After removing the intersection points, each blood vessel segment is written into the tree structure to obtain a fundus image with a tree structure and single-branched blood vessels.

[0081] Specifically, the bifurcation point of the skeleton line is calculated by using the intersection operator on each pixel and its surrounding pixels in the fundus image with the vascular centerline, and the calculated value is judged against the set threshold; the endpoint of the blood vessel is calculated by performing endpoint calculation on the edge pixels of each blood vessel segment, and the calculated value is judged against the set endpoint operator threshold.

[0082] Furthermore, the bifurcation points of the skeleton lines are calculated by using the intersection operator on each pixel and its surrounding pixels in the vascular tree with the vascular centerline, and then comparing the calculated value with a set threshold. Preferably, the intersection threshold is set to 4. Preferably, the calculation formula for the intersection operator is as follows:

[0083]

[0084] Wherein, j1-j5 are different convolutional kernels used to extract intersection point features, respectively extracting intersection point features of different shapes. When the extracted feature value is greater than the set threshold, it is marked as an intersection point; preferably, the set threshold is 4.

[0085] Furthermore, endpoint calculations are performed on the edge pixels of each blood vessel segment. The calculated feature values ​​and a set endpoint operator threshold are used to determine the blood vessel endpoints. Preferably, the formula for the endpoint operator is as follows:

[0086]

[0087]

[0088] Wherein, k1-k8 are different convolution kernels used to extract endpoint features, respectively extracting endpoint features of different shapes. When the extracted feature value is greater than the set endpoint operator threshold, it is marked as an endpoint; preferably, the set endpoint operator threshold is 255.

[0089] S106: Perform interpolation on fundus images with a tree-like structure and single-branch vessels to obtain the average diameter and maximum and minimum diameter of each vascular region.

[0090] In one embodiment, the interpolation operation employs one or more of the following methods: nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, interpolation, B-spline curve interpolation, nonlinear interpolation, and nearest neighbor interpolation. Preferably, the specific process of the interpolation operation is based on nonlinear interpolation along the normal direction of the branch. By finding non-integer contour points, pixel intensity contours are generated. Then, clustering is performed along the normal direction of the branch, connecting the pixel intensity contours to form contour lines and smoothing the edges of the contour lines. The contour lines are arranged side by side and straightened to generate contours with each blood vessel. Then, nearest neighbor interpolation is used to process the contours with each blood vessel. The length of the contours with each blood vessel is calculated pixel by pixel along the center line to obtain the average diameter and the maximum and minimum diameters of each blood vessel region.

[0091] Specifically, the diameters of each blood vessel include the optic disc area diameter, macular area diameter, temporal macular area diameter, superior temporal area diameter, inferior temporal area diameter, superior nasal area diameter, and inferior nasal area diameter.

[0092] In a specific embodiment, Figure 4 The diagram illustrates the process of locating the fovea and optic disc center points based on fundus photography, followed by region segmentation. Then, the vascular centerlines are extracted from each segmented region. The vessels are then branched and their segments are written into a tree structure. Finally, the fundus photograph with the tree structure and branched vessels is used to traverse the branched vessels according to the tree structure, and the normal vectors of the branches are drawn. Interpolation is then performed to obtain the results shown below. Figure 5 The image shown is a diagram of the retinal vessel diameter in a fundus photograph based on region segmentation.

[0093] The preferred method for calculating the normal vector is as follows:

[0094] (1) Calculate the vector V along the x-axis along the direction of the blood vessel. x The vector V along the y-axis y As shown in the formula below;

[0095] V x =A x+3 -A x-3

[0096] V y =A y+3 -A y-3

[0097] Among them, A x 、A y For the origin of the coordinate system where the normal vector needs to be calculated, A x+3 A x The third x-coordinate along the direction of the blood vessel, A x-3 Similarly; A y+3 A y The third y-value along the direction of the blood vessel, A y-3Similarly;

[0098] (2) Formula for calculating the coordinates of the endpoint of the normal vector (C) x , C y ), (D x D y As shown in the following formula, d is the preset normal vector length;

[0099]

[0100]

[0101]

[0102]

[0103] (3) Connect the coordinates of the endpoint of the normal vector and calculate the normal vector as shown in the following formula: τ x τ y As shown.

[0104] τ x =C x -D x

[0105] τ y =C y -D y

[0106] Where, τ x Let τ be the coordinate of the normal vector along the x-axis. y The coordinates of the normal vector along the y-axis.

[0107] The above method is used in a scheme for calculating the diameter of retinal vessels in fundus images based on region segmentation. Based on the fundus image to be processed, the target region is segmented and the diameter parameters of each vessel in the target region are calculated based on interpolation and normal vectors. Figure 5 This is an effect analysis diagram provided by an embodiment of the present invention, showing the effect diagram of the diameter of the automatically segmented target region. It demonstrates that the proposed method for calculating the diameter of retinal vessels in fundus photography based on region segmentation has strong discrimination and automatic analysis and calculation capabilities. Therefore, based on fundus photography with region segmentation, and then obtaining various vascular parameters, including the diameters of the inferotemporal artery, superior temporal artery, and macular vessels, through interpolation calculations, can better reflect the characteristics of each vascular parameter. Given that vessels in different regions have different predictive significance for different diseases, this application is more advantageous in the auxiliary analysis of disease occurrence and development related to fundus photography data.

[0108] The system for calculating the diameter of retinal vessels in fundus photography based on region segmentation provided in this embodiment of the invention includes:

[0109] The acquisition module is used to acquire fundus images to be processed;

[0110] The automatic positioning module marks the center point of the fundus image to be processed based on the automatic positioning model, and obtains the fundus image with the center point marked.

[0111] The region segmentation module performs region segmentation on the fundus image with center point markings based on an automatic segmentation model to obtain the segmented vascular regions.

[0112] The centerline extraction module is used to extract the initial centerline of the blood vessels from each segmented vascular region to obtain a fundus image with the vascular centerline.

[0113] The secondary segmentation module is used to perform secondary segmentation on fundus images with vascular centerlines to obtain fundus images with tree structure and single-branch vessels;

[0114] The calculation module is used to perform interpolation calculations on fundus images with tree-like structures and single-branch vessels, and to obtain the average, maximum, and minimum diameters of each vascular region through calculation.

[0115] Figure 6 This invention provides a calculation device for calculating the diameter of retinal blood vessels in fundus imaging based on region segmentation, comprising: a memory and a processor; the device may also include: an input device and an output device.

[0116] Memory, processor, input devices, and output devices can be connected via a bus or other means. Figure 6 The example shown is a bus connection.

[0117] Memory is used to store program instructions;

[0118] The processor is used to call program instructions, which, when executed, are used to perform the above-mentioned method for calculating the diameter of retinal vessels in fundus photography based on region segmentation.

[0119] The present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for calculating the diameter of retinal vessels in fundus photography based on region segmentation.

[0120] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0122] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0123] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The aforementioned integrated modules can be implemented in hardware or as software functional modules.

[0124] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0125] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0126] The computer device provided by the present invention has been described in detail above. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for calculating the diameter of retinal vessels in fundus photography based on region segmentation, including: Obtain fundus images to be processed; The center point of the fundus image to be processed is marked based on the automatic localization model to obtain a fundus image with center point marking; Based on the automatic segmentation model, the fundus image marked with the center point is segmented to obtain the segmented vascular regions. The initial centerline of the blood vessels in each segmented vascular region is extracted to obtain a fundus image with the centerline of the blood vessels. The fundus image with the vascular centerline is segmented twice to obtain a fundus image with a tree structure and single-branched vessels; Interpolation is performed on the fundus images with the tree structure and single-branch vessels to obtain the average, maximum, and minimum diameters of each vessel region. The specific interpolation process involves performing nonlinear interpolation along the normal vector direction of the branch, generating pixel intensity contours by finding non-integer contour points, clustering along the normal vector direction of the branch, connecting the pixel intensity contours to form contour lines, and smoothing the edges of the contour lines. Then, the contour lines are arranged side by side and straightened to generate the contours of each vessel. The contours of each vessel are then processed using nearest neighbor interpolation, and the length of the contours of each vessel is calculated pixel by pixel along the center line to obtain the average, maximum, and minimum diameters of each vessel region.

2. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The center point marker is obtained by automatically locating the center point of the optic cup and the fovea of ​​the macula in the fundus image to be processed using an automatic positioning model.

3. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The automatic localization model is constructed as follows: using labeled fundus images as the training set, the Swin-Unet model with Boundary loss and Focal loss is used for training. The obtained center point labels are compared with the labeled center points to generate loss values. The model is then optimized through backpropagation to obtain the trained automatic localization model.

4. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The automatic segmentation model employs one or more of the following algorithms: Unet+++, DeepLab v3+, SegNet, RefineNet, CSF-Net, and Swin-Unet.

5. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The automatic segmentation model is constructed based on the Drishti dataset as the training set, and is trained using the Swin-Unet model with Boundary loss and Focal loss. The model is optimized by generating loss values ​​and backpropagation to obtain the trained automatic segmentation model.

6. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The secondary segmentation uses a segmentation algorithm to automatically divide the vascular tree with the vascular centerline into vascular segments, and simultaneously calculates the bifurcation points of the skeleton line and the endpoints of the vessels to obtain fundus images with a tree structure and single-branched vessels.

7. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 6, characterized in that, The bifurcation points of the skeleton line are calculated by using the intersection operator on each pixel and its surrounding pixels in the fundus image with the vascular centerline. The calculated feature value is then compared with a set threshold. When the feature value is greater than the set threshold, it is marked as an intersection point. Similarly, the vascular endpoints are calculated by performing endpoint calculations on the edge pixels of each vascular segment. The calculated feature value is then compared with a set endpoint operator threshold. When the calculated feature value is greater than the set endpoint operator threshold, it is marked as an intersection point.

8. The method for calculating the diameter of retinal vessels in fundus photography based on region segmentation according to claim 1, characterized in that, The segmented vascular regions include the optic disc region, the macular region, the temporal side of the macula, the superior temporal side, the inferior temporal side, the superior nasal side, and the inferior nasal side.

9. A calculation system for retinal vessel diameter in fundus photography based on region segmentation, characterized in that, The system includes: The acquisition module is used to acquire fundus images to be processed; An automatic positioning module marks the center point of the fundus image to be processed based on an automatic positioning model, thereby obtaining a fundus image with the center point marked. The region segmentation module performs region segmentation on the fundus image marked with the center point based on the automatic segmentation model to obtain the segmented vascular regions. The centerline extraction module is used to extract the initial centerline of the blood vessels from each segmented blood vessel region to obtain a fundus image with the centerline of the blood vessels. The secondary segmentation module is used to perform secondary segmentation on the fundus image with the vascular centerline to obtain a fundus image with a tree structure and single-branched vessels. The calculation module is used to perform interpolation operations on the fundus image with the tree structure and single-branch vessels, and to obtain the average, maximum, and minimum diameter of each vessel region through calculation. The specific process of the interpolation operation is to perform nonlinear interpolation along the normal vector direction of the branch, generate pixel intensity contours by finding non-integer contour points, then perform clustering along the normal vector direction of the branch, connect the pixel intensity contours to form contour lines and smooth the edges of the contour lines, then arrange the contour lines side by side and straighten them to generate the contours of each vessel, then use nearest neighbor interpolation to process the contours of each vessel, and calculate the length of the contour of each vessel pixel by pixel along the center line to obtain the average, maximum, and minimum diameter of each vessel region.

10. A calculation device for retinal vessel diameter in fundus photography based on region segmentation, characterized in that, The device includes: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions, and when the program instructions are executed, they are used to execute the method for calculating the diameter of retinal vessels in fundus imaging based on region segmentation as described in any one of claims 1-8.

11. A computer-readable storage medium storing thereon a computer program for automatically calculating the diameter of retinal vessels in fundus photography based on region segmentation, characterized in that, When the computer program is executed by the processor, it implements the method for calculating the diameter of retinal vessels in fundus photography based on region segmentation as described in any one of claims 1-8.

Images