Ultra-wide-angle fundus image analysis method and system

Abstract

According to the ultra-wide-angle fundus image analysis method and system, an ultra-wide-angle fundus image is divided into a macular region, a posterior pole region and a peripheral region, the focus of each region is automatically recognized, and then auxiliary diagnosis of various fundus diseases of the whole image is achieved. According to the method and the system, efficient, accurate and automatic fundus disease analysis and evaluation are realized through region partitioning and focus analysis of the ultra-wide-angle fundus image, particularly, the method and the system have remarkable advantages in retina peripheral region lesion detection, and the system can be applied to screening and auxiliary diagnosis of clinical ophthalmic diseases on the basis of a deep learning algorithm model. Along with the rapid development of deep learning, a convolutional neural network (CNN) obtains a significant result in the field of image classification, and the method carries out the partitioning of an ultra-wide-angle fundus image based on a deep learning model, and carries out the analysis and evaluation of common fundus diseases and lesions of the whole image, and optic nerve diseases including the lesions of glaucoma. The invention provides an automatic and accurate fundus image auxiliary diagnosis method.

Description

Technical Field

[0001] The present invention discloses a fundus image analysis method, and more particularly relates to an ultra-wide-angle fundus image analysis method and system based on a deep learning model, which is specifically applied to the automated auxiliary diagnosis of fundus diseases and belongs to the technical field of medical image processing. Background Art

[0002] Ultra-wide-angle fundus imaging is an important examination technology in the field of ophthalmology, mainly used to detect and evaluate the health status of the fundus (structures such as the retina, optic nerve and choroid). Compared with traditional fundus imaging technology (such as conventional fundus photography), ultra-wide-angle fundus imaging technology can capture a wider range of retinal images, thereby providing more comprehensive fundus information, especially suitable for detecting diabetic retinopathy, peripheral retinal diseases such as peripheral degeneration, retinal holes, high myopia fundus lesions and peripheral exudation, hemorrhage, vascular lesions and other eye diseases.

[0003] The technical principle of ultra-wide-angle fundus imaging is to collect high-resolution images of the retina through special imaging equipment. These devices usually use a technology called "laser scanning" or "optical photography" and are combined with specific lens designs to capture wide areas of the retina.

[0004] The ultra-wide-angle fundus imaging technology in the existing technology can provide retinal images with a field of view of 100 to 200 degrees or even more in one imaging, which is far beyond the 45° to 90° field of view of traditional fundus imaging. Compared with traditional fundus images, it can cover a wider area, especially for the discovery of lesions in the peripheral area of ​​the fundus. However, the regional division of ultra-wide-angle images and the automatic analysis of lesions in different areas still have great technical challenges. Due to the complexity of the eyeball structure, some areas may be blurred or noisy during the image acquisition process, affecting the accuracy of diagnosis, especially when the patient's pupil is small or there are lesions in the fundus, which may affect the imaging quality. Traditional zoning and lesion analysis methods often rely on manual annotation, which is inefficient and prone to errors, and missed or misdiagnosed due to the uneven level of doctors and subjective judgment. Summary of the invention

[0005] In view of the shortcomings of the above-mentioned prior art ultra-wide-angle fundus image technology in the zoning and lesion analysis method that rely on manual labeling, are inefficient and prone to errors, the present invention provides an ultra-wide-angle fundus image analysis method and system, which divides the ultra-wide-angle fundus image into the macular area, the posterior pole area and the peripheral area, and automatically identifies the lesions in each area, thereby achieving auxiliary analysis and diagnosis of various fundus diseases in the whole image including the optic nerve and glaucoma, and presenting them in the form of risk assessment and quantitative parameters.

[0006] The technical solution adopted by the present invention to solve the technical problem is: an ultra-wide-angle fundus image analysis method, the method comprising the following steps: Step S1, establishing a basic database: an imaging device collects high-resolution ultra-wide-angle fundus images of the retina to establish a basic database; Step S2, modeling: using a deep learning model to train the ultra-wide-angle fundus images in the basic database of step S1, generate model weights, and the training objects include a normal standard database and various lesion disease databases to establish an artificial intelligence primary model; Step S3, ultra-wide-angle fundus image partitioning: the high-resolution ultra-wide-angle fundus image of the retina acquired by the imaging device in step S1 is divided into regions according to the structural characteristics of the retina, wherein the fovea is taken as the center of the circle, the distance from the center of the optic disc to the fovea is taken as a unit dm, the macular area is a circle with a radius of 0.5 dm and the posterior pole area is a circle with a radius of 1.5 dm and the part outside the posterior pole area is the peripheral area; the optic disc area is divided separately, the optic disc boundary and the optic cup boundary are divided to separate the optic disc and the optic cup, and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned; Step S4, regional auxiliary lesion detection: each segmented region will be subjected to auxiliary lesion detection, such as macular hole, macular edema, epiretinal membrane, retinal peripheral degeneration, atrophy, exudation, retinal hemorrhage, laser spot, and optic nerve disease; Step S5, whole-image disease-aided detection: after completing the lesion-aided detection of each partition, further whole-image assisted analysis of the disease information of the fundus image is performed to assist in the detection of the following eye diseases, provide auxiliary reference information for doctors, and assess the risk probability of the corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, high myopia fundus lesions, retinitis pigmentosa, wet age-related macular degeneration, hypertensive retinopathy, retinal detachment, retinopathy of prematurity, retinal holes, leopard-shaped fundus, fibrosis, cataracts, and drusen; Step S6, lesion auxiliary verification: the model output result and the image annotation are calculated to achieve auxiliary verification; Step S7, auxiliary verification of diseases in the whole image: The auxiliary detection results of multiple diseases in the whole image are calculated through the output of the model and the doctor's annotation results to provide reference information for the doctor's diagnosis.

[0007] An ultra-wide-angle fundus image analysis system, the system comprising the following module units: Basic database establishment unit: used to establish a basic database by collecting high-resolution ultra-wide-angle fundus images of the retina through imaging equipment; Modeling unit: Use deep learning models to train the ultra-wide-angle fundus images in the basic database of the basic database establishment unit to generate model weights. The training objects include normal standard databases and databases of various lesions and diseases to establish a primary artificial intelligence model. Ultra-wide-angle fundus image partitioning unit: used to divide the ultra-wide-angle fundus image into regions according to the structural characteristics of the retina, wherein the fovea is taken as the center of the circle, the distance from the center of the optic disc to the fovea is a unit of dm, the macular area is a circle with a radius of 0.5 dm and the posterior pole area is a circle with a radius of 1.5 dm and the part outside the posterior pole area is the peripheral area; the optic disc area is divided separately, and the optic disc boundary and the optic cup boundary are divided to separate the optic disc and the optic cup, and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned; Regional auxiliary lesion detection unit: used to perform auxiliary lesion detection on each segmented area, such as macular hole, macular edema, epiretinal membrane, retinal peripheral degeneration, atrophy, exudation, retinal hemorrhage, laser spot, and optic nerve disease; Whole-image disease auxiliary diagnosis and detection unit: used to further analyze the disease information of the fundus image after completing the auxiliary analysis of each partition lesion. After completing the auxiliary detection of each partition lesion, further analyze the disease information of the fundus image of the whole image to assist in the detection of the following eye diseases, provide auxiliary reference information for doctors, and evaluate the risk probability of the corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, high myopia fundus lesions, retinitis pigmentosa, wet age-related macular degeneration, hypertensive retinopathy, retinopathy of prematurity, retinal detachment, retinal tears, leopard-shaped fundus, fibrous proliferation, cataract and drusen; Lesion auxiliary verification unit: used for calculation with image annotation to assist in disease verification; Whole-image disease auxiliary verification unit: used to calculate the auxiliary detection results of multiple diseases in the whole image through the output of the model and the doctor's annotation results.

[0008] The technical solution adopted by the present invention to solve the technical problem further includes: The step S1, when establishing the basic database, includes the following sub-steps: Step S1-1, image quality screening: During the image quality screening process, the quality of the collected ultra-wide-angle fundus images is evaluated by manual annotation. Then, the doctor further reviews the screening results to ensure the consistency of the screening criteria and the quality of the images. Step S1-2, labeling: labeling various lesions and disease types for the qualified images screened out in step S1-1.

[0009] In step S7, the auxiliary verification of the whole image disease type is to use sensitivity, specificity, accuracy and AUC to evaluate the comprehensive performance of the model.

[0010] The basic database establishment unit includes the following subunits: Image quality screening subunit: used to evaluate the quality of the collected ultra-wide-angle fundus images in the image quality screening process, usually by manual annotation; Then, the doctor further reviews the screening results to ensure the consistency of the screening criteria and the quality of the images; Labeling subunit: used to label various lesions and diseases on qualified images screened out in the image quality screening subunit.

[0011] The whole-image disease auxiliary verification unit uses the comprehensive performance of the sensitivity, specificity, accuracy and AUC evaluation model for evaluation and verification.

[0012] The beneficial effects of the present invention are as follows: the present invention realizes efficient and accurate automated fundus disease analysis and evaluation through regional partitioning and lesion analysis of ultra-wide-angle fundus images, especially has significant advantages in the detection of lesions in the peripheral area of ​​the retina. The system is based on a deep learning algorithm model and can be applied to the screening and auxiliary diagnosis of clinical ophthalmic diseases. With the rapid development of deep learning, convolutional neural networks (CNNs) have achieved remarkable results in the field of image classification. The present invention provides an automated and accurate fundus image-assisted diagnosis method based on deep learning models to partition and analyze common fundus diseases and lesions in the entire ultra-wide-angle fundus images, as well as lesion analysis and evaluation of optic nerve diseases including glaucoma.

[0013] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the eye structure in the present invention.

[0015] Figure 2 It is a schematic diagram of the structural area division of the retina in the present invention.

[0016] Figure 3 This is a screenshot of the operation of Example 1 of retinal lesion risk assessment in the present invention.

[0017] Figure 4 This is a screenshot of the operation of Example 2 of retinal lesion risk assessment in the present invention.

[0018] Figure 5 This is a screenshot of the retinal disease and lesion risk assessment operation in the present invention. DETAILED DESCRIPTION

[0019] This embodiment is a preferred implementation mode of the present invention. Other embodiments whose principles and basic structures are the same or similar to those of this embodiment are within the protection scope of the present invention.

[0020] Please refer to the attached Figure 1 To Attachment Figure 5 The present invention mainly protects an ultra-wide-angle fundus image analysis method, which is based on ultra-wide-angle fundus images and uses a deep learning model to perform regional partitioning and disease and lesion identification, and mainly includes the following steps: Step S1, establishment of basic database: imaging equipment collects high-resolution ultra-wide-angle fundus images of the retina to establish a basic database. In this embodiment, laser scanning or new optical photography is used to obtain fundus images. Different from traditional optical photography, the laser scanning fundus range is much larger than the range of optical photography, which can reach 100-200 degrees, or even more, while traditional optical photography is only 45-60 degrees. Fundus equipment with new optical photography technology can reach 130 degrees. Therefore, laser scanning fundus images or new optical photography can obtain more fundus lesions, which are applicable to common fundus diseases and lesions. One anteroposterior film and four upper, lower, nasal and temporal deviation maps are obtained for each eye. In this embodiment, the establishment of the basic database includes the following sub-steps: Step S1-1, image quality screening: In the process of image quality screening, manual annotation is usually used to evaluate the quality of the collected ultra-wide-angle fundus images. In this embodiment, manual annotation will first be performed by an experienced annotation engineer to perform a first round of preliminary screening to screen out obviously unqualified images, such as: images that are too blurred, overexposed or underexposed, out of focus, or images whose retina is partially blocked. In this embodiment, the screening criteria for clarity are to visually assess whether the image is clear, especially whether important areas such as the optic disc, macular area and vascular structure are clearly visible and sharp enough to support the ophthalmologist's diagnosis. Specifically, the standards for clarity and visibility may include: clear optic disc edge contour; discernible macular details and no blur in the fovea; the main retinal blood vessels (such as the central retinal artery and vein) can be clearly distinguished, and their bifurcation points can be observed; if the details of these anatomical structures in the image cannot be clearly distinguished, the image will be considered to be substandard in clarity, and the unqualified images will be discarded; the screening criteria for image brightness uniformity are to visually check the bright spots or shadows in the image to ensure uniform illumination and avoid local overbrightness or overdarkness causing loss of details. Brightness uniformity mainly refers to whether the light distribution in the image is balanced and there are no obvious bright spots or dark spots. The standards include: there are no overexposed or underexposed areas in the image; there should be no significant difference in brightness between the optic disc and macular area to ensure uniform contrast in each area and avoid visual confusion in the lesion area; local overbrightness or overdarkness will cover up detailed information, such as the bifurcation of the obscuring blood vessels or lesions in the macular area, The screening criteria for focus issues are that visual images should ensure good focus, especially without blur at important anatomical structures. Focus mainly refers to that the focus of the image should be on key anatomical structures, such as the optic disc, macular area, etc. The specific standards are: the optic disc area should be able to show clear boundaries and central depressions; the fovea of ​​the macular area should be free of blur to ensure that any lesions (such as holes or hemorrhages) can be clearly distinguished; the vascular system should be able to show sharp edges to distinguish subtle changes in the vascular wall. If important structures are blurred, the image will be considered to be poorly focused and unqualified, and unqualified images will be discarded; the screening criteria for lesion masking is that if the lesion or other factors such as eyelids and eyelashes cover important areas in the image, the image will also be judged as unqualified. When factors such as lesions, eyelids or eyelashes cover key areas in the image, the image will usually be judged as unqualified. However, if the lesion itself is an important feature required for diagnosis, annotation engineers and doctors will evaluate whether the coverage of the lesion affects the visibility of other key structures and discard unqualified images.

[0021] Then, senior doctors (generally doctors with more than 10 years of clinical experience are called senior doctors) will further review the screening results to ensure the consistency of the screening criteria and the quality of the images, so as to better support accurate clinical judgment.

[0022] Step S1-2, labeling: the qualified images screened out in step S1-1 are labeled with various lesions and diseases (1 for disease presence and 0 for no disease). Generally, two or more middle-aged doctors (doctors with 5 to 10 years of clinical experience are usually called middle-aged doctors) are required to label the images to ensure the accuracy of the results. In practice, the images may be labeled by multiple middle-aged doctors and then reviewed by expert doctors (expert doctors usually refer to doctors with profound knowledge, rich experience and high technical level in a certain medical field). If the labeling results of the middle-aged doctors are inconsistent, the opinions of the reviewing expert doctors shall prevail.

[0023] Step S2, modeling: using a deep learning model to train the ultra-wide-angle fundus images in the basic database of step S1, generate model weights (in this embodiment, the InceptionV3 model is used), the training objects include a normal standard database and various lesion disease databases, and establish an artificial intelligence primary model; In this embodiment, the training model includes three components: an input layer, a hidden layer, and an output layer. The input layer is used to receive the input of the ultra-wide-angle fundus image and pass it to the hidden layer, and finally pass the signal to the output layer to finally obtain the prediction result of the lesion or disease type; Step S3, ultra-wide-angle fundus image partitioning: The high-resolution ultra-wide-angle fundus image of the retina captured by the imaging device in step S1 is divided into regions according to the structural characteristics of the retina. The ultra-wide-angle fundus image is divided into three regions based on the conventional definition of ophthalmology and clinical experience as the segmentation standard. The fovea is taken as the center of the circle, and the distance from the center of the optic disc to the fovea is taken as one unit (disc-to-macular distance, expressed in dm). The macular area is a circle with a radius of 0.5 dm and a posterior pole area is a circle with a radius of 1.5 dm and a posterior pole area. The part outside the posterior pole area is the peripheral area. Among them, Macular area: This is the central area of ​​the retina, usually located near the optic nerve head. The segmentation of this area is based on the geometric position of the macula, usually a certain distance from the center of the optic disc, with the fovea as the center and a radius of 0.5 times the distance from the optic disc to the center of the macula (disc-to-macular distance, dm), which is defined as the macular area.

[0024] The posterior pole: The distance from the optic disc to the center of the macula is taken as the unit, with a radius of 1.5 times dm. It covers the macular area and part of the optic disc area and is regarded as the central area of ​​the retina.

[0025] Peripheral area: All areas outside the posterior pole are defined as the peripheral area, which is the key area for detecting peripheral retinal lesions (such as peripheral retinal detachment and other lesions).

[0026] Optic disc area: This is the area where the optic nerve head is located. It is an anatomical structure where the retinal nerve fiber layer converges into the optic nerve in the retina, usually near the macula. The optic disc area is divided separately, and the optic disc boundary and optic cup boundary are divided to separate the optic disc (the optic disc is usually located in the central and inner area of ​​the fundus, where the optic nerve converges. Through fundus examination, the optic disc appears as a round or oval area with a lighter color and clear edges) and the optic cup (the optic cup is located in the center of the optic disc and is a concave area in the optic disc. It usually looks slightly darker than the surrounding optic disc part and appears as a shallower concave. The shape of the optic cup is usually round or oval, and the size and depth can vary. The difference between the optic disc and the optic cup is that the optic disc is a relatively flat area, while the optic cup is the concave part in the center), and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned.

[0027] In this embodiment, in ophthalmological examination, slightly pale and pale are medical terms that describe the color change of the optic disc. Generally, physicians can distinguish them by observation. They are mainly used to determine whether there is a lesion in the optic nerve, especially whether there is glaucoma, optic atrophy and other problems: 1. A normal optic disc is usually light red in color and has clear edges.

[0028] 2. Slightly pale: refers to the optic disc becoming lighter in color, but the change is not obvious, and usually a certain pink or light color can still be seen. This situation may be related to mild optic atrophy, early glaucoma or other mild blood supply deficiency. Slightly pale optic discs are usually more subtle, with clear edges and more local color changes, and usually the entire optic disc will not become pale.

[0029] 3. Pallor: refers to the optic disc becoming noticeably lighter in color, usually close to white or grayish white, lacking the original reddish color. Pallor usually indicates atrophy or death of optic nerve fibers, indicating that the optic nerve is severely damaged. Severe optic nerve atrophy, such as glaucoma, optic neuropathy, optic neuritis, etc., may cause optic disc pallor. In a pale optic disc, the optic cup may appear more obvious.

[0030] Step S4, regional auxiliary lesion detection: each segmented region will be subjected to auxiliary lesion detection. For example: Auxiliary analysis of macular lesions: used to assist in the identification and detection of the following macular lesions, such as: macular hole, macular hemorrhage, macular exudation, macular atrophy, macular edema, epiretinal membrane, etc.

[0031] Auxiliary analysis of posterior pole lesions: used to assist in the identification of the following lesions, such as: posterior pole retinal detachment, hemorrhage, exudation, atrophy, laser spots, leopard-shaped fundus, drusen, retinal artery occlusion, high myopia fundus lesions, wet and age-related maculopathy, branch retinal vein occlusion, retinal artery occlusion, hypertensive retinopathy, and identification of peridiscal retinal nerve fiber layer and defects.

[0032] Auxiliary analysis of peripheral lesions: used to assist in the identification of lesions in the retinal periphery, and to assist in the analysis of the following lesions: peripheral retinal detachment, peripheral holes, peripheral degeneration, peripheral hemorrhage, peripheral laser spots, and peripheral exudation.

[0033] The optic disc area is used to quantify parameters such as the cup-to-disc ratio and color identification (such as pale, slightly pale, and red), and to assist in the analysis of optic nerve diseases, optic atrophy, and glaucoma.

[0034] The segmented images can be used for regional quantitative analysis to calculate parameters such as the lesion area and lesion location distribution in different regions.

[0035] Step S5, auxiliary detection of diseases in the whole image: after completing the auxiliary detection of lesions in each partition, further assist in analyzing the disease information of the fundus image of the whole image, assist in detecting the following eye diseases, provide auxiliary reference information for doctors, and evaluate the risk probability of corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, retinal artery occlusion, high myopia fundus lesions, retinopathy of prematurity, retinitis pigmentosa, retinal detachment, retinal holes, macular holes, wet age-related maculopathy, hypertensive retinopathy, retinal artery occlusion, leopard fundus, fibrous proliferation, retinal hemorrhage, laser spots, drusen, cataracts, atrophy, etc.

[0036] Step S6: Lesion auxiliary verification: Auxiliary verification of macular lesions: Calculate the sensitivity, specificity, accuracy and other indicators of the auxiliary detected macular holes, hemorrhages, exudates, atrophy lesions and image annotations.

[0037] Auxiliary verification of lesions in the posterior pole area: For lesions in the posterior pole area (such as retinal detachment, hemorrhage, etc.) that are assisted in detection, auxiliary verification is performed through image annotation to ensure the accuracy of the model.

[0038] Auxiliary verification of peripheral area lesions: For lesions in the peripheral area that are assisted in detection (such as peripheral retinal detachment, etc.), auxiliary verification is performed through the annotation results of the peripheral area images.

[0039] Step S7: Auxiliary verification of disease types in the whole image: The auxiliary detection results of multiple diseases in the whole image (such as diabetic retinopathy, retinal detachment, etc.) are calculated through the output of the model and the doctor's annotation results. The comprehensive performance of the model is evaluated using sensitivity, specificity, accuracy, AUC, etc., to provide reference information for the doctor's diagnosis.

[0040] The present invention also protects an ultra-wide-angle fundus image analysis system, which is based on ultra-wide-angle fundus images and uses a deep learning model to perform regional partitioning and disease and lesion identification. The system mainly includes the following module units: The basic database establishment unit is used to collect high-resolution ultra-wide-angle fundus images of the retina through imaging equipment to establish a basic database. In this embodiment, laser scanning or new optical photography is used to obtain fundus images. Different from traditional optical photography, the laser scanning fundus range is much larger than the range of optical photography, which can reach 100-200 degrees or even more, while traditional optical photography is only 45-60 degrees. The fundus equipment of the new optical photography technology can reach 130 degrees. Therefore, laser scanning fundus images or new optical photography can obtain more fundus lesions, which are applicable to common fundus diseases and lesions. Each eye obtains one anteroposterior film and four upper, lower, nasal and temporal deviation maps. In this embodiment, the basic database establishment unit includes the following subunits: Image quality screening subunit: used to evaluate the quality of the collected ultra-wide-angle fundus images in the image quality screening process, usually by manual annotation. In this embodiment, the manual annotation will first be performed by an experienced annotation engineer to perform a first round of preliminary screening to screen out obviously unqualified images, such as: images that are too blurred, overexposed or underexposed, out of focus, or images with partial retinal obstruction. In this embodiment, the screening criteria for clarity are to visually assess whether the image is clear, especially whether important areas such as the optic disc, macular area and vascular structure are clearly visible and sharp enough to support the ophthalmologist's diagnosis. Specifically, the standards for clarity and visibility may include: clear optic disc edge contour; discernible macular details and no blur in the fovea; the main retinal blood vessels (such as the central retinal artery and vein) can be clearly distinguished, and their bifurcation points can be observed; if the details of these anatomical structures in the image cannot be clearly distinguished, the image will be considered to be substandard in clarity, and the unqualified images will be discarded; the screening criteria for image brightness uniformity are to visually check the bright spots or shadows in the image to ensure uniform illumination and avoid local overbrightness or overdarkness causing loss of details. Brightness uniformity mainly refers to whether the light distribution in the image is balanced and there are no obvious bright spots or dark spots. The standards include: there are no overexposed or underexposed areas in the image; there should be no significant difference in brightness between the optic disc and macular area to ensure uniform contrast in each area and avoid visual confusion in the lesion area; local overbrightness or overdarkness will cover up detailed information, such as the bifurcation of the obscuring blood vessels or lesions in the macular area, The screening criteria for focus issues are that visual images should ensure good focus, especially without blur at important anatomical structures. Focus mainly refers to that the focus of the image should be on key anatomical structures, such as the optic disc, macular area, etc. The specific standards are: the optic disc area should be able to show clear boundaries and central depressions; the fovea of ​​the macular area should be free of blur to ensure that any lesions (such as holes or hemorrhages) can be clearly distinguished; the vascular system should be able to show sharp edges to distinguish subtle changes in the vascular wall. If important structures are blurred, the image will be considered to be poorly focused and unqualified, and unqualified images will be discarded; the screening criteria for lesion masking is that if the lesion or other factors such as eyelids and eyelashes cover important areas in the image, the image will also be judged as unqualified. When factors such as lesions, eyelids or eyelashes cover key areas in the image, the image will usually be judged as unqualified. However, if the lesion itself is an important feature required for diagnosis, annotation engineers and doctors will evaluate whether the coverage of the lesion affects the visibility of other key structures and discard unqualified images.

[0041] Then, senior doctors (generally doctors with more than 10 years of clinical experience are called senior doctors) will further review the screening results to ensure the consistency of the screening film selection criteria and the quality of the images, so as to better support accurate clinical judgment.

[0042] Labeling subunit: It is used to label various lesions and diseases (1 for disease presence and 0 for no disease) on qualified images screened out in the image quality screening subunit. Generally, it is labeled by two or more middle-aged doctors (doctors with 5 to 10 years of clinical experience are usually called middle-aged doctors) to ensure the accuracy of the results. In specific implementation, it can also be labeled by multiple middle-aged doctors, and then reviewed by expert doctors (expert doctors usually refer to doctors with profound knowledge, rich experience and high technical level in a certain medical field). If the labeling results of the middle-aged doctors are inconsistent, the opinions of the reviewing expert doctors shall prevail.

[0043] Modeling unit: Use deep learning models to learn and train the ultra-wide-angle fundus images in the basic database of the basic database establishment unit. The training objects include the normal standard database and the database of various lesions and diseases, and establish an artificial intelligence primary model; In this embodiment, the training model includes three components: an input layer, a hidden layer, and an output layer. The input layer is used to receive the input of the ultra-wide-angle fundus image and pass it to the hidden layer, and finally pass the signal to the output layer to finally obtain the prediction result of the lesion or disease type; Ultra-wide-angle fundus image partitioning unit: used to divide the ultra-wide-angle fundus image into regions according to the structural characteristics of the retina. Based on the conventional definition of ophthalmology and clinical experience as the segmentation standard, the ultra-wide-angle fundus image is divided into three regions, among which, the fovea is taken as the center of the circle, the distance from the center of the optic disc to the fovea is taken as one unit (disc-to-macular distance, expressed in dm), the macular area is a circle with a radius of 0.5dm and the posterior pole area is a circle with a radius of 1.5dm and the part outside the posterior pole area is the peripheral area. Among them, Macular area: This is the central area of ​​the retina, usually located near the optic nerve head. The segmentation of this area is based on the geometric position of the macula, usually a certain distance from the center of the optic disc, with the fovea as the center and a radius of 0.5 times the distance from the optic disc to the center of the macula (disc-to-macular distance, dm), which is defined as the macular area.

[0044] The posterior pole: The distance from the optic disc to the center of the macula is taken as the unit, with a radius of 1.5 times dm. It covers the macular area and part of the optic disc area and is regarded as the central area of ​​the retina.

[0045] Peripheral area: All areas outside the posterior pole are defined as the peripheral area, which is the key area for detecting peripheral retinal lesions (such as peripheral retinal detachment and other lesions).

[0046] Optic disc area: This is the area where the optic nerve head is located. It is an anatomical structure where the retinal nerve fiber layer converges into the optic nerve in the retina, usually near the macula. The optic disc area is divided separately, and the optic disc boundary and optic cup boundary are divided to separate the optic disc (the optic disc is usually located in the central and inner area of ​​the fundus, where the optic nerve converges. Through fundus examination, the optic disc appears as a round or oval area with a lighter color and clear edges) and the optic cup (the optic cup is located in the center of the optic disc and is a concave area in the optic disc. It usually looks slightly darker than the surrounding optic disc part and appears as a shallower concave. The shape of the optic cup is usually round or oval, and the size and depth can vary. The difference between the optic disc and the optic cup is that the optic disc is a relatively flat area, while the optic cup is the concave part in the center), and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned.

[0047] Regional auxiliary lesion detection unit: used to perform auxiliary lesion detection on each segmented area. For example: Auxiliary analysis of macular lesions: used to assist in the identification and detection of the following macular lesions, such as: macular hole, macular hemorrhage, macular exudation, macular atrophy, macular edema, epiretinal membrane, etc.

[0048] Auxiliary analysis of posterior pole lesions: used to assist in the identification of the following lesions, such as: posterior pole retinal detachment, hemorrhage, exudation, atrophy, laser spots, leopard-shaped fundus, drusen, cataracts, retinal artery occlusion, high myopia fundus lesions, wet and age-related maculopathy, branch retinal vein occlusion, retinal artery occlusion, hypertensive retinopathy, and identification of peridiscal retinal nerve fiber layer and defects.

[0049] Auxiliary analysis of peripheral lesions: used to assist in the identification of lesions in the retinal periphery, and to assist in the analysis of the following lesions: peripheral retinal detachment, peripheral holes, peripheral degeneration, peripheral hemorrhage, peripheral laser spots, and peripheral exudation.

[0050] The optic disc area is used to quantify parameters such as the cup-to-disc ratio and color identification (such as pale, slightly pale, and red), and to assist in the analysis of optic nerve diseases, optic atrophy, and glaucoma.

[0051] The segmented images can be used for regional quantitative analysis to calculate parameters such as the lesion area and lesion location distribution in different regions.

[0052] Whole-image disease auxiliary detection unit: used to further assist in the analysis of disease information of the fundus image after completing the auxiliary detection of lesions in each partition, to assist in the detection of the following eye diseases, to provide auxiliary information for doctors, and to assess the risk probability of the corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, high myopia fundus lesions, retinitis pigmentosa, retinopathy of prematurity, retinal detachment, retinal holes, macular holes, wet age-related maculopathy, hypertensive retinopathy, retinal artery occlusion, leopard-shaped fundus, fibrous proliferation, retinal hemorrhage, laser spots, drusen, atrophy, etc.

[0053] Lesion auxiliary verification unit: used to assist in the verification of the following diseases, Auxiliary verification of macular lesions: Calculate the sensitivity, specificity, accuracy and other indicators of the auxiliary detected macular holes, hemorrhages, exudates, atrophy lesions and image annotations.

[0054] Auxiliary verification of lesions in the posterior pole area: For lesions in the posterior pole area (such as retinal detachment, hemorrhage, etc.) that are assisted in detection, auxiliary verification is performed through image annotation to ensure the accuracy of the model.

[0055] Auxiliary verification of peripheral area lesions: For lesions in the peripheral area that are assisted in detection (such as peripheral retinal detachment, etc.), auxiliary verification is performed through the annotation results of the peripheral area images.

[0056] Whole-image disease auxiliary verification unit: It is used to assist in the detection of multiple diseases in the entire image (such as diabetic retinopathy, retinal detachment, etc.). The model output is calculated with the doctor's annotation results, and the comprehensive performance of the evaluation model is evaluated and verified using sensitivity, specificity, accuracy, AUC, etc.

[0057] In this embodiment, the model validation method uses an independent test set (independent of the training set and validation set) to further validate the generalization ability of the model and evaluate its stability and accuracy in actual clinical applications. The independent test set is an independent test set that does not overlap with the training set to evaluate the actual effect of the model. The independent test data set comes from multiple ophthalmology hospitals to ensure data diversity.

[0058] The indicator calculation is to calculate the output results of the model with the annotation results, and verify the clinical practicality of the model through indicators such as sensitivity, specificity, and ROC curve.

[0059] After testing, taking diabetic retinopathy as an example, the sensitivity was 96.0%, the specificity was 96.0%, and the AUC was 0.9915. The test results showed that the diabetic retinopathy classification model was effective.

[0060] The present invention realizes efficient and accurate automated fundus disease analysis and evaluation through regional partitioning and lesion analysis of ultra-wide-angle fundus images, especially in the detection of lesions in the peripheral area of ​​the retina. The system is based on a deep learning algorithm model and can be applied to the screening and auxiliary diagnosis of clinical ophthalmic diseases. With the rapid development of deep learning, convolutional neural networks (CNNs) have achieved remarkable results in the field of image classification. The present invention provides an automated and accurate fundus image-assisted diagnosis method based on deep learning models to partition and analyze common fundus diseases and lesions in the entire ultra-wide-angle fundus images, as well as lesion analysis and evaluation of optic nerve diseases including glaucoma.

Claims

1. A method for analyzing ultra-wide-angle fundus images, characterized in that: The method comprises the following steps: Step S1, establishing a basic database: an imaging device collects high-resolution ultra-wide-angle fundus images of the retina to establish a basic database; Step S2, modeling: using a deep learning model to train the ultra-wide-angle fundus images in the basic database of step S1, generate model weights, and the training objects include a normal standard database and various lesion disease databases to establish an artificial intelligence primary model; Step S3, ultra-wide-angle fundus image partitioning: the high-resolution ultra-wide-angle fundus image of the retina acquired by the imaging device in step S1 is divided into regions according to the structural characteristics of the retina, wherein the fovea is taken as the center of the circle, the distance from the center of the optic disc to the fovea is taken as a unit dm, the macular area is a circle with a radius of 0.5 dm and the posterior pole area is a circle with a radius of 1.5 dm and the part outside the posterior pole area is the peripheral area; the optic disc area is divided separately, the optic disc boundary and the optic cup boundary are divided to separate the optic disc and the optic cup, and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned; Step S4, regional auxiliary lesion detection: each segmented region will be subjected to auxiliary lesion detection, such as macular hole, epiretinal membrane, macular edema, retinal peripheral degeneration, atrophy, exudation, retinal hemorrhage, laser spot, and optic nerve disease; Step S5, whole-image disease-aided detection: after completing the lesion-aided detection of each partition, further perform the whole-image assisted analysis of the disease information of the fundus image, assist in the detection of the following eye diseases, provide auxiliary reference information for doctors, and assess the risk probability of the corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, retinal artery occlusion, high myopia fundus lesions, retinitis pigmentosa, wet age-related macular degeneration, hypertensive retinopathy, retinopathy of prematurity, retinal detachment, retinal tears, leopard-shaped fundus, fibrosis, cataracts and drusen; Step S6, lesion auxiliary verification: the model output result and the image annotation are calculated to achieve auxiliary verification; Step S7, auxiliary verification of diseases in the whole image: The auxiliary detection results of multiple diseases in the whole image are calculated through the output of the model and the doctor's annotation results to provide reference information for the doctor's diagnosis.

2. The ultra-wide-angle fundus image analysis method according to claim 1, characterized in that: The step S1, when establishing the basic database, includes the following sub-steps: Step S1-1, image quality screening: During the image quality screening process, the quality of the collected ultra-wide-angle fundus images is evaluated by manual annotation. Then, the doctor further reviews the screening results to ensure the consistency of the screening criteria and the quality of the images. Step S1-2, labeling: labeling various lesions and disease types for the qualified images screened out in step S1-1.

3. The ultra-wide-angle fundus image analysis method according to claim 1, characterized in that: In step S7, the auxiliary verification of the whole image disease type is to use sensitivity, specificity, accuracy and AUC to evaluate the comprehensive performance of the model.

4. An ultra-wide-angle fundus image analysis system, characterized in that: The system comprises the following module units: Basic database establishment unit: used to establish a basic database by collecting high-resolution ultra-wide-angle fundus images of the retina through imaging equipment; Modeling unit: used to use a deep learning model to learn and train the ultra-wide-angle fundus images in the basic database of the basic database establishment unit, generate model weights, and the training objects include a normal standard database and various lesion disease databases to establish an artificial intelligence primary model; Ultra-wide-angle fundus image partitioning unit: used to divide the ultra-wide-angle fundus image into regions according to the structural characteristics of the retina, wherein the fovea is taken as the center of the circle, the distance from the center of the optic disc to the fovea is a unit of dm, the macular area is a circle with a radius of 0.5 dm and the posterior pole area is a circle with a radius of 1.5 dm and the part outside the posterior pole area is the peripheral area; the optic disc area is divided separately, and the optic disc boundary and the optic cup boundary are divided to separate the optic disc and the optic cup, and the optic disc and the optic cup are distinguished by light red, slightly pale and pale colors, and the retinal nerve fiber layer and defects around the optic disc are trained and learned; Regional auxiliary lesion detection unit: used to perform auxiliary lesion detection on each segmented area, such as macular hole, epiretinal membrane, macular edema, peripheral retinal degeneration, atrophy, exudation, retinal hemorrhage, laser spot, and optic nerve disease; Whole-image disease auxiliary detection unit: used to further assist in the analysis of disease information of fundus images after completing auxiliary detection of lesions in each partition, assist in detecting the following eye diseases, provide auxiliary reference information for doctors, and evaluate the risk probability of corresponding diseases, such as: diabetic retinopathy, retinal vein occlusion, retinal artery occlusion, retinal artery occlusion, high myopia fundus lesions, retinitis pigmentosa, wet age-related macular degeneration, retinopathy of prematurity, hypertensive retinopathy, retinal detachment, retinal tears, leopard-shaped fundus, fibrosis, cataracts and drusen; Lesion auxiliary verification unit: used to calculate the model output results and image annotations to assist in disease verification; Whole-image disease auxiliary verification unit: used to calculate the auxiliary detection results of multiple diseases in the whole image through the output of the model and the doctor's annotation results.

5. The ultra-wide-angle fundus image analysis system according to claim 4, characterized in that: The basic database establishment unit includes the following subunits: Image quality screening subunit: used to evaluate the quality of the collected ultra-wide-angle fundus images in the image quality screening process, usually by manual annotation; Then, the doctor further reviews the screening results to ensure the consistency of the screening criteria and the quality of the images; Labeling subunit: used to label various lesions and diseases on qualified images screened out in the image quality screening subunit.

6. The ultra-wide-angle fundus image analysis system according to claim 4, characterized in that: The whole-image disease auxiliary verification unit uses the comprehensive performance of the sensitivity, specificity, accuracy and AUC evaluation model for evaluation and verification.

Images