Method and system for identifying and tracking features of fundus images

Abstract

To provide a system and method for enabling individuals with retinal lesion to monitor fundus images at home, the system being applicable to handheld mobile devices.SOLUTION: A method for identifying and tracking features of fundus images, includes: tracking and aiming at a first reference coordinate of a fundus of an individual using an optic disc cupping and retinal blood vessel target; tracking and aiming at a second reference coordinate of the fundus of the individual in cooperation with a foveal target, where the first reference coordinate is selected from an optic disc, a retinal vein branch, or a retinal artery branch of the fundus of the individual, and the second reference coordinate is a central fovea of the fundus of the individual; and acquiring a fundus image, where the fundus image is acquired when the first reference coordinate and the second reference coordinate are successfully tracked and aimed by a composite aiming point.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a method and system for recognizing and tracking features in fundus images, in particular to a method and system for recognizing and tracking features in time-lapse fundus images of an individual, and more particularly to a method and system for recognizing and tracking features in time-lapse fundus images for portable and home use for patients with retinal pathology. [Background technology]

[0002] Causes of retinal lesions include high myopia, triple risk factors (hypertension, hyperglycemia, and hyperlipidemia), and diabetes. Some populations also suffer from macular degeneration as a genetic disease. Retinal capillaries are prone to lesions, resulting in retinal hemorrhage, edema, or necrosis, impairing visual function. Diabetic retinopathy in diabetic patients may exhibit no symptoms in the early stages. Generally, fundus lesions begin to appear in 30% of patients 10 years or more after the onset of diabetes, and the incidence rate reaches 80% in patients with a 25-year history of the disease. Diabetic retinopathy primarily develops when a prolonged hyperglycemic environment damages the endothelium of retinal blood vessels, leading to a series of fundus lesions. Diabetic retinopathy may not exhibit any symptoms in the early stages. However, poor blood glucose control, insulin-dependent diabetes, hypertension, hyperlipidemia, and renal involvement can all accelerate the onset of diabetic retinopathy and further accelerate the development of fundus lesions. It is currently known that almost all patients with type 1 diabetes develop retinal lesions after 15 to 20 years, with 20 to 30% of these patients going on to become blind. In addition, more than 60% of patients with type 2 diabetes develop retinal lesions, which are the leading cause of blindness in the 20 to 65 year old population.

[0003] For example, in the diagnosis of conventional retinal pathologies, such as diabetic retinopathy, fundus examination is essential to observe changes in the retina at the fundus. Generally, in conventional fundus examinations, the pupil is first dilated using a mydriatic, and then the presence or absence of vascular proliferation or other lesions in the fundus must be examined using an ophthalmoscope and a slit lamp. However, this method has drawbacks, such as being somewhat time-consuming, relatively difficult to record the examination results, and not allowing for objective comparison of the condition before and after treatment. Therefore, patients generally find it difficult to accept these procedures, resulting in a low examination rate.

[0004] In view of the above, how to develop a device that allows medical personnel to examine or diagnose patients with retinal diseases, or that can be easily operated by patients with retinal diseases in a home environment without causing discomfort during the operation, that can easily recognize and track the features of fundus images on a regular basis for examination, that has high-definition standardized retinal images, that improves retinal image quality, that makes it easy to create image files, and that can recognize changes in images at different points in time after a certain period of time has passed, thereby solving the shortcomings of the prior art and ensuring the accuracy of diagnosis, is currently a pressing issue that needs to be resolved by those skilled in the art. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, the object of the present invention is to further improve upon the problems of the prior art by providing a device that can be operated more easily by medical professionals or others in collaboration than by the prior art, or that can be easily operated by an individual on his or her own, that allows the user to easily recognize, track, and examine the characteristics of fundus images of both eyes at intervals of a certain period of time, and that is convenient for easily creating standardized high-definition fundus images and effectively comparing them, thereby improving convenience for the user.

[0006] Another object of the present invention is to provide a method for early warning and monitoring of microbleeds, hard exudates, petechiae, neovascularization, cotton wool spots, etc. in the fundus of an individual, thereby effectively solving the problems of the prior art. [Means for solving the problem]

[0007] The method for recognizing and tracking features in a fundus image according to the present invention includes the following steps: moving an image capture unit equipped with a composite aim point to a first examination position, which allows the fundus of an individual to be positioned within the focal length range of the image capture unit; the composite aim point further includes an optic disc cup retinal vascular target and a fovea target; tracking and aiming a first reference coordinate of the fundus of the individual using the optic disc cup retinal vascular target, and tracking and aiming a second reference coordinate of the fundus of the individual using the fovea target in cooperation with the optic disc cup retinal vascular target; the first reference coordinate is selected from the optic disc, branch retinal vein, or branch retinal artery of the fundus of the individual, and the second reference coordinate is the fovea of the fundus of the individual; and then acquiring a fundus image. The fundus image is obtained when the first reference coordinate and the second reference coordinate are successfully tracked and aimed by the composite aim point.

[0008] In one embodiment of the present invention, the optic disc further comprises an optic cup.

[0009] In one embodiment of the present invention, the branch retinal vein is selected from the superior lateral vein (venula temporalis retinae superior), the superior medial vein (venula nasalis retinae superior), the inferior medial vein (venula nasalis retinae inferior), and the branch retinal artery is selected from the superior lateral artery (arteriola temporalis retinae superior), the superior medial artery (arteriola nasalis retinae superior), the inferior medial artery (arteriola nasalis retinae inferior), and the inferior lateral artery (arteriola temporalis retinae inferior).

[0010] In one embodiment of the present invention, the optic disc cup retinal vascular target and the foveal target are ring-shaped, cross-ring-shaped, oval-shaped, triangular, inverted triangle, square, diamond-shaped, trapezoidal, inverted trapezoidal, hexagonal, octagonal, or X-shaped.

[0011] In one embodiment of the present invention, the optic disc cup retinal vessel target further comprises a circular target sight and a bidirectional fork sight, the bidirectional fork sights being radially arranged from the circular target sight in a generally X-shape.

[0012] In one embodiment of the present invention, the circular target aimer is capable of tracking and aiming at the optic disc cup.

[0013] In one embodiment of the present invention, the forward / reverse fork sight further includes four branches for tracking and targeting the superior lateral vein or artery, the superior medial vein or artery, the inferior medial vein or artery, and the inferior lateral vein or artery, respectively.

[0014] In one embodiment of the present invention, if tracking and aiming of the first and second reference coordinates using the composite aim point fails, an error message is sent. Then, the focal length relative to the fundus is evaluated by calculating a difference value with the fundus image, and the image acquisition unit is then moved to a second test position. The second test position is an appropriate focal plane position. The following steps are repeated: the image acquisition unit provided with the composite aim point is moved to a first test position. The first test position allows the fundus of the individual to be positioned within the focal length range of the image acquisition unit. The composite aim point further includes an optic disc cup retinal vascular target and a foveal target. The optic disc cup retinal vascular target is used to track and aim the first reference coordinate of the fundus of the individual, and the foveal target is used to track and aim the second reference coordinate of the fundus of the individual in cooperation with the optic disc cup retinal vascular target. This allows another fundus image to be acquired.

[0015] In one embodiment of the present invention, fundus images acquired at different time periods are compared, in particular images of the optic disc, retinal vein or artery branches and fovea of the fundus of an individual.

[0016] In one embodiment of the present invention, when acquiring the fundus image, an aiming beam is first aimed at the individual's eye to assist positioning and aiming, thereby enabling the orientation of the individual's eyeball to be standardized.

[0017] In one embodiment of the present invention, a fundus boundary is defined based on the contrast ratio of the fundus image, and a fundus section obtained from the fundus image is selected by the composite aim point with a circle. Then, a feature identification and comparison unit calculates whether the fundus section and the fundus image previously created at the composite aim point meet a reference value of overlapping area, which is at least 72%.

[0018] The fundus image feature recognition and tracking system of the present invention includes an image acquisition unit. The image acquisition unit is provided with a composite aim point. The composite aim point further includes an optic disc cup retinal vascular target and a fovea target. The image acquisition unit moves to a first examination position. The first examination position allows the fundus of the individual to be located within the focal length range of the image acquisition unit. The optic disc cup retinal vascular target tracks and aims at a first reference coordinate of the fundus of the individual, and the fovea target cooperates with the first reference coordinate of the fundus of the individual. The first reference coordinate is selected from the optic disc, retinal vein branch, or retinal artery branch of the fundus of the individual, and the second reference coordinate is the fovea of the fundus of the individual. A fundus image is acquired when the composite aim point successfully tracks and aims at the first and second reference coordinates.

[0019] In one embodiment of the present invention, the optic disc cup retinal vascular target further includes a circular target sight and a forward and reverse fork sight. The forward and reverse fork sights are arranged radially from the circular target sight in a generally X-shape. The circular target sight tracks and aims at the optic disc cup. The forward and reverse fork sight further includes four branches. These branches track and aim at the superior lateral vein or superior lateral artery, the superior medial vein or superior medial artery, the inferior medial vein or inferior medial artery, and the inferior lateral vein or inferior lateral artery, respectively.

[0020] In one embodiment of the present invention, the apparatus further includes a feature identification and comparison unit, which is used to identify and compare the optic disc, retinal vein branches or retinal artery branches, and fovea in the fundus images acquired at different time periods.

[0021] In one embodiment of the present invention, the system further includes a fundus image database for storing fundus images acquired at different time periods.

[0022] In one embodiment of the present invention, the device further includes a display unit electrically connected to the image acquisition unit and the fundus image database, to which fundus images are transmitted and displayed from the image acquisition unit, or to which fundus images are transmitted and displayed from the fundus image database.

[0023] In order to make the above features and advantages of the present invention clearer and more concise, the present invention will be described in detail below with reference to the accompanying drawings and examples. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a schematic diagram of the operation according to the method of the present invention. [Figure 2] FIG. 2 is a schematic diagram of the operation of another method of the present invention. [Figure 3] FIG. 3 is a schematic diagram of the operations in a further method of the invention. [Figure 4] FIG. 4 is a schematic diagram of the operations in the tracking and targeting method of the present invention. [Figure 5] FIG. 5 is a schematic diagram of the operation process of another tracking and aiming method of the present invention. [Figure 6] FIG. 6 is a schematic diagram showing a case where the optic disc cup retinal vascular target and the foveal target in the present invention work together to track and aim at the first and second reference coordinates on the fundus, and targeting is successful. [Figure 7] FIG. 7 is a schematic diagram illustrating a case where the first and second reference coordinates are tracked and aimed at, but the targeting fails, in another embodiment of the present invention. [Figure 8] FIG. 8 is a schematic block diagram of a system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0025] The technical contents of the present invention will be described below through specific specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. However, the present invention may be implemented or applied in a variety of different forms without departing from the spirit of the present invention.

[0026] The term "one embodiment" of the present invention described in the specification is used to describe and indicate that a particular function, structure, or feature is included in the present invention. The term "one embodiment" as used in the specification does not necessarily refer to all the same embodiment, nor is it an individual or alternative embodiment that excludes other embodiments. That is, some embodiments may describe some specific features that are not described in other embodiments. In addition, structures, components, or connections that are well-known in the relevant field are not described in particular detail to avoid obscuring the novel and unique features of the present invention.

[0027] In one embodiment of the present invention, the term "individual" as used herein refers to a user or operator, such as a patient with a retinal disease, particularly a patient with diabetic retinopathy. In another embodiment, the present invention may be operated by a medical professional, by a third party in a home environment with the assistance of the patient with a retinal disease, or by the patient themselves in a home environment. Alternatively, the individual may recognize and track fundus image features for one of their two eyes, acquire and review a fundus image, and then switch to acquire and review a fundus image for the other eye.

[0028] 1 to 8 , in one embodiment of the method for recognizing and tracking features in a fundus image of the present invention, the image acquisition unit 1 is moved to an examination position to acquire an image of the fundus of the individual. The examination position is located in front of the individual's eye. At the same time, an aiming beam 51 emitted from a light source 5 may be first aimed at the individual to assist positioning and aiming. The light source 5 may be installed in a handheld mobile device 101. In other words, the individual's eye E may look directly at a target light emitted from the image acquisition unit 1 (e.g., a lens) of the handheld mobile device 101 (i.e., a portable smart device or a mobile phone) of the present invention. This ensures that the individual's eye can look directly at the target light, thereby standardizing the direction of the individual's eyeball. At the same time, an image of the individual's eye and an image of the target light may be transmitted to a display unit 3. In other words, the target light can standardize the direction of the individual's eyeball on the display unit 3.

[0029] The present invention also provides a method for recognizing and tracking features in a fundus image. The method includes the following steps: moving an image capture unit 1 equipped with a composite aim point to a first examination position, which allows the fundus of an individual to be located within the focal length range of the image capture unit 1. The composite aim point further includes an optic disc cup retinal vascular target T1a and a fovea target T2a. The optic disc cup retinal vascular target T1a is used to track and aim a first reference coordinate R1 of the fundus of the individual, and the fovea target T2a is used to track and aim a second reference coordinate R2 of the fundus of the individual. The first reference coordinate R1 is selected from the optic disc, retinal vein branch, or retinal artery branch of the fundus of the individual, and the second reference coordinate R2 is the fovea. The optic disc, retinal vein branch, retinal artery branch, and fovea can be considered to be three coplanar points on the fundus. The tracking and aiming can be automatic, semi-automatic, or manual. The optic disc further includes an optic disc cup. The retinal vein branch is selected from the superior lateral vein, superior medial vein, inferior medial vein, and inferior lateral vein, and the retinal artery branch is selected from the superior lateral artery, superior medial artery, inferior medial artery, and inferior lateral artery. The first reference coordinate R1 is closer to the nasal side than the second reference coordinate R2. In other words, the second reference coordinate R2 is closer to the temporal side than the first reference coordinate R1. Next, a fundus image is acquired. This fundus image is obtained when the first reference coordinate R1 and the second reference coordinate R2 are successfully tracked and aimed using the composite aim point.

[0030] In one embodiment of the present invention, the optic disc cup retinal vascular target T1a and the fovea target T2a may be, but are not limited to, a ring shape, a cross ring shape, an ellipse, a triangle, an inverted triangle, a square, a diamond, a trapezoid, an inverted trapezoid, a hexagon, an octagon, or an X shape. For example, in the case of an X-shaped optic disc cup retinal vascular target T1b, the X-shaped optic disc cup retinal vascular target T1b is further provided with a circular target sight T11 and a forward and reverse fork-shaped sight T12.

[0031] In one embodiment of the present invention, the circular target sight T11 tracks and aims at the optic disc cup. The bifurcated fork sight T12 further includes four branches arranged radially from the circular target sight in a generally X-shape. The four branches track and aim at retinal vein branches or retinal artery branches. In other words, the first of the four branches, the first branch, can track and aim at the superior lateral vein or superior lateral artery, the second branch, the second branch, can track and aim at the superior medial vein or superior medial artery, the third branch, the third branch, the third branch, the third branch, the third branch, the fourth ...

[0032] In another embodiment of the present invention, if the tracking and aiming of the first and second reference coordinates R1 and R2 using the composite aiming point fails, an error message is sent. Then, the focal length relative to the fundus is evaluated by calculating the difference value with the fundus image, and the image acquisition unit is moved to a second examination position, which is a suitable focal plane position. Then, the above steps are repeated to acquire another fundus image.

[0033] In a further embodiment of the present invention, fundus images acquired at different time periods can be compared, particularly images of the optic disc, retinal vein branches or retinal artery branches, and fovea of the fundus of an individual. In this way, by comparing fundus images acquired at different time periods, medical professionals or patients can easily determine whether changes have occurred in the blood vessels of the fundus at different time periods using a simple method or simple operation procedures, without the need for expensive medical examination equipment or complicated operation procedures.

[0034] The present invention also provides a system 100 for recognizing and tracking features in fundus images. For example, the system 100 is a simple, convenient, and effective handheld mobile system that allows medical personnel, other people, or patients with retinal disease to recognize and track features in fundus images anywhere, such as at a medical institution or at home. The system 100 is applicable to a handheld mobile device 101, which may include, but is not limited to, a smart mobile device. The system 100 includes at least an image acquisition unit 1, a fundus image database 2, a display unit 3, and a feature identification and comparison unit 4. The feature identification and comparison unit 4 is used to identify and compare the optic disc, retinal vein branches, retinal artery branches, and fovea in fundus images acquired at different time periods. The fundus image database 2 is used to store fundus images acquired at different time periods. In other words, the display unit 3 is electrically connectable to the image acquisition unit 1 and the fundus image database 2, and is capable of transmitting and displaying fundus images from the image acquisition unit 1 or from the fundus image database 2. The image acquisition unit 1, fundus image database 2, display unit 3 and feature identification and comparison unit 4 are mounted on a handheld mobile device and can be electrically connected to each other.

[0035] Of course, the present invention is not limited to the above. A fundus boundary is defined based on the contrast ratio of the fundus image, and a fundus section obtained from the fundus image is circled using a ring-shaped aim point C provided on the image acquisition unit 1. The feature identification and comparison unit 4 then calculates whether the fundus section and a fundus image previously captured using the ring-shaped aim point C meet a reference overlap area. The reference overlap area is at least 72%. If the reference overlap area does not meet the reference overlap area, an error message can be sent from the handheld mobile device. For example, the display unit 3 can display an error message, such as an image error message, or the handheld mobile device 101 can send an audio error message, but this is not limiting. In other words, the present invention also provides an auxiliary method for defining a fundus boundary based on the contrast ratio of the fundus image, and selecting a fundus section using a circle. Then, the fundus section is compared with a fundus image previously captured using the ring-shaped aim point C to determine whether the reference overlap area meets the reference overlap area. In other words, the coverage rate of the ring-shaped aim point C can indicate whether or not there is a deviation in the aim point when the fundus image is acquired, and also indicates whether or not an appropriate focal plane exists within the distance from the image acquisition unit 1 to the fundus of the individual. [Example]

[0036] See Figures 5 and 6. Using the optic disc cup retinal vascular target T1b and the foveal target T2b, the first reference coordinate R1 and the second reference coordinate R2 were tracked and aimed in tandem. In this case, tracking and aiming were successful, targeting was completed, and a fundus image was acquired. [Example]

[0037] See Figure 7. Using the optic disc cup retinal vascular target T1b and the foveal target T2b, the first reference coordinate R1 and the second reference coordinate R2 were tracked and aimed in coordination. However, tracking and aiming failed, and targeting could not be completed, so a fundus image could not be effectively acquired. Note that the tracking and aiming could be automatic, semi-automatic, or manual.

[0038] The technical contents of the present invention have been described above through certain specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed herein. However, the present invention may be embodied or applied in a variety of different forms without departing from the spirit of the present invention. [Explanation of symbols]

[0039] 100 Fundus Image Feature Recognition and Tracking System 101 Handheld Mobile Devices 1 Image Acquisition Unit 2 Fundus image database 3 Display Unit 4. Feature discrimination and comparison unit 5 light source 51 Rays of light C-ring-shaped aiming point E eyes R1 First reference coordinate R2 2nd reference coordinate T1a optic disc retinal vascular target T1b optic disc retinal vascular target T11 Circular Target Sight T12 Fork Sight T2a foveal target T2b foveal target

Claims

1. A method for recognizing and tracking features in a fundus image, comprising: moving an image capture unit provided with a composite aim point to a first examination position, the first examination position allowing the fundus of the individual to be positioned within a focal length section of the image capture unit, the composite aim point further comprising an optic disc cup retinal vascular target and a fovea target; The optic disc cup retinal vascular target is used to track and aim at a first reference coordinate on the fundus of the individual, and the foveal target is used to track and aim at a second reference coordinate on the fundus of the individual, wherein the first reference coordinate is selected from the optic disc, a retinal vein branch, or a retinal artery branch on the fundus of the individual, and the second reference coordinate is the fovea of the fundus of the individual; acquiring a fundus image, the fundus image being obtained when the first reference coordinate and the second reference coordinate are successfully tracked and aimed at by the composite aim point; and a method comprising the steps of:

2. The method of claim 1 , wherein the optic disc further comprises an optic disc cup.

3. 3. The method of claim 2, wherein the retinal vein branch is selected from the superior lateral vein, superior medial vein, inferior medial vein, and inferior lateral vein, and the retinal artery branch is selected from the superior lateral artery, superior medial artery, inferior medial artery, and inferior lateral artery.

4. 4. The method of claim 3, wherein the optic disc cup retinal vascular target and the foveal target are ring-shaped, cross-ring-shaped, elliptical, triangular, inverted triangle, square, diamond, trapezoid, inverted trapezoid, hexagon, octagon, or X-shaped.

5. 5. The method of claim 4, wherein the optic disc cup retinal vascular target further comprises a circular target sight and a forward and reverse fork sight, the forward and reverse fork sights being arranged radially from the circular target sight in a generally X-shape.

6. The method of claim 5 , wherein the circular target aimer tracks and aims at the optic disc cup.

7. 7. The method of claim 6, wherein the forward and reverse fork-shaped sight further includes four branches, which track and aim at the superior lateral vein or artery, the superior medial vein or artery, the inferior medial vein or artery, and the inferior lateral vein or artery, respectively.

8. The method of claim 1, wherein if the tracking and aiming of the first reference coordinate and the second reference coordinate using the composite aiming point fails, an error message is sent, and the focal length to the fundus is evaluated by calculating a difference value with the fundus image, and then the image acquisition unit is moved to a second examination position, which is an appropriate focal plane position, and the steps described in claim 1 are repeated to acquire another fundus image.

9. 9. The method according to claim 8, wherein the fundus images acquired at different time periods are compared, in particular images of the optic disc, retinal vein or artery branches and fovea of the fundus of the individual.

10. 10. The method of claim 9, wherein the fundus image is first acquired by aiming a light beam at the individual's eye to assist in positioning and aiming, thereby enabling standardization of the orientation of the individual's eyeball.

11. The method of claim 10, further comprising: defining a fundus boundary obtained from the contrast ratio of the fundus image; selecting a fundus section obtained from the fundus image by a circle using the composite aim point; and calculating whether the fundus section and the fundus image previously created at the composite aim point meet a reference value of overlap area using a feature identification and comparison unit; and the reference value of overlap area is at least 72%.

12. A system for recognizing and tracking features in fundus images, comprising: an image acquisition unit, the image acquisition unit being provided with a composite aim point, the composite aim point further being provided with an optic disc cup retinal vascular target and a fovea target, the image acquisition unit being moved to a first examination position, the first examination position being capable of positioning the fundus of the individual within a focal length section of the image acquisition unit, the optic disc cup retinal vascular target tracking and aiming at a first reference coordinate of the fundus of the individual, the fovea target being linked to track and aiming at a second reference coordinate of the fundus of the individual, the first reference coordinate being selected from the optic disc, a retinal vein branch or a retinal artery branch of the fundus of the individual, the second reference coordinate being the fovea of the fundus of the individual; A system in which a fundus image is acquired when the first reference coordinate and the second reference coordinate are successfully tracked and aimed by the compound aim point.

13. 13. The system of claim 12, wherein the optic disc cup retinal vascular target further includes a circular target aimer and a forward and reverse fork aimer, the forward and reverse fork aimer being arranged radially from the circular target aimer in a generally X-shape, the circular target aimer tracking and aiming at the optic disc cup, and the forward and reverse fork aimer further including four branches, which track and aim at the superior lateral vein or superior lateral artery, the superior medial vein or superior medial artery, the inferior medial vein or inferior medial artery, and the inferior lateral vein or inferior lateral artery, respectively.

14. The system of claim 12 further comprising a feature identification and comparison unit, which is used to identify and compare the optic disc, retinal vein branches or retinal artery branches, and fovea in the fundus images acquired at different periods.

15. 15. The system of claim 14, further comprising a fundus image database for storing said fundus images acquired at different time periods.

16. The system of claim 15 further comprising a display unit electrically connected to the image acquisition unit and the fundus image database, to which the fundus image is transmitted and displayed from the image acquisition unit or to which the fundus image is transmitted and displayed from the fundus image database.

Images