Optic nerve cup image processing system and method
Through deep learning models, the problem of difficult labeling of optic nerve cup contours is solved, and faster labeling and more efficient risk assessment are achieved.
Patent Information
- Application Number
- CN202410427289.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-01-22
- Filing Date
- 2024-04-10
- Publication Date
- 2025-07-22
AI Technical Summary
It is difficult to label the contour of the optic nerve cup in the fundus map, especially in highly myopic images, resulting in increased difficulty and longer labeling time when ophthalmologists interpret images.
The deep learning model is used to identify the optic nerve disc and the optic nerve cup in the fundus map through the image cutting model, and provides annotation function. Combined with the display device and the input device, doctors allow them to modify or accept the annotation to assist in the annotation of the optic nerve cup and disc profile.
The labeling process of optic nerve cup disc profile is simplified, reducing the doctor's labeling time and improving the efficiency of risk assessment and interpretation quality.
Smart Images
Figure CN120355639A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an image processing system and method, and particularly to an image processing system and method for the optic nerve cup and disc. Background Art
[0002] Fundus images (also known as ophthalmoscopic images) are common and easily captured medical images, and are often the first-line diagnostic tools for ophthalmologists to interpret signs for risk assessment of eye injuries. Among them, the relative size relationship between the optic nerve cup and the optic nerve disc included in the fundus image is an important sign of various eye injuries, such as glaucoma, optic neuritis, pseudotumor cerebri, Leber's hereditary optic neuropathy, etc.
[0003] However, during the risk assessment process, since the optic nerve cup is not an anatomical structure with obvious boundaries, even when labeled by an ophthalmologist, various factors affecting the contour of the optic nerve cup (such as: color change of the optic nerve disc, vascular turning change of the optic nerve disc... etc.) need to be considered to determine the contour of the optic nerve cup. Therefore, the labeling difficulty is much higher than that of general anatomical structures. Even when labeled by a skilled ophthalmologist, it takes a longer time (for example, more than 3 minutes) to perform the assessment and actual labeling. In addition, if the image belongs to a highly myopic subject, the edge of the optic nerve cup is more difficult to judge than that of a non-highly myopic subject, increasing the difficulty of the physician in interpreting the image and the time required for labeling the optic nerve cup and the optic nerve disc.
[0004] Therefore, for the interpretation of fundus images, adding an auxiliary labeling function for the contour of the optic nerve cup and disc is indeed expected to improve the interpretation quality and at the same time reduce the time for physicians to perform labeling and interpretation assessment. Summary of the Invention
[0005] In view of this, an embodiment of the present disclosure provides an optic nerve cup and disc image processing system, including: a processor that accesses a program to execute: receiving a fundus image, and identifying an optic nerve disc and an optic nerve cup in the fundus image through an image segmentation model, and labeling the contour of each of the optic nerve disc and the optic nerve cup; in an interpretation application mode, generating an interpretation data for risk assessment at least based on the contour of the optic nerve disc and the contour of the optic nerve cup. Wherein, the image segmentation model is a deep learning model trained using pre-collected fundus images as training data. Moreover, the pre-collected fundus image training data has the contours of the optic nerve disc and the optic nerve cup labeled by an ophthalmologist in each of the fundus image training data.
[0006] According to an embodiment of the present disclosure, the optic nerve cup-disc image processing system further includes a display device and an input device. Wherein, the processor performs: in a marking application mode, marking the contours of each of the above-mentioned optic nerve disc and the above-mentioned nerve cup with a plurality of contour points respectively, and displaying them on the above-mentioned display device; receiving, through the above-mentioned input device, instructions from the above-mentioned ophthalmologist to modify, clear, or accept the above-mentioned plurality of contour points.
[0007] Another embodiment of the present disclosure provides an optic nerve cup-disc image processing method, which is applicable to an electronic device. The above method includes: obtaining a fundus image by a processor in the above-mentioned electronic device, and identifying an optic nerve disc and an optic nerve cup in the above-mentioned fundus image through an image cutting model, and marking the contours of each of the above-mentioned optic nerve disc and the above-mentioned optic nerve cup; providing an interpretation application mode, and generating, by the above-mentioned processor, at least based on the contours of the above-mentioned optic nerve disc and the above-mentioned optic nerve cup, an interpretation data for risk assessment. Wherein, the above-mentioned image cutting model is a deep learning model trained using pre-collected fundus image training data, and the above-mentioned pre-collected fundus images are those in which ophthalmic professionals have marked the contours of the optic nerve disc and the optic nerve cup in each of the above-mentioned fundus image training data. Brief Description of the Drawings
[0008] Figure 1 It is a block diagram of an optic nerve cup-disc image processing system shown according to an embodiment of the present invention.
[0009] Figure 2 It shows a flowchart of an optic nerve cup-disc image processing method according to an embodiment of the present invention.
[0010] Figure 3 It shows the original fundus image and the fundus image with the contours of the marked optic nerve disc and optic nerve cup.
[0011] Among them, the reference numerals are explained as follows:
[0012] 100: Optic nerve cup-disc image processing system
[0013] 102: Processor
[0014] 104: Storage device
[0015] S201~S207: Steps
[0016] 310: Fundus image
[0017] 312: Optic nerve disc
[0018] 314: Optic nerve cup
[0019] 330: Vessel cutting map
[0020] IRM: Image Cutting Model Detailed Implementation Manner
[0021] The following description is a preferred implementation manner for implementing the invention, aiming to describe the basic spirit of the present invention, but not to limit the present invention. The actual content of the invention must refer to the subsequent claims.
[0022] It must be understood that words such as "comprising" and "including" used in this specification are used to indicate the existence of specific technical features, numerical values, method steps, operations, elements, and / or components, but do not exclude the addition of more technical features, numerical values, method steps, operations, elements, components, or any combination of the above. Words such as "first", "second", and "third" used in the claims are used to modify the elements in the claims, and do not indicate a priority order, precedence relationship, or that one element precedes another element, or the chronological order when performing method steps, but are only used to distinguish elements with the same name.
[0023] Figure 1 It is a block diagram of an optic nerve cup and disc image processing system (hereinafter simply referred to as the image processing system) 100 illustrated according to an embodiment of the present invention. In different embodiments, the image processing system 100 is, for example, an electronic device such as various computer devices and / or intelligent devices, but is not limited thereto. In this embodiment, the image processing system 100 is, for example, a notebook computer.
[0024] As Figure 1 shown, the image processing system 100 includes a processor 102 and a storage device 104. The storage device 104, for example, is any type of random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or other similar devices, or a combination of the above devices, and can be used to record multiple programs, models, or modules.
[0025] The processor 102 is coupled to a storage device 104, such as a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array circuit (FPGA), any other kind of integrated circuit, a state machine, a processor based on an advanced reduced instruction set machine (ARM), and similar electronic products.
[0026] Referring Figure 1 , the imaging processing system 100 can obtain a fundus image of the eye of the detection object by externally connecting a fundus lens device 106 to perform imaging processing and analysis of the optic nerve cup and disc.
[0027] In one embodiment, the fundus lens device 106 can be a direct ophthalmoscope or an indirect ophthalmoscope. Taking the direct ophthalmoscope as an example, the fundus lens device 106 can directly examine the fundus without dilating the pupil and perform the examination in a dark room.
[0028] In one embodiment, the fundus lens device 106 is a digital fundus camera, generally using a digital camera with more than 2 million pixels to obtain a high-definition fundus image. The digital camera is connected to the special interface of the fundus camera, takes the required fundus image, and then transmits it to the imaging processing system 100 for image analysis processing, storage, printing, etc. In one embodiment, the processor 102 is responsible for receiving the fundus image and performing image analysis.
[0029] In one embodiment, the fundus image captured by the fundus lens device 106, such as Figure 3 the fundus map 310, can be transmitted to the imaging processing system 100 by wired or wireless transmission.
[0030] In addition, the imaging processing system 100 can also be connected to a medical database (not shown) through a communication network such as the Internet or an internal network to obtain a fundus image, such as Figure 3 the fundus map 310.
[0031] The method for processing the optic nerve cup and disc image of the present invention can be implemented by the processor 102 in the imaging processing system 100 to read and execute programs and models from the storage device 104.
[0032] Figure 2 Show a flowchart of the method for processing the optic nerve cup and disc image according to an embodiment of the present invention.
[0033] Referring to Figure 2 , in step S201, the processor 102 in the image processing system 100 obtains a fundus map (for example, Figure 3 fundus map 310).
[0034] In one embodiment, the fundus map 310 is an eye image. The overall eye image is generally red (or similar colors such as red-orange). Slightly different color blocks (for example, yellow) will appear at the Optic Disc 312 and the Optic Cup 314.
[0035] In step S202, the processor 102 identifies an Optic Disc 312 and an Optic Cup 314 in the above-mentioned fundus map 310 through an Image Recognition Model IRM (refer to Figure 3 ), and marks the contours of each of the above-mentioned Optic Disc 312 and the above-mentioned Optic Cup 314 (as shown by the dotted lines in Figure 3 ).
[0036] In step S203, the processor 102 can execute an interpretation application mode. Additionally, in step S204, the processor 102 can execute a marking application mode.
[0037] When executing the interpretation application mode, in step S205, the processor 102 generates an interpretation data for risk assessment at least based on the contours of the above-mentioned Optic Disc 312 and the above-mentioned Optic Cup 314.
[0038] When executing the marking application mode, in step S206, the processor 102 displays the contours of each of the above-mentioned Optic Disc 312 and the above-mentioned Optic Cup 314 marked by a plurality of contour points on a display device (not shown in Figure 1 ) of the image processing system 100 (such as a notebook computer). Then, in step S207, the processor 102 receives instructions from the above-mentioned ophthalmologist to modify, clear, or accept the above-mentioned plurality of contour points through an input device (not shown in Figure 1 ) of the image processing system 100.
[0039] In one embodiment, in the annotation application mode, the processor 102 further has an annotation tool with a "pre-annotation" contour function. It should be noted that the contours of the optic disc 312 and the optic cup 314 annotated by the aforementioned image segmentation model IRM (referred to as "pre-annotation" contours) are suggestions or temporary annotations, which can be used by the ophthalmologist to observe the characteristics of the optic disc and the optic cup. In this way, the ophthalmologist can use the "pre-annotation" contours to observe the contours of the optic disc and the optic cup suggested by the image segmentation model IRM, and use the "pre-annotation" contours as a reference for a second expert opinion.
[0040] In one embodiment, the annotation application mode can be used to assist an ophthalmologist in annotating the optic cup and disc in an original fundus image. In this embodiment, the processor 102 obtains the original fundus image in step S201, and then in step S202, the processor 102 uses the image segmentation model IRM to identify an optic disc 312 and an optic cup 314 and generate corresponding "pre-annotation" contours. The processor 102 executes the annotation application mode for the physician to annotate the optic cup and disc, and there are the following scenarios.
[0041] Scenario (1): The processor 102 displays the fundus image and the "pre-annotation" contours on the display device for the ophthalmologist to view. When the ophthalmologist fully accepts the "pre-annotation" contours, the processor 102 ends the annotation of the optic cup and disc according to the instruction input by the ophthalmologist through the input device.
[0042] Scenario (2): If the ophthalmologist only partially accepts the "pre-annotation" contours in the fundus image based on his professional and practical insights, the processor 102 further provides a modification tool, enabling the ophthalmologist to partially correct the "pre-annotation" contours through the input device, such as adjusting or changing the positions of the plural contour points representing the "pre-annotation" contours, and then ending the annotation of the optic cup and disc.
[0043] Scenario (3): After referring to the "pre-annotation" contours, the ophthalmologist can also clear all the contour points through the input device and manually annotate the contours of the optic disc or / and the optic cup.
[0044] Traditionally, when an ophthalmologist annotates the optic cup and disc, he has to manually mark coordinate points one by one to depict the contour, which is quite time-consuming. However, in this example, due to the annotation application mode executed by the processor 102, the annotation process can be simplified by changing the contour points (coordinate points) of the "pre-annotation" contours, thereby accelerating the annotation time, such as in the case of scenario (2) above, but not limited thereto.
[0045] In an embodiment of the present invention, the above-mentioned image segmentation model IRM is obtained, for example, by using a large number of pre-collected fundus images as training data and inputting the training data into a deep learning model for training. Note that the above-mentioned large number of pre-collected fundus image training data have the contours of the optic nerve head and the optic nerve cup pre-labeled by an ophthalmologist in each of the above fundus images. In this way, the image segmentation model IRM used in this embodiment (i.e., the trained deep learning model) will, when receiving an original fundus image (an unlabeled fundus image), identify and label the optic nerve head and the optic nerve cup from the fundus image based on the color changes around the optic nerve head, the color changes between the optic nerve head and the optic nerve cup, and / or the vascular turning changes in the optic nerve head.
[0046] U-Net is a convolutional neural network developed for biomedical image segmentation. U-Net, based on a fully convolutional network, is modified and extended in structure so that it can produce more accurate segmentation and identification with fewer training images. In an embodiment, the image segmentation model IRM is implemented using U-Net, for example, but is not limited thereto.
[0047] In an embodiment of the present invention, the image segmentation model IRM is also pre-trained by deep learning using fundus images with labeled vascular distributions. After training, the image segmentation model IRM can identify and obtain a vascular segmentation map from the original fundus image. In addition, the trained image segmentation model IRM can be output to various application devices, such as a hospital computer device, a physician's laptop, or a handheld medical device, etc. In this embodiment, for example, it is the storage device 104 of a laptop (image processing system 100).
[0048] Therefore, after receiving the fundus image 310, the processor 102 can also segment the vascular part in the fundus image 310. The processor 102 can further selectively generate a vascular segmentation map 330 corresponding to the fundus image 310 through the image segmentation model IRM according to the application situation. In this way, the processor can extract the shape and thickness of the blood vessels from the vascular segmentation map to more accurately identify the optic nerve cup and the optic nerve head in combination with the color differences of the optic nerve cup and the optic nerve head in the fundus image.
[0049] For example, the processor 102 can identify the optic disc in the above fundus image based on the color difference between the optic disc and its surroundings through the image segmentation model IRM; or, simultaneously based on the vessel turns, shapes, and thicknesses in the above angiogram and the aforementioned color difference in the above fundus image, identify and label the above optic disc. In addition, the processor 102 can identify and label the optic cup from the above optic discs based on the vessel shapes and turns in the above angiogram and the color and shape differences between the optic cup and the optic disc through the image segmentation model IRM.
[0050] In one embodiment, in the interpretation application mode, in step S205, the processor 102 generates interpretation data based on the contours of the above optic disc 312 and the above optic cup 314, such as any one of the cup-to-disc ratio (CDR), vertical cup-to-disc ratio (VCDR), or rim-to-disc ratio (RDR) of the optic cup to the optic disc, for an ophthalmologist to assess the risk of eye damage and can further provide suggestions to the detection object.
[0051] By means of the annotation application mode and the interpretation application mode provided in the optic cup and disc image processing system and method of the present invention, it is possible to assist the annotation function of the optic cup and disc contours, thereby reducing the time for physicians to perform annotations, and at the same time helping to accelerate the risk assessment process and improve the interpretation quality.
[0052] Although the present invention has been disclosed above in an implementation manner, it is not intended to limit the present invention. Any person skilled in this art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to that defined by the appended claims.
Claims
1. An optic nerve cup-disc image processing system, comprising: A processor that accesses a program to execute: Receiving a fundus map, and identifying an optic nerve disc and an optic nerve cup in the fundus map through an image cutting model, and marking the contours of the optic nerve disc and the optic nerve cup; In an interpretation application mode, generating an interpretation data for risk assessment at least based on the contours of the optic nerve disc and the optic nerve cup; Wherein, the image cutting model is a deep learning model trained by using pre-collected fundus maps as training data, and the pre-collected fundus map training data is marked with the contours of the optic nerve disc and the optic nerve cup by an ophthalmologist for each in the training data.
2. The optic nerve cup-disc image processing system according to claim 1, further comprising a display device and an input device; wherein The processor executes: In a marking application mode, marking the contours of each of the optic nerve disc and the nerve cup with a plurality of contour points respectively, and displaying them on the display device; Receiving, through the input device, instructions from the ophthalmologist to modify, clear, or accept the plurality of contour points.
3. The optic nerve cup-disc image processing system according to claim 1, wherein, The image cutting model further identifies a blood vessel cutting map in the fundus map, and identifies the optic nerve disc according to the color difference in the fundus map, or simultaneously identifies the optic nerve disc according to the color differences in the blood vessel map and the fundus map.
4. The optic nerve cup-disc image processing system according to claim 1, wherein, The image cutting model further identifies a blood vessel cutting map in the fundus map, and identifies the optic nerve cup from the optic nerve disc according to the blood vessel shape and turning in the blood vessel map, and the color and shape differences from the optic nerve disc.
5. The optic nerve cup-disc image processing system according to claim 1, wherein, The interpretation data is the optic nerve cup-disc ratio, vertical cup-disc ratio, or disc ring ratio of the optic nerve cup to the optic nerve disc obtained by the processor according to the contours of the optic nerve cup and the optic nerve disc.
6. An optic nerve cup-disc image processing method, applicable to an electronic device, comprising: Obtaining a fundus map by a processor in the electronic device, and identifying an optic nerve disc and an optic nerve cup in the fundus map through an image cutting model, and marking the contours of the optic nerve disc and the optic nerve cup; Providing an interpretation application mode, and generating an interpretation data for risk assessment by the processor at least based on the contours of the optic nerve disc and the optic nerve cup; Wherein, the image cutting model is a deep learning model trained by using pre-collected fundus maps as training data, and the pre-collected fundus maps are pre-marked with the contours of the optic nerve disc and the optic nerve cup by an ophthalmologist.
7. The optic nerve cup-disc image processing method according to claim 6, further comprising: By the above-mentioned processor, in a marking application mode, the contours of each of the above-mentioned optic nerve head and the above-mentioned optic nerve cup are respectively marked with a plurality of contour points and displayed on the above-mentioned display device; Through an input device of the above-mentioned electronic device, receive instructions from the above-mentioned ophthalmologist to modify, clear, or accept the above-mentioned plurality of contour points.
8. The method for processing an image of an optic nerve cup and disc according to claim 6, wherein, The above-mentioned image cutting model further identifies a blood vessel cutting map in the above-mentioned fundus image, and identifies the above-mentioned optic nerve head based on the color difference in the above-mentioned fundus image, or simultaneously identifies the above-mentioned optic nerve head based on the color differences in the above-mentioned blood vessel map and the above-mentioned fundus image.
9. The method for processing an image of an optic nerve cup and disc according to claim 6, wherein, The above-mentioned image cutting model further identifies a blood vessel cutting map in the above-mentioned fundus image, and identifies the above-mentioned optic nerve cup from the above-mentioned optic nerve head based on the turning of the blood vessel shape in the above-mentioned blood vessel map and the differences in color and shape from the above-mentioned optic nerve head.
10. The method for processing an image of an optic nerve cup and disc according to claim 6, wherein, The above-mentioned interpretation data is the optic nerve cup-disc ratio, vertical cup-disc ratio, or disc ring ratio of the above-mentioned optic nerve cup to the above-mentioned optic nerve head obtained by the above-mentioned processor according to the contours of the above-mentioned optic nerve cup and the above-mentioned optic nerve head.