Apparatus and method for detecting ocular symptoms

Abstract

To provide an apparatus and a method for detecting ocular conditions.SOLUTION: A method for detecting ocular conditions according to the present invention includes: receiving a first fundus image corresponding to a first tag, and generating a masked fundus image by masking a part of the first fundus image; generating a first prediction result by inputting the first fundus image into a machine learning model, and generating a second prediction result by inputting the masked fundus image into the machine learning model; training the machine learning model by updating the machine learning model based on the first prediction result, the second prediction result, and the first tag; and receiving a second fundus image, generating a third prediction result for the ocular conditions by inputting the second fundus image into the trained machine learning model, and outputting the third prediction result.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to image data processing technology, and particularly to an eye symptom detection device and a detection method.

Background Art

[0002] Supporting the examination of fundus images using an artificial intelligence (AI) model has attracted attention in recent years in the application and development of the smart medical field. The AI model can analyze the input fundus image to examine whether the subject has a disease in the eyes. Generally, the input data of the AI model is the entire fundus image. However, in practice, when a doctor examines a fundus image, they do not look at the entire fundus image, but only observe a specific part of the fundus image to determine whether the subject has a disease in the eyes. For example, in the case of myopic Macular degeneration (MMD), the doctor determines whether the subject has choroidal neovascularization (CNV) based on the macular area of the fundus image. It can be seen from this that images other than the macular area may have an adverse effect on the determination result of the AI model.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present invention provides an eye symptom detection device and a detection method that can assist a doctor in determining whether a subject is at risk of suffering from an eye disease based on a fundus image.

Means for Solving the Problems

[0004] The ophthalmic symptom detection device of the present invention comprises a processor, a storage medium, and a transceiver. A machine learning model is stored in the storage medium. The processor is coupled to the storage medium and the transceiver and is configured to receive a first fundus image corresponding to a first tag via the transceiver, mask a portion of the first fundus image to generate a masked fundus image, input the first fundus image to the machine learning model to generate a first prediction result, input the masked fundus image to the machine learning model to generate a second prediction result, update the machine learning model based on the first prediction result, the second prediction result, and the first tag to train the machine learning model, and also receive a second fundus image via the transceiver, input the second fundus image to the trained machine learning model to generate a third prediction result for ophthalmic symptoms, and output the third prediction result via the transceiver.

[0005] In one embodiment of the present invention, the processor is configured to update a machine learning model based on a loss function, the loss function comprising the sum of a first function associated with a masked fundus image and a first tag, and a second function associated with a masked fundus image and a first fundus image.

[0006] In one embodiment of the present invention, the first function includes a binary cross-entropy between a second prediction result of a masked fundus image and a first tag.

[0007] In one embodiment of the present invention, the second function includes the product of the mean squared error between the second prediction result of the masked fundus image and the first prediction result of the first fundus image and a weight.

[0008] In one embodiment of the present invention, the training process of the machine learning model includes a plurality of iterations, each corresponding to a plurality of indices, wherein the first iteration of the plurality of iterations updates the weights according to the first index of the plurality of indices, the first index and the first iteration correspond to each other, and updates the machine learning model based on the updated weights.

[0009] In one embodiment of the present invention, the weight and the first index are proportional.

[0010] In one embodiment of the present invention, the machine learning model includes a deep neural network.

[0011] In one embodiment of the present invention, the deep neural network includes a feature extraction block, a spatial attention block, and a density block, the spatial attention block being located between the feature extraction block and the density block.

[0012] In one embodiment of the present invention, the processor is further configured to perform preprocessing on a first fundus image, input the preprocessed first fundus image into a machine learning model to generate a first prediction result, and perform preprocessing on a masked fundus image, input the preprocessed masked fundus image into a machine learning model to generate a second prediction result.

[0013] In one embodiment of the present invention, the processor is further configured to generate a masked fundus image by masking a portion of the first fundus image based on a preset region.

[0014] The present invention provides a method for detecting ocular symptoms, which includes receiving a first fundus image corresponding to a first tag, masking a portion of the first fundus image to generate a masked fundus image, inputting the first fundus image into a machine learning model to generate a first prediction result, inputting the masked fundus image into the machine learning model to generate a second prediction result, updating the machine learning model based on the first prediction result, the second prediction result, and the first tag to train the machine learning model, and receiving a second fundus image, inputting the second fundus image into the trained machine learning model to generate a third prediction result for ocular symptoms, and outputting the third prediction result. [Effects of the Invention]

[0015] Based on the above, the detection device of the present invention generates a masked fundus image based on the fundus image, and by training a machine learning model based on the fundus image and the masked fundus image, the trained machine learning model can focus its attention on a specific region of the fundus image (e.g., the macula). Therefore, when analyzing the input fundus image to predict whether an ocular symptom has occurred in the subject's eye, the machine learning model can produce a more accurate judgment result. [Brief explanation of the drawing]

[0016] [Figure 1] This is a schematic diagram of an eye symptom detection device according to an embodiment of the present invention. [Figure 2] This flowchart shows the training or updating of a machine learning model using one embodiment of the present invention. [Figure 3] This is a schematic diagram of a machine learning model based on one embodiment of the present invention. [Figure 4] This flowchart shows a method for detecting ocular symptoms according to one embodiment of the present invention. [Modes for carrying out the invention]

[0017] To facilitate understanding of the present invention, embodiments in which the present invention can be carried out are illustrated below. Furthermore, wherever possible, members, components, and processes with the same reference numerals used in the drawings and embodiments represent the same or similar members.

[0018] Figure 1 is a schematic diagram of an ocular symptom detection device according to an embodiment of the present invention. The detection device 100 may include a processor 110, a storage medium 120, and a transceiver 130.

[0019] The processor 110 is, for example, a central processing unit (CPU), or other programmable general-purpose or application-specific microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components, or a combination of the above components. The processor 110 can be coupled to a storage medium 120 and a transceiver 130, and can access and execute multiple modules and various applications stored in the storage medium 120.

[0020] The memory medium 120 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components or a combination of the above components, and is used to store a plurality of modules or various applications executable by the processor 110. In this embodiment, the machine learning model 200 can be stored in the memory medium 120.

[0021] The transceiver 130 transmits or receives signals wirelessly or wired. The transceiver 130 can also perform operations such as, for example, low noise amplification, impedance matching, mixing, up or down frequency conversion, filtering, amplification, and similar operations.

[0022] The machine learning model 200 can perform feature extraction by focusing on a specific region (for example, the macula) of the fundus image through training or updating, thereby generating a more accurate prediction result of eye symptoms. FIG. 2 is a flowchart showing the training or updating of the machine learning model 200 according to an embodiment of the present invention, and this flowchart can be implemented by the detection device 100 shown in FIG. 1.

[0023] In step S201, the processor 110 can receive a fundus image having one or more tags via the transceiver 130. The tag of the fundus image can be used to indicate whether the fundus image corresponds to an eye having an eye symptom. For example, the tag of the fundus image can indicate whether the fundus image corresponds to an eye at risk of developing CNV.

[0024] In step S202, the processor 110 can generate a masked fundus image by masking a part of the fundus image. In one embodiment, the processor 110 masks the fundus image with a preset region to generate a masked fundus image. For example, assuming that the center point of the captured fundus image is the location of the macula, the preset region of the fundus image can be a region including the center point. The processor 110 can also generate a masked fundus image by masking the region outside the preset region of the fundus image.

[0025] In step S203, the processor 110 can selectively perform preprocessing on the fundus image or the masked fundus image.

[0026] In step S204, the processor 110 can input the (preprocessed) fundus image into the machine learning model 200 to generate a prediction result of the fundus image, and can also input the (preprocessed) masked fundus image into the machine learning model 200 to generate a prediction result of the masked fundus image. The prediction result may be used to indicate whether the fundus image or the masked fundus image corresponds to an eye at risk of developing an eye symptom.

[0027] In one embodiment, the machine learning model 200 may include a deep neural network (DNN). Figure 3 is a schematic diagram of the machine learning model 200 according to one embodiment of the present invention. The machine learning model 200 may include an input layer 210, an output layer 230, and a hidden layer interposed between the input layer 210 and the output layer 230. The hidden layer may include one or more feature extraction blocks 221, one or more spatial attention blocks 222, and one or more dense blocks 223, of which the spatial attention block 222 can be interposed between the feature extraction block 221 and the dense block 223. Specifically, the input terminal of the feature extraction block 221 can be connected to the output terminal of the input layer 210, the input terminal of the spatial attention block 222 can be connected to the output terminal of the feature extraction block 221, the input terminal of the density block 223 can be connected to the output terminal of the spatial attention block 222, and the input terminal of the output layer 230 can be connected to the output terminal of the density block 223.

[0028] JPEG0007842821000001.jpg101167

[0029] JPEG0007842821000002.jpg98167

[0030] JPEG0007842821000003.jpg46168

[0031] In step S206, the processor 110 can determine whether the training or update of the machine learning model 200 is complete. If the training or update of the machine learning model 200 is complete, the process proceeds to step S207. If the training or update of the machine learning model 200 is not complete, the processor 110 can re-execute step S204 to train or update the machine learning model 200 based on the next fundus image and masked fundus image.

[0032] In one embodiment, the processor 110 can determine that training of the machine learning model 200 is complete based on the number of iterations of the training process of the machine learning model 200 being greater than a threshold, and can also determine that training of the machine learning model 200 is not yet complete based on the number of iterations of the training process being less than or equal to a threshold.

[0033] In one embodiment, the processor 110 may determine that training of the machine learning model 200 is complete based on the value of the loss function of the machine learning model 200 being less than a threshold, or determine that training of the machine learning model 200 is not yet complete based on the value of the loss function being greater than or equal to a threshold.

[0034] In step S207, the processor 110 can stop training or updating the machine learning model 200. After training or updating the machine learning model 200 is complete, the processor 110 can receive the fundus images awaiting prediction via the transceiver 130 and input the fundus images into the machine learning model 200 to generate prediction results for ocular symptoms. Here, the prediction results can be used to indicate whether there is a risk of ocular symptoms in the eye corresponding to the fundus image. The processor 110 can output the prediction results via the transceiver 130 for the reference of a physician.

[0035] Figure 4 is a flowchart showing a method for detecting ocular symptoms according to one embodiment of the present invention, and this detection method can be carried out by the detection device 100 shown in Figure 1. In step S401, a first fundus image corresponding to the first tag is received, and a part of the first fundus image is masked to generate a masked fundus image. In step S402, the first fundus image is input to a machine learning model to generate a first prediction result, and the masked fundus image is input to the machine learning model to generate a second prediction result. In step S403, the machine learning model is updated and trained based on the first prediction result, the second prediction result and the first tag. In step S404, a second fundus image is received, the second fundus image is input to the trained machine learning model to generate a third prediction result for ocular symptoms, and the third prediction result is output.

[0036] In summary, the detection device of the present invention generates a masked fundus image based on a fundus image, and by training a machine learning model based on the fundus image and the masked fundus image, the trained machine learning model can focus its attention on a specific region of the fundus image (e.g., the macula). The detection device can train or update the machine learning model based on a loss function that includes functions such as binary cross-entropy or mean squared error. When analyzing an input fundus image to predict whether a subject's eye is at risk of developing an eye condition, the trained machine learning model can produce more accurate judgment results. [Industrial applicability]

[0037] The detection device of the present invention may be suitable for applications related to the diagnosis or treatment of eye diseases. [Explanation of Symbols]

[0038] 100: Detection device 110: Processor 120:Storage medium 130: Transmitter / Receiver 200: Machine Learning Models 210: Input Layer 221: Feature extraction block 222: Spatial attention block 223: Dense Block 230: Output layer S201, S202, S203, S204, S205, S206, S207, S401, S402, S403, S404: Step

Claims

1. Transmitter and receiver, A storage medium on which a machine learning model is stored, The system comprises a storage medium and a processor coupled to the transceiver, The aforementioned processor, The transceiver receives a first fundus image corresponding to a first tag, a portion of the first fundus image is masked to generate a masked fundus image, and the first tag indicates whether the masked fundus image corresponds to an eye with ocular symptoms. The first fundus image is input to the machine learning model to generate a first prediction result, and the masked fundus image is input to the machine learning model to generate a second prediction result. The machine learning model is retrained based on the first prediction result, the second prediction result, and the first tag. The system is configured to receive a second fundus image via the transceiver, input the second fundus image into the newly trained machine learning model to generate a third prediction result for ocular symptoms, and output the third prediction result via the transceiver. A device for detecting eye symptoms.

2. The ocular symptom detection device according to claim 1, wherein the processor is configured to further perform retraining the machine learning model based on a loss function, the loss function comprising the sum of a first function associated with the masked fundus image and the first tag and a second function associated with the masked fundus image and the first fundus image.

3. The ocular symptom detection device according to claim 2, wherein the first function includes the binary cross-entropy between the second prediction result of the masked fundus image and the first tag.

4. The ocular symptom detection device according to claim 2, wherein the second function includes the product of the mean squared error between the second prediction result of the masked fundus image and the first prediction result of the first fundus image and a weight.

5. The eye symptom detection device according to claim 4, wherein the process of retraining the machine learning model comprises a plurality of iterations, the kth iteration of which updates the weights according to the index k of the kth iteration, where k is a positive integer, and retraining the machine learning model based on the updated weights.

6. The ocular symptom detection device according to claim 5, wherein the weight and the index k are proportional.

7. The eye symptom detection device according to claim 1, wherein the machine learning model includes a deep neural network.

8. The ocular symptom detection device according to claim 7, wherein the deep neural network includes a feature extraction block, a spatial attention block, and a crowd block, the spatial attention block being located between the feature extraction block and the crowd block.

9. The aforementioned processor, Performing preprocessing on the first fundus image, inputting the preprocessed first fundus image into the machine learning model to generate the first prediction result, The ocular symptom detection device according to claim 1, further configured to perform preprocessing on the masked fundus image, input the preprocessed masked fundus image into the machine learning model to generate the second prediction result.

10. The ocular symptom detection device according to claim 1, wherein the processor is configured to further perform the function of generating the masked fundus image by masking a portion of the first fundus image based on a preset region.

11. A processor receives a first fundus image corresponding to a first tag, and generates a masked fundus image by masking a portion of the first fundus image, wherein the first tag indicates whether the first fundus image corresponds to an eye having an eye symptom. The processor inputs the first fundus image to a machine learning model to generate a first prediction result, and inputs the masked fundus image to the machine learning model to generate a second prediction result. The processor retrains the machine learning model based on the first prediction result, the second prediction result, and the first tag. A method for operating a medical device, comprising: receiving a second fundus image with the processor; inputting the second fundus image into the trained machine learning model; generating a third prediction result for ophthalmic symptoms; and outputting the third prediction result.

Images