Method and system for processing fundus image, electronic device and storage medium

Abstract

This disclosure discloses a method and system for processing fundus images. The method includes: acquiring a three-dimensional fundus OCT image and at least one fundus OCTA projection image of a target eye; inputting the three-dimensional fundus OCT image into a lesion detection model to obtain a first detection result of fundus lesions in the target eye; inputting at least one fundus OCTA projection image into a blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result of the target eye; and obtaining a second detection result of fundus lesions in the target eye based on the first detection result of fundus lesions and at least one fundus blood flow abnormality detection result.

Description

Technical Field

[0001] This disclosure relates to the field of AI in healthcare, and more particularly to a method, system, electronic device, and storage medium for processing fundus images. Background Technology

[0002] Optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) are both non-invasive imaging techniques. OCT images can provide anatomical information about the retina and choroid. OCTA images can provide not only anatomical information about the retina and choroid but also microvascular blood flow information about the retina and choroid. Nevertheless, there are still some limitations in clinical diagnosis based on OCT and OCTA images. For example, the amount of macular OCTA data for a single patient is enormous, requiring doctors to spend a significant amount of time interpreting the images to locate lesions and make a final diagnosis, resulting in lengthy diagnostic processes and heavy reliance on the doctor's experience. Summary of the Invention

[0003] This disclosure aims to at least partially address one of the technical problems in the related art.

[0004] To achieve the above objectives, the first aspect of this disclosure provides a method for processing fundus images, comprising:

[0005] Acquire three-dimensional fundus OCT images and at least one fundus OCTA projection image of the target eye;

[0006] The three-dimensional fundus OCT image is input into the lesion detection model to obtain the first detection result of the fundus lesion of the target eye;

[0007] The at least one fundus OCTA projection image is input into the blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result for the target eye; and

[0008] Based on the first detection result of the fundus lesion and the at least one detection result of fundus blood flow abnormality, a second detection result of the fundus lesion of the target eye is obtained.

[0009] To achieve the above objectives, a second aspect of this disclosure provides a system for processing fundus images, comprising:

[0010] The image acquisition module is configured to acquire a three-dimensional fundus OCT image of the target eye and at least one fundus OCTA projection image;

[0011] The OCT image processing module is configured to input the three-dimensional fundus OCT image into the lesion detection model to obtain the first detection result of the fundus lesion of the target eye;

[0012] An OCTA projection image processing module is configured to input the at least one fundus OCTA projection image into a blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result for the target eye; and

[0013] The processing result acquisition module is configured to obtain a second detection result of fundus lesions in the target eye based on the first detection result of the fundus lesions and the at least one detection result of abnormal fundus blood flow.

[0014] To achieve the above objectives, a third aspect of this disclosure provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0015] The memory stores computer-executed instructions;

[0016] The processor executes computer execution instructions stored in the memory to implement the fundus image processing method described in the first aspect above.

[0017] To achieve the above objectives, the fourth aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the fundus image processing method described in the first aspect.

[0018] To achieve the above objectives, the fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect. Attached Figure Description

[0019] The advantages of the embodiments of this disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

[0020] Figure 1 This is a flowchart of a method for processing fundus images according to an embodiment of the present disclosure;

[0021] Figure 2a This is a schematic diagram illustrating a three-dimensional fundus OCT image provided according to an embodiment of the present disclosure;

[0022] Figure 2b This is a schematic diagram illustrating a three-dimensional fundus OCTA image provided according to an embodiment of the present disclosure;

[0023] Figure 3a This is an OCTA projection image of the retinal surface provided according to an embodiment of the present disclosure;

[0024] Figure 3b This is a deep retinal OCTA projection image provided according to an embodiment of this disclosure;

[0025] Figure 3c An OCTA projection image of the avascular layer of the retina provided according to an embodiment of this disclosure;

[0026] Figure 3d A retinal OCTA projection image provided according to an embodiment of this disclosure;

[0027] Figure 4 This is a flowchart of a method for processing fundus images according to another embodiment of the present disclosure;

[0028] Figure 5 This is a flowchart illustrating a lesion detection model provided according to an embodiment of the present disclosure.

[0029] Figure 6 This is a flowchart of a fundus blood flow abnormality detection model provided according to an embodiment of the present disclosure;

[0030] Figure 7 This is a flowchart of a quantitative analysis provided according to an embodiment of the present disclosure;

[0031] Figure 8 This is a flowchart of a method for processing fundus images according to yet another embodiment of the present disclosure;

[0032] Figure 9 This is a schematic diagram of the structure of a fundus image processing system provided according to an embodiment of the present disclosure. Detailed Implementation

[0033] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0034] OCTA (Optical Coherence Tomography) is a fundus imaging technique based on optical coherence tomography. As a non-invasive imaging technique, it can be used to observe the microvascular structure of the retina and choroid, providing a new perspective and tool for the early diagnosis, monitoring, and management of systemic fundus diseases. OCTA plays an important role in the diagnosis of fundus diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and retinal vein occlusion (RVO). It can provide not only anatomical information of the retina and choroid but also information on microvascular blood flow in these areas. However, despite being a high-resolution, non-invasive imaging technique that provides anatomical and blood flow information, OCTA still has some limitations in clinical diagnosis, mainly including the following aspects:

[0035] Firstly, diagnostic efficiency and effectiveness are limited. A single patient's fundus OCTA data is massive, containing data in two modalities: three-dimensional retinal and choroidal structural information (i.e., fundus OCTA images or fundus OCTA volume data) and multi-level retinal and choroidal microvascular blood flow information (i.e., two-dimensional fundus OCTA planar images or fundus OCTA projection images). Doctors need to spend a significant amount of time interpreting the images to locate lesions and comparing abnormalities across multiple modalities to arrive at a final diagnosis, resulting in lengthy diagnostic times and heavy reliance on the doctor's experience.

[0036] Secondly, quantitative analysis of lesions is limited. Doctors rely on manually measuring key indicators such as vascular density, foveal avascular zone (FAZ), and vascular branching structure. This method is not only time-consuming, but may also lead to subjective errors in diagnosis due to differences in the experience of individual doctors.

[0037] Thirdly, there is a lack of effective automated comprehensive analysis tools. Clinically, there is a reliance on doctors manually comparing multiple image datasets, lacking effective automated diagnostic systems based on artificial intelligence, and thus failing to efficiently integrate multimodal information. Furthermore, although artificial intelligence (AI) and deep learning technologies have made groundbreaking progress in medical image analysis, particularly demonstrating excellent performance in lesion detection, segmentation, and classification tasks, there is currently a lack of feasible methods and techniques for the assisted diagnosis of various fundus diseases.

[0038] The present disclosure will now be described in detail with reference to specific embodiments.

[0039] Figure 1 This is a flowchart of a method for processing fundus images according to an embodiment of the present disclosure.

[0040] This method can be implemented using a computer program and can run on a fundus image processing system. The computer program can be integrated into an application or run as a standalone utility application. The fundus image processing system can be implemented in software and / or hardware. The fundus image processing method can be executed by an electronic device.

[0041] like Figure 1 As shown, the method for processing this fundus image may include the following operations:

[0042] In operation S101, a three-dimensional fundus OCT image of the target eye and at least one fundus OCTA projection map are acquired.

[0043] In this embodiment of the disclosure, the target eye may also be referred to as the eye being tested, which may be the eye of the patient undergoing diagnosis of fundus diseases.

[0044] In this embodiment, the three-dimensional fundus OCT image can also be simply referred to as fundus OCT image, which is obtained by performing a fundus OCT C-scan (C-scan) on the target eye using an ophthalmic OCT device. Similarly, the three-dimensional fundus OCTA image can also be simply referred to as fundus OCTA image, which is also obtained by performing a fundus OCT C-scan (C-scan) on the target eye using an ophthalmic OCT device. In operation S101, the three-dimensional fundus OCT image of the target eye (e.g., ...) can be obtained from the ophthalmic OCT device through a server that is communicatively connected to the ophthalmic OCT device. Figure 2a (as shown) and three-dimensional fundus OCTA images (such as...) Figure 2b As shown, the server can obtain at least one fundus OCTA projection image by projecting the three-dimensional fundus OCTA image.

[0045] In one embodiment of this disclosure, at least one fundus OCTA projection map may include at least one of the following OCTA projection maps: a surface retinal OCTA projection map, a deep retinal OCTA projection map, an avascular retinal OCTA projection map, and a retinal OCTA projection map. The retinal OCTA projection map may also be referred to as a full-thickness retinal OCTA projection map or a layer-specific retinal OCTA projection map.

[0046] In this embodiment, a retinal surface OCTA projection image can be obtained through the following operations: A server connected to an ophthalmic OCT device acquires a fundus OCTA image of the target eye from the ophthalmic OCT device; the server layers the acquired fundus OCTA image to extract the retinal surface OCTA image; the server projects the retinal surface OCTA image to obtain a retinal surface OCTA projection image (e.g., ...). Figure 3a (As shown). Similarly, in this embodiment, a deep retinal OCTA projection image can be obtained through the following operations: a server connected to an ophthalmic OCT device acquires a fundus OCTA image of the target eye from the ophthalmic OCT device; the server layers the acquired fundus OCTA image to extract a deep retinal OCTA image; the server projects the deep retinal OCTA image to obtain a deep retinal OCTA projection image (as shown). Figure 3b (As shown). Similarly, in this embodiment, the OCTA projection image of the avascular layer of the retina can be obtained through the following operations: A server connected to an ophthalmic OCT device acquires a fundus OCTA image of the target eye from the ophthalmic OCT device; the server layers the acquired fundus OCTA image to extract the OCTA image of the avascular layer of the retina; the server projects the OCTA image of the avascular layer of the retina to obtain the OCTA projection image of the avascular layer of the retina (as shown). Figure 3c (As shown). Similarly, in this embodiment, a retinal OCTA projection image can be obtained through the following operations: a server connected to an ophthalmic OCT device acquires a fundus OCTA image of the target eye from the ophthalmic OCT device; the server layers the acquired fundus OCTA image to extract the retinal OCTA image; the server projects the retinal OCTA image to obtain a retinal OCTA projection image (as shown). Figure 3d (As shown).

[0047] In this embodiment, the OCTA projection map of the retinal surface corresponds to the inner limiting membrane to the inner plexus layer of the retina, and can be used to detect fundus diseases such as diabetic retinopathy and retinal vein occlusion. In this embodiment, the OCTA projection map of the deep retina corresponds to the inner nuclear layer to the outer plexus layer of the retina, and can be used to detect fundus diseases such as diabetic macular edema, diabetic retinopathy, and retinal vein occlusion. In this embodiment, the OCTA projection map of the avascular layer of the retina corresponds to the outer nuclear membrane to the Bruch's membrane, and can be used to detect fundus diseases such as age-related macular degeneration, myopic macular degeneration, and central serous chorioretinopathy. In this embodiment, the retinal OCTA projection map corresponds to the inner limiting membrane to the outer plexus membrane, including a comprehensive display of overall blood flow, and can be used to detect fundus diseases such as diabetic retinopathy.

[0048] In implementing the embodiments of this disclosure, the inventors discovered that the blood supply to the retina is distributed across different retinal structural layers, and the fundus diseases corresponding to blood flow abnormalities in different retinal structural layers are not entirely the same. Therefore, in the embodiments of this disclosure, by selecting fundus OCTA projection images from one or more retinal structural layers in the aforementioned fundus OCTA projection images, the pathological changes of blood vessels in each retinal structural layer can be analyzed, avoiding missed diagnoses due to improper selection of fundus OCTA projection images, thereby improving the accuracy of fundus blood flow abnormality detection.

[0049] In operation S102, the three-dimensional fundus OCT image is input into the lesion detection model to obtain the first detection result of fundus lesions in the target eye.

[0050] In this embodiment, as described above, the fundus OCT image is obtained by performing a fundus OCTC scan (C-scan) on the target eye using an ophthalmic OCT device. It should be understood that performing an OCT line scan (OCTB scan, B-scan) on any specified location of the fundus of the target eye yields a single frame of fundus OCT cross-sectional image. Performing OCT B-scans on multiple adjacent locations of the fundus of the target eye yields multiple consecutive frames of fundus OCT cross-sectional images, which together constitute a fundus OCT image. Therefore, in operation S102, the fundus OCT cross-sectional images from the fundus OCT image can be input frame by frame into the lesion detection model to obtain the first detection result of fundus lesions in the target eye. In this embodiment, the OCT cross-sectional image can also be referred to as an OCT slice.

[0051] In this embodiment, the lesion detection model can intelligently identify and classify lesions in the input three-dimensional fundus OCT images, thereby quickly and accurately detecting fundus lesions in the three-dimensional fundus OCT images and realizing qualitative analysis of fundus lesions in the target eye.

[0052] In this embodiment, the first detection result of fundus lesions output by the lesion detection model is similar to the three-dimensional fundus OCT image input to the model; it is a three-dimensional fundus OCT image containing lesion detection information. In this embodiment, the lesion detection information included in the first detection result of fundus lesions includes, but is not limited to, lesion location, lesion type, and confidence level.

[0053] In operation S103, at least one fundus OCTA projection image is input into the blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result of the target eye.

[0054] In this embodiment, as described above, at least one fundus OCTA projection map may include at least one of the following OCTA projection maps: a surface retinal OCTA projection map, a deep retinal OCTA projection map, an avascular retinal OCTA projection map, and a retinal OCTA projection map. By inputting fundus OCTA projection maps from different layers into the blood flow abnormality detection model, the blood flow abnormality detection results of the target eye in the corresponding fundus structural layers can be obtained. Taking the at least one fundus OCTA projection map including a surface retinal OCTA projection map, a deep retinal OCTA projection map, an avascular retinal OCTA projection map, and a retinal OCTA projection map as an example, by sequentially inputting these projection maps into the blood flow abnormality detection model, the blood flow abnormality detection results of the target eye in the surface retina, deep retina, avascular retina, and the entire retinal layer can be obtained, respectively.

[0055] Similar to the three-dimensional fundus OCT image structure described above, the three-dimensional fundus OCTA image includes multiple frames of fundus OCTA cross-sectional views. In this embodiment, the OCTA cross-sectional view can also be referred to as an OCTA slice. In some embodiments, when performing fundus blood flow abnormality detection, the three-dimensional OCTA images on each fundus structural layer may not be projected; instead, the three-dimensional OCTA images on each fundus structural layer are directly input frame by frame into the corresponding fundus OCTA blood flow abnormality detection model to obtain the blood flow abnormality detection results of the target eye on each fundus structural layer. In this embodiment, the fundus lesion blood flow abnormality detection results output by the fundus OCTA blood flow abnormality detection model are similar to the three-dimensional fundus OCTA image; it is a three-dimensional fundus OCTA image containing blood flow detection information. Figure 2b As shown, since the microvessels of the retina and choroid appear as dot-like structures in the fundus OCTA cross-sectional image rather than as a complete and intuitive vascular structure, blood flow abnormality detection based directly on three-dimensional fundus OCTA images may not yield complete, accurate, and intuitive results compared to blood flow abnormality detection based on fundus OCTA projection images.

[0056] Therefore, in this embodiment of the present disclosure, using fundus OCTA projection maps instead of three-dimensional fundus OCTA images for blood flow abnormality detection can obtain more complete, accurate, and intuitive blood flow abnormality detection results.

[0057] In this embodiment, by analyzing the fundus OCTA projection image through the blood flow abnormality detection model, abnormal changes in fundus blood flow (such as ischemia, hyperemia, neovascularization, etc.) can be automatically identified, located, and quantified.

[0058] In this embodiment, the fundus blood flow abnormality detection result output by the blood flow abnormality detection model is similar to the fundus OCTA projection map input to the model; it is a two-dimensional fundus OCTA projection map containing blood flow detection information. In this embodiment, the blood flow detection information included in the at least one fundus blood flow abnormality detection result includes, but is not limited to, the type of blood flow abnormality in the abnormal region, the location of the lesion, and the confidence level.

[0059] In operation S104, based on the first detection result of fundus lesions and at least one detection result of abnormal fundus blood flow, a second detection result of fundus lesions in the target eye is obtained.

[0060] In this embodiment, although three-dimensional fundus OCT images can provide morphological information about the retinal layered structure (such as thickness and reflectivity), they are not sensitive to functional information such as microvascular blood flow. Therefore, the first detection result of fundus lesions obtained based on fundus OCT images may miss lesions related to blood flow information. However, fundus OCTA projection images can clearly show the microvascular network of the retina / choroid, and the fundus blood flow abnormality detection result obtained based on the fundus OCTA projection image can directly indicate lesions related to blood flow. Therefore, in this embodiment, by combining the above-mentioned first detection result of fundus lesions with at least one of the above-mentioned fundus blood flow abnormality detection results to obtain the corresponding second detection result of fundus lesions, a more comprehensive, intuitive, and realistic pathological feature can be covered, making the detection results more accurate.

[0061] This embodiment utilizes artificial intelligence big data models (such as lesion detection models and blood flow abnormality detection models) to automatically and intelligently analyze two modalities of data—fundus OCT images and one or more fundus OCTA projection maps—acquired from the target eye of a patient. This yields two intelligent analysis results, which are then combined to obtain an intelligent diagnostic result for fundus lesions in the target eye (i.e., the second detection result for fundus lesions). In this way, for the massive amounts of fundus OCTA data from a single patient with diverse data modalities, doctors no longer need to spend a significant amount of time interpreting images to locate lesions, nor do they need to compare abnormal locations across multiple modalities to arrive at a final diagnosis. Doctors typically only need to spend a relatively small amount of time focusing on the aforementioned AI-assisted diagnostic results to make a final diagnosis. This not only reduces diagnostic time but also decreases reliance on doctors' experience, making the diagnosis more efficient, objective, and reliable.

[0062] Furthermore, in some embodiments, three-dimensional fundus OCT images and three-dimensional fundus OCTA images can also be used for the auxiliary diagnosis of fundus diseases. Since the vascular network appears as a point-like structure in each OCTA slice of a three-dimensional fundus OCTA image, while it appears as a mesh-like structure in a fundus OCTA projection image, using a fundus OCTA projection image provides a more intuitive and complete view of the fundus vascular structure compared to using three-dimensional fundus OCTA images, which is more conducive to the detection of fundus blood flow abnormalities. Therefore, compared to using three-dimensional fundus OCT images and three-dimensional fundus OCTA images for the auxiliary diagnosis of fundus diseases, using three-dimensional fundus OCT images and at least one fundus OCTA projection image can obtain more accurate and complete results for detecting blood flow abnormalities, thereby improving the effectiveness of auxiliary diagnosis of fundus diseases.

[0063] As an optional embodiment, such as Figure 4 The method for processing fundus images includes, in addition to, methods such as... Figure 1 In addition to operations S101-S104 shown, the method may also include operations S401-S402, that is, the method may also include the following operations:

[0064] Operation S101: Acquire a three-dimensional fundus OCT image of the target eye and at least one fundus OCTA projection image;

[0065] Operation S102 inputs the three-dimensional fundus OCT image into the lesion detection model to obtain the first detection result of fundus lesions in the target eye;

[0066] Operation S103: Input at least one fundus OCTA projection image into the blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result of the target eye.

[0067] In step S104, based on the first detection result of fundus lesions and at least one detection result of abnormal fundus blood flow, a second detection result of fundus lesions in the target eye is obtained.

[0068] Operate S401 to obtain quantitative indicators of the fundus of the target eye;

[0069] Operation S402 inputs the second detection result of fundus lesions and fundus quantitative indicators into the disease detection model to obtain the fundus disease detection result of the target eye.

[0070] Operations S101-S104 in this embodiment are the same as or similar to operations S101-S104 in the previous embodiment, and will not be repeated here.

[0071] In this embodiment, during operation S401, the retinal thickness value and / or choroidal thickness value and other membrane thickness indicators of the target eye can be obtained based on the three-dimensional fundus OCT image of the target eye, and other retinal quantitative indicators of the target eye can be obtained based on at least one fundus OCTA projection map, such as a retinal surface OCTA projection map.

[0072] This embodiment allows for further analysis of the second detection result of fundus lesions using quantitative fundus indicators to determine the corresponding disease. Combining quantitative fundus indicators with the second detection result of fundus lesions improves the accuracy and reliability of the detection results.

[0073] In this embodiment, during operation S102, the lesion detection model can be used to detect lesions based on fundus OCT images, including but not limited to epiretinal membranes, retinal atrophy, retinal pigment epithelium disorder, retinal tears, drusen, serous retinal pigment epithelium detachment, non-serous retinal pigment epithelium detachment, subretinal effusion, intraretinal effusion, retinal schisis, preretinal hemorrhage, subretinal hyperreflective material, posterior vitreous detachment, vitreomacular traction, and focal ellipsoidal band loss. Furthermore, during operation S102, in addition to the above-mentioned lesion detection functions, the lesion detection model can also detect and output lesion information such as the number, location, volume, projected area, and confidence level of various lesions.

[0074] For example, such as Figure 5 As shown, the lesion detection process of the lesion detection model may include: reading B-scan slices frame by frame from the three-dimensional fundus OCT image (as shown in (a) in the figure); performing lesion detection on each B-scan slice to obtain continuous frame B-scan slices carrying lesion detection information ((b) in the figure represents one of the slices). For example, as shown in (b) in the figure, for the detected lesion information, the position of the lesion in the B-scan slice can be represented by a rectangle. The lesion named "non-serous retinal pigment epithelial detachment" with a confidence level of "0.51" is represented by "NsPED 0.51", and the lesion named "non-serous retinal pigment epithelial detachment" with a confidence level of "0.84" is represented by "Non-serous retinal pigment epithelial detachment" with a confidence level of "0.84". The continuous frame B-scan slices carrying lesion detection information (as shown in (c) in the figure) are subjected to volume data smoothing to obtain the first detection result of fundus lesions of the target eye. In three-dimensional fundus OCT images, lesions usually span multiple B-scan slices. By performing volume data smoothing on the lesion detection results, fragmentation, duplicate counting, or omission of lesions caused by single-frame detection can be avoided.

[0075] In this embodiment, for example, a lesion detection model can be obtained by training an initial lesion detection model using supervised learning on a first dataset. Exemplary models include, but are not limited to, Yolov5, Yolov7, Faster-RCNN, and DETR models. The first dataset can be obtained by annotating a large number of B-scan slices, for example, manually annotated by a qualified physician.

[0076] As an optional implementation, to optimize model performance, several post-processing techniques, such as confidence threshold screening and non-maximum suppression (NMS), can be connected to the output of the lesion detection model to perform data post-processing on the results output by the lesion detection model, so as to improve the accuracy of the first detection results of fundus lesions.

[0077] For example, in this embodiment, when performing volume data smoothing on the obtained continuous frame B-scan slices carrying lesion detection information, a neighboring frame lesion association strategy can be adopted to match and aggregate regions belonging to the same lesion in the continuous frame B-scan slices to achieve inter-frame smoothing and complete the three-dimensional lesion reconstruction.

[0078] For example, lesion detection results from two adjacent B-scan slices with the same lesion category can be region-matched in the volume data. If the lesion locations in the volume data of the two B-scan slices are within the same region, the lesion locations of the two B-scan slices are aggregated in the volume data to complete the 3D lesion reconstruction. Exemplarily, the strategies used for region matching may include matching candidate construction, matching metric design, matching methods, etc., and the relevant parameters can be configured or adjusted according to the actual application scenario; this disclosure does not impose any limitations on these aspects. Exemplarily, the strategies used for aggregation processing may include trajectory maintenance, etc., and the relevant parameters can also be configured or adjusted according to the actual application scenario; this disclosure does not impose any limitations on these aspects.

[0079] As an optional embodiment, after fusing lesion detection information on consecutive B-scan slices to complete the three-dimensional lesion reconstruction, a three-dimensional bounding box of the lesion can be obtained. Based on this three-dimensional bounding box, the volume, projected area, and confidence level of the lesion can be further calculated, thereby obtaining the first detection result of the fundus lesion. For example, for the first detection result of the fundus lesion, the three-dimensional bounding box of the lesion under the projection view is as follows: Figure 5 As shown in (c) in the figure.

[0080] In this embodiment, the blood flow abnormality detection model can be used to detect blood flow abnormality areas based on fundus OCTA projection images, including but not limited to microaneurysm areas, neovascularization areas, and retinal microvascular abnormality areas.

[0081] For example, such as Figure 6 As shown, the process of detecting fundus blood flow abnormalities using the blood flow abnormality detection model may include: acquiring a fundus OCTA projection image; and performing blood flow abnormality detection on the acquired fundus OCTA projection image. For example, the following can be acquired: Figure 6 The fundus OCTA projection image shown in (a) was used to detect abnormal blood flow, and the results were as follows: Figure 6 The results of fundus blood flow abnormality detection shown in (b) are illustrated in the figure. The location of the blood flow abnormality area is indicated by a rectangle, and the blood flow abnormality information such as the type and confidence level are also displayed.

[0082] In this embodiment, for example, a blood flow abnormality detection model can be obtained by training the initial blood flow abnormality detection model using supervised learning through the construction of a second dataset. The second dataset can be obtained by annotating a large number of fundus OCTA projection images. For example, it can be obtained through manual annotation by a qualified physician.

[0083] As an optional implementation, to optimize model performance, several post-processing techniques, such as confidence threshold screening and NMS, can be connected to the output of the blood flow abnormality detection model to perform data post-processing on the data output by the blood flow abnormality detection model, so as to improve the accuracy of the obtained fundus blood flow abnormality area detection results.

[0084] In this embodiment, during operation S104, multimodal information fusion can be performed on the first detection result of fundus lesions and at least one of the above-mentioned fundus blood flow abnormality detection results to obtain the second detection result of fundus lesions of the target eye.

[0085] As an optional embodiment, obtaining a second detection result of fundus lesions in the target eye based on the first detection result of fundus lesions and the above-mentioned at least one fundus blood flow abnormality detection result may include: projecting the first detection result of fundus lesions onto a retinal structural layer corresponding to at least one fundus OCTA projection map to obtain fundus lesion detection results on the corresponding at least one retinal structural layer; matching the fundus lesion detection results on the at least one retinal structural layer with the corresponding at least one fundus blood flow abnormality detection result to obtain at least one corresponding matching result; and obtaining a second detection result of fundus lesions in the target eye based on the at least one matching result.

[0086] As mentioned above, in this embodiment, the first detection result of fundus lesions is a three-dimensional fundus OCT image carrying or incorporating lesion detection information, and the at least one fundus blood flow abnormality detection result is a two-dimensional OCTA projection image carrying or incorporating fundus blood flow abnormality detection information. Therefore, the first detection result of fundus lesions and the at least one fundus blood flow abnormality detection result belong to different modalities of data.

[0087] In this embodiment, the first detection result of fundus lesions is projected onto the retinal structural layer corresponding to each of the at least one fundus OCTA projection images mentioned above, to obtain a fundus OCT projection image carrying or incorporating lesion detection information on the corresponding retinal structural layer. This fundus OCT projection image carrying or incorporating lesion detection information is the fundus lesion detection result on the corresponding retinal structural layer. Based on this, the obtained fundus OCT projection image carrying or incorporating lesion detection information is then matched with the fundus blood flow abnormality detection result on the corresponding retinal structural layer to obtain the second detection result of fundus lesions in the target eye. In this way, the matching of detection results from different modalities can be converted into the matching of detection results from the same modality. Thus, the detection of lesion types, i.e., disease categories, under multimodal conditions can be realized, providing more effective and clear auxiliary diagnostic basis for subsequent diagnosis by doctors. For example, this method can be used to detect lesions and diseases with diverse manifestations and complex origins, such as high reflective foci (HRF) and non-serous retinal pigment epithelial detachment (NsPED).

[0088] Taking at least one of the above-mentioned fundus blood flow abnormality detection results, including a surface retinal OCTA projection image, a deep retinal OCTA projection image, an avascular retinal OCTA projection image, and a full-thickness retinal OCTA projection image, as an example, for the first detection result of fundus lesions: it can be projected onto the Enface direction of the surface retina to obtain a surface retinal OCT projection image, which represents the detection result of fundus lesions in the surface retina; it can be projected onto the Enface direction of the deep retina to obtain a deep retinal OCT projection image, which represents the detection result of fundus lesions in the deep retina; it can be projected onto the Enface direction of the avascular retina to obtain an avascular retinal OCT projection image, which represents the detection result of fundus lesions in the avascular retina; and it can be projected onto the Enface direction of the full-thickness retina to obtain a full-thickness retinal OCT projection image, which represents the detection result of fundus lesions in the full-thickness retina. In this way, a two-dimensional mapping of the lesion detection result in the Enface direction can be achieved, ensuring spatial alignment between the fundus lesion detection result and the fundus blood flow abnormality detection result.

[0089] As an optional embodiment, the above-described matching of fundus lesion detection results on at least one retinal structural layer with corresponding fundus blood flow abnormality detection results may include, but is not limited to, feature comparison of the two detection results. For example, feature comparison may include comparing the abnormal blood flow regions in the registered fundus blood flow abnormality detection results of each retinal structural layer and the lesion regions in the registered fundus lesion detection results of each retinal structural layer using pixel-level, region-level, or other pairing methods.

[0090] As an optional embodiment, the above-mentioned matching of fundus lesion detection results on at least one retinal structural layer with corresponding fundus blood flow abnormality detection results may include at least one of the following: matching the position of fundus lesion detection results on at least one retinal structural layer with corresponding fundus blood flow abnormality detection results; matching the morphological features of fundus lesion detection results on at least one retinal structural layer with corresponding fundus blood flow abnormality detection results.

[0091] In this embodiment, different matching thresholds can be set for different lesion types and different blood flow abnormality types. The matching thresholds include, but are not limited to, location overlap thresholds and morphological feature similarity thresholds.

[0092] In this embodiment, for the detection results of fundus lesions and fundus blood flow abnormalities on the same retinal structural layer, the matching or mismatching of the two detection results in terms of location, and the matching or mismatching of the two detection results in terms of morphological features, may correspond to different second detection results of fundus lesions. In this way, lesions can be located more accurately, providing more reliable auxiliary diagnosis for subsequent doctors.

[0093] In some embodiments, decision rules can be constructed based on prior knowledge, and the mapping relationship between different matching results and different second detection results of fundus lesions can be defined in the decision rules. Therefore, obtaining the second detection result of fundus lesions in the target eye based on the matching result can include: searching a pre-constructed decision rule based on the matching result obtained by performing the above matching operation to determine the second detection result of fundus lesions corresponding to the matching result. Prior knowledge includes, but is not limited to, medical experience, medical knowledge rules, expert experience, and prior knowledge from literature.

[0094] For example, for the same retinal structural layer, if the fundus lesion detection result includes HRF, but the fundus blood flow abnormality detection result does not show obvious blood flow signals at the corresponding location, the second fundus lesion detection result is tended to be identified as microglial cell aggregation or hard exudate. However, if the fundus lesion detection result includes HRF, and the fundus blood flow abnormality detection result shows local microvascular dilation / aneurysm at the corresponding location, the second fundus lesion detection result is tended to be identified as microaneurysm or focal leakage point.

[0095] For example, for the same retinal structural layer, if the fundus lesion detection result includes NsPED, but the fundus blood flow abnormality detection result does not detect abnormal blood flow signals at the corresponding location, the second fundus lesion detection result is tended to be identified as including retinal pigment epithelium (RPE) detachment accompanied by choroidal neovascularization (CNV). However, if the fundus lesion detection result includes NsPED, and neovascularization is visible at the corresponding location of the fundus blood flow abnormality detection result, the second fundus lesion detection result is tended to be identified as NsPED containing active neovascularization components. This should be treated differently from NsPED in which the fundus blood flow abnormality detection result does not detect abnormal blood flow signals at the corresponding location, and anti-vascular endothelial growth factor (VEGF) treatment may be required.

[0096] In some embodiments, for the same retinal structural layer, if the corresponding matching threshold is met, the positional matching result between the fundus lesion detection result and the corresponding fundus blood flow abnormality detection result can be used as the lesion location in the second fundus lesion detection result, and the positional matching result and the morphological feature matching result between the fundus lesion detection result and the corresponding fundus blood flow abnormality detection result can be used as the lesion type in the second fundus lesion detection result. The final confidence level of the lesion in the second fundus lesion detection result is determined based on the confidence level of the lesion in the first fundus lesion detection result and the confidence level of the lesion in the fundus blood flow abnormality detection result.

[0097] In some embodiments, for the same retinal structural layer, if the corresponding matching threshold is not met, the first detection result of fundus lesions and the detection result of abnormal fundus blood flow can be retained. That is, the first detection result of fundus lesions is output as a second detection result of fundus lesions, and the detection result of abnormal fundus blood flow is output as another second detection result of fundus lesions.

[0098] As an optional embodiment, obtaining the fundus quantitative indicators of the target eye includes at least one of the following: performing retinal layering on the three-dimensional fundus OCT image based on a retinal layering model to obtain retinal layering results, and obtaining retinal thickness values ​​and / or choroidal thickness values ​​based on the retinal layering results; performing image segmentation on the retinal surface OCTA projection map based on an image segmentation model to obtain the target segmentation region, and obtaining the corresponding retinal quantitative indicators based on the target segmentation region.

[0099] In some embodiments, the retinal layering model can perform retinal layering on three-dimensional fundus OCT images by identifying the retinal and choroidal boundaries, thereby obtaining retinal layering results. Furthermore, the retinal layering model can calculate the corresponding membrane thickness based on the identified retinal and choroidal boundaries, obtaining retinal thickness values ​​and / or choroidal thickness values. In this way, using a big data model to calculate fundus quantitative indicators such as retinal and choroidal thickness values ​​can improve computational efficiency and enhance the accuracy and reliability of the calculation results.

[0100] Taking the calculation of retinal thickness as an example, for an OCT B-scan slice of the macula, a circular partition with a diameter of 1 mm centered on the fovea of ​​the macula can be selected. First, the average number of pixels between the upper boundary line ILM (internal limiting membrane of the retina) and the lower boundary line BM (Bruch's membrane) of the partition is taken, and then the average value is converted into physical size to obtain the corresponding retinal thickness value.

[0101] In some embodiments, the target segmentation region may include at least one of the following: the foveal avascular zone (FAZ) and the fundus non-perfusion zone. The image segmentation model identifies and segments the foveal avascular zone and / or the fundus non-perfusion zone in the OCTA projection image of the retinal surface, and obtains corresponding retinal quantitative indicators based on the segmented image regions. In this way, by using a big data model to segment the foveal avascular zone and the fundus non-perfusion zone, and thereby obtaining the corresponding retinal quantitative indicators, computational efficiency can be improved, and the accuracy and reliability of the calculation results can be enhanced.

[0102] In this embodiment, in addition to performing image segmentation on the surface retinal OCTA projection map and obtaining the corresponding retinal quantitative index, the above-mentioned image segmentation model can also perform image segmentation on at least one of the fundus OCTA projection maps of other structural layers, such as the deep retinal OCTA projection map, the avascular retinal OCTA projection map, and the retinal OCTA projection map, to obtain the corresponding retinal quantitative index.

[0103] For example, such as Figure 7As shown, the operation of processing three-dimensional fundus OCT images using a retinal layering model to obtain fundus quantitative indicators may include: obtaining, for example, from the three-dimensional fundus OCT images... Figure 7 The B-scan slice shown in (a) is as follows; Figure 7 As shown in (b), the B-scan slices are processed for retinal layering; the thickness is calculated based on the retinal layering results, as shown in Figure (b). Figure 7 The values ​​shown in (c) are the central retinal thickness (CST) and choroidal thickness. See also: Figure 7 The process of using an image segmentation model to process OCTA projection images of the fundus to obtain quantitative indicators of the fundus can include: obtaining... Figure 7 The OCTA projection image of the retinal surface shown in (d) is used to segment the acquired image to obtain, as shown in the figure. Figure 7 The image (e) shows the FAZ and the non-perfusion region; based on the image segmentation results, retinal quantitative indicators can be calculated, such as the area, perimeter, and roundness of the FAZ. Figure 7 As shown in (f), these retinal quantitative indicators can be projected and superimposed on the OCTA projection map of the retinal surface.

[0104] It should be understood that the layers of the retina include, but are not limited to, the internal limiting membrane, nerve fiber layer, ganglion cell layer, internal plexiform layer, nuclear layer, external plexiform layer, external nuclear layer, retinal pigment epithelium, external membrane, and choroidal layer. In this embodiment, retinal layering is performed to determine the boundaries between these layers.

[0105] In some embodiments, a retinal layering model can be obtained by training the initial retinal layering model using supervised learning on a third dataset. This third dataset can be obtained by annotating retinal layering lines on a large number of B-scan slices. For example, it can be manually annotated by a qualified physician.

[0106] In some embodiments, an image segmentation model can be obtained by training the initial image segmentation model in a supervised learning manner using a fourth dataset. This fourth dataset can be obtained by annotating a large number of OCTA projection maps of the retinal surface. For example, it can be manually annotated by a qualified physician.

[0107] According to some embodiments, the second detection result of fundus lesions and fundus quantitative indicators can be fused in multiple dimensions to obtain a multidimensional feature vector, and the multidimensional feature vector can be input into the disease detection model to obtain the fundus disease detection result of the target eye.

[0108] In some embodiments, the lesion type, number, location, volume, Enface projection area, and confidence level in the second detection result of fundus lesions can be fused with fundus quantitative indicators such as retinal thickness value, choroidal thickness value, FAZ area, perimeter, and roundness to form a multidimensional feature vector.

[0109] In this embodiment, using fundus quantitative indicators as a supplement to the second detection result of fundus lesions can further clarify the severity and subtype of lesions and obtain more accurate fundus disease detection results.

[0110] In this embodiment, the disease detection model can predict diseases including but not limited to diabetic retinopathy, diabetic macular edema, age-related macular degeneration, myopic macular degeneration, macular hole, vitreous traction syndrome, epiretinal membrane, central serous chorioretinopathy, retinal vein occlusion, and retinal artery occlusion.

[0111] In some embodiments, the fundus disease detection results for the target eye may include, but are not limited to, a set of fundus diseases identified through disease detection and the probability of each fundus disease in the set. For example, the fundus disease detection results for the target eye may include diabetic retinopathy, with a corresponding probability of 80%. In some embodiments, the disease probability can be determined based on the confidence level output by the disease detection model. For example, a confidence level of 0.8 corresponds to a disease probability of 80%.

[0112] In some embodiments, the fundus disease detection results of the target eye can be displayed as auxiliary diagnostic information, or the fundus disease detection results can be further processed to generate auxiliary diagnostic information.

[0113] For example, such as Figure 8 As shown, fundus diseases can be detected through the following procedure:

[0114] In operation S801, a three-dimensional fundus OCT image of the target eye and at least one fundus OCTA projection image are acquired;

[0115] When operating the S802, the lesion detection model is used to detect lesions in three-dimensional fundus OCT images.

[0116] In operation S803, blood flow abnormality detection is performed on at least one fundus OCTA projection image using a blood flow abnormality detection model.

[0117] In operation S804, quantitative analysis is performed on the three-dimensional fundus OCT image of the target eye and at least one fundus OCTA projection image to obtain the fundus quantitative index of the target eye.

[0118] In operation S805, multimodal information fusion is performed on the first detection result of fundus lesions obtained by lesion detection using the lesion detection model and at least one fundus blood flow abnormality detection result obtained by blood flow abnormality detection using the blood flow abnormality detection model.

[0119] In operation S806, the second detection result of fundus lesions obtained by multimodal information fusion and the fundus quantitative indicators obtained in operation S804 are subjected to multidimensional feature fusion to obtain a multidimensional feature vector.

[0120] In operation S807, the obtained multidimensional feature vector is used as an input vector to the disease detection model to detect diseases and obtain the retinal diseases and corresponding confidence scores output by the disease detection model.

[0121] This embodiment utilizes big data models such as lesion detection models, blood flow abnormality detection models, and disease detection models to intelligently analyze multimodal medical imaging data, including three-dimensional fundus OCT images and fundus OCTA projection maps, to quickly obtain accurate and reliable auxiliary diagnostic results, thereby helping doctors to make efficient and accurate medical diagnoses.

[0122] In some embodiments, a disease detection model can be obtained by training the initial disease detection model using supervised learning through the construction of a fifth dataset. This fifth dataset can be obtained by labeling sample data. The initial disease detection model may include, but is not limited to, lightweight deep neural networks, ensemble learning models, etc., and ensemble learning models may include, but are not limited to, XGBoost (eXtreme Gradient Boosting), random forests, etc.

[0123] Through the embodiments of this disclosure, qualitative analysis of fundus lesions in the target eye is performed using lesion detection models and blood flow abnormality detection models, and quantitative analysis of fundus lesions in the target eye is performed by combining fundus quantitative indicators. This enables comprehensive and efficient analysis of fundus images of the target eye. Furthermore, based on the second detection results of fundus lesions obtained from the qualitative analysis and the fundus quantitative indicators obtained from the quantitative analysis, clinicians can be provided with efficient, objective, and highly interpretable fundus disease detection results, thereby achieving auxiliary diagnosis of fundus diseases and improving the diagnostic efficiency and effectiveness of fundus diseases.

[0124] This disclosure also provides a system for processing fundus images.

[0125] Figure 9 This is a schematic diagram of the structure of a fundus image processing system provided according to an embodiment of the present disclosure.

[0126] like Figure 9As shown, the fundus image processing system 900 includes: an image acquisition module 901, an OCT image processing module 902, an OCTA projection image processing module 903, and a processing result acquisition module 904.

[0127] The image acquisition module 901 is configured to acquire a three-dimensional fundus OCT image and at least one fundus OCTA projection image of the target eye; the OCT image processing module 902 is configured to input the three-dimensional fundus OCT image into a lesion detection model to obtain a first detection result of fundus lesions in the target eye; the OCTA projection image processing module 903 is configured to input at least one fundus OCTA projection image into a blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result of the target eye; and the processing result acquisition module 904 is configured to obtain a second detection result of fundus lesions in the target eye based on the first detection result of fundus lesions and at least one fundus blood flow abnormality detection result.

[0128] As an optional embodiment, the system further includes an indicator acquisition module and a disease detection module. The indicator acquisition module is configured to acquire fundus quantitative indicators of the target eye; and the disease detection module is configured to input the second detection result of fundus lesions and the fundus quantitative indicators into a disease detection model to obtain fundus disease detection results of the target eye.

[0129] Optionally, the indicator acquisition module includes a first indicator acquisition unit and / or a second indicator acquisition unit. Acquiring fundus quantitative indicators of the target eye through the indicator acquisition module includes at least one of the following: using the first indicator acquisition unit to perform retinal layering on a three-dimensional fundus OCT image based on a retinal layering model, obtaining retinal layering results, and obtaining retinal thickness values ​​and / or choroidal thickness values ​​based on the retinal layering results; using the second indicator acquisition unit to perform image segmentation on an OCTA projection map of the retinal surface based on an image segmentation model, obtaining a target segmented region, and obtaining corresponding retinal quantitative indicators based on the target segmented region.

[0130] Optionally, the target segmentation region includes at least one of the following: the avascular area of ​​the fovea and the non-perfusion area of ​​the fundus.

[0131] Optionally, at least one fundus OCTA projection image includes at least one of the following: a surface retinal OCTA projection image, a deep retinal OCTA projection image, an avascular retinal OCTA projection image, and a retinal OCTA projection image.

[0132] Optionally, the processing result acquisition module includes: a projection unit, a matching unit, and a processing result acquisition unit. The projection unit is configured to project the first detection result of fundus lesions onto a retinal structural layer corresponding to at least one fundus OCTA projection image, thereby obtaining the corresponding fundus lesion detection result on at least one retinal structural layer; the matching unit is configured to match the fundus lesion detection result on at least one retinal structural layer with the corresponding at least one fundus blood flow abnormality detection result, thereby obtaining at least one corresponding matching result; and the processing result acquisition unit is configured to obtain the second detection result of fundus lesions in the target eye based on at least one matching result.

[0133] Optionally, the matching unit includes at least one of the following: a first matching subunit and a second matching subunit. The first matching subunit is configured to perform positional matching between the detection results of fundus lesions on at least one retinal structural layer and the corresponding detection results of at least one fundus blood flow abnormality; the second matching subunit is configured to perform morphological feature matching between the detection results of fundus lesions on at least one retinal structural layer and the corresponding detection results of at least one fundus blood flow abnormality.

[0134] It should be noted that the explanation of the above-described method for processing fundus images also applies to the fundus image processing system of this embodiment, and will not be repeated here.

[0135] To implement the above embodiments, this disclosure also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the fundus image processing method described in any of the foregoing embodiments.

[0136] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the fundus image processing method described in any of the foregoing embodiments.

[0137] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the fundus image processing method described in any of the foregoing embodiments.

[0138] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0139] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0141] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disks (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, fiber optic devices, and compact disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0143] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0144] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0145] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0146] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for processing fundus images, comprising: Acquire three-dimensional fundus OCT images of the target eye; At least one fundus OCTA projection image is selected, wherein the at least one fundus OCTA projection image corresponds one-to-one with at least one retinal structural layer; The three-dimensional fundus OCT image is input into the lesion detection model to obtain the first detection result of the fundus lesion of the target eye, wherein the first detection result of the fundus lesion is the OCT detection result; The at least one fundus OCTA projection image is input into the blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result for the target eye, wherein the at least one fundus blood flow abnormality detection result is an OCTA detection result; and The first detection result of the fundus lesion is projected onto the retinal structural layer corresponding to the at least one fundus OCTA projection map to obtain the fundus lesion detection result on the corresponding at least one retinal structural layer. The detection results of fundus lesions on at least one retinal structural layer are matched with the corresponding detection results of at least one fundus blood flow abnormality to obtain at least one matching result. Based on the at least one matching result, a second detection result of fundus lesions in the target eye is obtained; Among them, the first detection result of fundus lesions, the at least one detection result of fundus blood flow abnormality, and the second detection result of fundus lesions are used as intelligent auxiliary diagnostic results for doctors' clinical diagnosis.

2. The method according to claim 1, further comprising: Obtain the fundus quantitative indicators of the target eye; as well as The second detection result of the fundus lesion and the fundus quantitative index are input into the disease detection model to obtain the fundus disease detection result of the target eye.

3. The method according to claim 2, wherein, Obtaining fundus quantitative indicators of the target eye, including at least one of the following: Based on the retinal layering model, the three-dimensional fundus OCT image is subjected to retinal layering to obtain retinal layering results, and retinal thickness values ​​and / or choroidal thickness values ​​are obtained based on the retinal layering results. Based on the image segmentation model, the OCTA projection map of the retinal surface is segmented to obtain the target segmentation region, and the corresponding retinal quantification index is obtained based on the target segmentation region.

4. The method according to claim 3, wherein, The target segmentation region includes at least one of the following: the avascular area of ​​the fovea and the non-perfusion area of ​​the fundus.

5. The method according to claim 1, wherein, The at least one fundus OCTA projection image includes at least one of the following: retinal surface OCTA projection image, retinal deep OCTA projection image, retinal avascular layer OCTA projection image, and retinal OCTA projection image.

6. The method according to claim 1, wherein, Matching the detection results of fundus lesions on at least one retinal structural layer with the corresponding detection results of at least one fundus blood flow abnormality includes at least one of the following: The location of the detection results of fundus lesions on at least one retinal structural layer is matched with the corresponding detection results of at least one fundus blood flow abnormality. The morphological features of the detection results of fundus lesions on at least one retinal structural layer are matched with the corresponding detection results of at least one fundus blood flow abnormality.

7. A system for processing fundus images, comprising: The image acquisition module is configured to acquire three-dimensional fundus OCT images of the target eye; At least one fundus OCTA projection image is selected, wherein the at least one fundus OCTA projection image corresponds one-to-one with at least one retinal structural layer; The OCT image processing module is configured to input the three-dimensional fundus OCT image into the lesion detection model to obtain the first detection result of the fundus lesion of the target eye, wherein the first detection result of the fundus lesion is the OCT detection result; An OCTA projection image processing module is configured to input the at least one fundus OCTA projection image into a blood flow abnormality detection model to obtain at least one fundus blood flow abnormality detection result for the target eye, wherein the at least one fundus blood flow abnormality detection result is an OCTA detection result; and The processing result acquisition module includes a projection unit, a matching unit, and a processing result acquisition unit. The projection unit is configured to project the first detection result of the fundus lesion onto the retinal structural layer corresponding to the at least one fundus OCTA projection map, thereby obtaining the fundus lesion detection result on the corresponding at least one retinal structural layer. The matching unit is configured to match the detection results of fundus lesions on the at least one retinal structural layer with the corresponding detection results of at least one fundus blood flow abnormality, and obtain at least one corresponding matching result. The processing result acquisition unit is configured to obtain a second detection result of fundus lesions in the target eye based on the at least one matching result; Among them, the first detection result of fundus lesions, the at least one detection result of fundus blood flow abnormality, and the second detection result of fundus lesions are used as intelligent auxiliary diagnostic results for doctors' clinical diagnosis.

8. An electronic device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method for processing fundus images as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method for processing fundus images as described in any one of claims 1 to 6.

Images