Sdoct modality dr and rvos edema biomarker discovery methods

Abstract

The application discloses an SDOCT modal DR and RVO edema biomarker discovery method. The method first trains a deep learning model using collected SDOCT data to automatically classify SDOCT images of two different retinal diseases. Then, using feature visualization technology, the attention area of the deep model in the classification decision is displayed in the form of a heat map, and the distribution of the attention area of the deep model is counted, and accordingly, potential new biomarkers are proposed to distinguish the two diseases, and the new biomarkers include a group of quantitative features such as retinal outer layer (outer limiting membrane (ELM), elliptical zone (EZ) and retinal pigment epithelium (RPE)) fracture rate, blocking rate and average gray scale. Finally, statistical analysis is performed on these quantitative features, and features with strong statistical differences (p value less than 0.001) are used as the final newly discovered biomarkers.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of marker analysis, and in particular to a deep learning guided SDOCT biomarker discovery method for macular edema in diabetic retinopathy (DR) and retinal vein occlusion (RVO) images. BACKGROUND

[0002] Macular edema is the accumulation of fluid in the macular region of the retina, which can cause moderate to severe visual impairment. Macular edema can occur in a variety of clinical diseases, such as diabetic retinopathy (DR), retinal vein occlusion (RVO), age-related macular degeneration, and pathological myopia, etc. In addition, it is estimated that the prevalence of DR and RVO will increase significantly in the next 30 years, which will bring a heavy social and economic burden to patients and the medical system.

[0003] The spectral domain optical coherence tomography (SDOCT) technology is a fast and non-invasive imaging technology that can obtain high-resolution three-dimensional retinal images and effectively present the structure of each tissue layer of the retina and the clinical and pathological characteristics of the lesion. Clinical experiments show that SDOCT images can quantitatively analyze retinal macular edema and are the best imaging modality for observing the morphological characteristics of macular edema.

[0004] In recent years, some effective macular edema treatment methods have emerged, such as anti-VEGF therapy and dexamethasone intraocular implant. However, some macular edema cases do not respond well to these treatments, and the relationship between macular edema characteristics and treatment response is still unclear because the causes of macular edema are complex and the pathological mechanisms have not been fully understood. Currently, most studies have focused on finding postoperative factors related to vision in DR and RVO patients. However, few studies have attempted to find biomarkers that can distinguish between DR and RVO, which means that the causes of macular edema have not been fully explored. Because the pathophysiology of these two diseases is different, there should be differences in macular edema morphology between DR and RVO. If a biomarker related to the pathogenesis of macular edema can be found and further observed whether it can reflect the efficacy of injection therapy, it may help to clarify the pathophysiology of the disease, thereby helping to predict treatment response and reduce unnecessary treatment. Therefore, focusing on finding disease-specific (i.e., DR or RVO) characteristics of macular edema to improve the understanding of the pathological mechanisms of these two retinal diseases is of great significance for better individualized treatment plans later. SUMMARY

[0005] The purpose of the present application is to provide a biomarker discovery method for macular edema in SDOCT images of diabetic retinopathy and retinal vein occlusion.

[0006] The technical solution for achieving the object of the present application is as follows: an SDOCT modality DR and RVO edema biomarker discovery method, the method comprising the following steps:

[0007] Step 1, collecting SDOCT retinal images of diabetic retinopathy DR and retinal vein occlusion RVO; the SDOCT is frequency domain optical coherence tomography;

[0008] Step 2, constructing and training a deep classification model for automatically classifying SDOCT retinal images of DR and RVO;

[0009] Step 3, for the trained deep classification model, using feature visualization technology to obtain the attention area of the model in the classification decision;

[0010] Step 4, counting the distribution of the attention area in the retina;

[0011] Step 5, according to the distribution, discovering the biomarker for distinguishing DR and RVO.

[0012] Further, the deep classification model in step 2 adopts a deep three-dimensional classification model.

[0013] Step 2, the deep classification model for automatically classifying SDOCT retinal images of DR and RVO is constructed and trained, and the specific process comprises:

[0014] Step 2-1, constructing a deep classification model f θ The model comprises an input layer, a residual connection layer and an output layer connected in sequence;

[0015] The input layer comprises two three-dimensional convolutions, a batch normalization layer and a ReLU activation operation, which are used to increase the number of feature map channels from 1 to 16 and 32 in sequence, and the convolution kernel sizes of the two three-dimensional convolutions are 1×7×7 and 3×1×1, and the strides are 1×2×2 and 1×1×1;

[0016] The residual connection layer comprises four sequentially connected residual blocks, and the number of channel of the output feature map of the residual block is 32, 64, 128 and 256 in sequence, each residual block comprises two three-dimensional convolution blocks, a batch normalization layer and a ReLU activation operation, each three-dimensional convolution block comprises two three-dimensional convolution operations with convolution kernel sizes of 1×3×3 and 3×1×1; the input of each residual block and the result of the input after the convolution operation are summed to realize the residual connection operation, and the sum result is taken as the final output of the residual block; wherein the first residual block does not perform down-sampling operation on the feature map, and the other three residual blocks all perform two times down-sampling operation on the feature map, and the down-sampling operation is realized by two three-dimensional convolutions with sizes of 1×3×3 and 3×1×1 and strides of 1×2×2 and 2×1×1.

[0017] The output layer includes a global average pooling operation and a fully connected layer for outputting final classification results.

[0018] Step 2-2, based on the images collected in step 1, a five-fold cross-validation dataset is constructed, including a training set and a test set;

[0019] Step 2-3, the area under the receiver operating characteristic curve (AUC) is selected as the evaluation index of the classification performance of the model, and the cross-entropy loss is selected for supervised training of the classification model, which is represented as L ce :

[0020]

[0021] Where N is the size of each batch, y i represents the label of the i-th input data, DR is 0 class, RVO is 1 class, p i represents the probability that the i-th input data is predicted to be DR class.

[0022] Step 2-4, the training set is used to train the deep classification model f θ until the preset training threshold is reached, and the trained deep classification model is obtained.

[0023] Further, step 2-4 in the training process further includes four times down-sampling for each two-dimensional retinal image B-scan, and no down-sampling operation is performed in the vertical direction.

[0024] Further, step 2-4 in the training process further includes data augmentation for input data in each training round.

[0025] Further, step 3, for the trained deep classification model, the model is verified in the test set of each fold cross experiment, and the attention area of the model in the classification decision is obtained by using the feature visualization technology, specifically:

[0026] The attention map of the deep classification model is obtained by gradient class activation map (Grad-CAM), and the specific process includes:

[0027] For the deepest image feature map A ∈ R (d,h,w,n) , where d, h, w, and n represent the depth, height, width, and channel number of the feature map, respectively, and the gradient class activation map M c is represented as:

[0028]

[0029]

[0030] where c is the class, c e {1, 2}, l c is the output of the deep model for a class c input, A i is the feature map of the i-th channel, is the pixel point at (e, a, b) in the i-th channel feature map;

[0031] The gradient class activation map M c is up-sampled to the original SDOCT volume data size, and is superimposed on the original SDOCT image in the form of a pseudo-color map to display the attention map of the depth classification model, thereby obtaining the attention area of the depth classification model in the classification decision.

[0032] Further, step 4 describes the distribution of the attention area in the retina, specifically including:

[0033] The classification results predicted correctly by the depth classification model are screened out for class activation map statistics;

[0034] According to the internal limiting membrane ILM, the outer plexiform layer OPL and the Bruch's membrane BM, the retinal area is divided into the inner layer, the outer layer and the whole layer; the inner layer is the ILM to OPL region, the outer layer is the OPL to BM region, and the whole layer is the ILM to BM region;

[0035] The distribution of the attention area in the inner layer, the outer layer and the whole layer of the retina is counted.

[0036] Further, step 5 describes finding biomarkers for distinguishing DR and RVO according to the distribution, specifically including:

[0037] Step 5-1, for the inner layer or outer layer or whole layer area where the attention area is most distributed, the layer fracture rate, layer occlusion rate and average gray value feature in the area are counted;

[0038] The definitions of the retinal layer fracture rate, layer occlusion rate and average gray value feature in a certain area are as follows:

[0039]

[0040]

[0041] Average Grayvalue=mean(grayvalue[layer first : layer last ])

[0042] wherein, disrupted_layer_num is the image frame number of layer disruption of a certain layer of retina, obscured_layer_num is the image frame number of layer obscuration of a certain layer of retina, total_layer_num is the total frame number of an SDOCT volume data, mean represents the average operation, and grayvalue[layer first : layer last ] represents the gray value of the retina region between the first layer layer first and the last layer layer last in a certain region.

[0043] Step 5-2, calculate the significant difference p value of each feature, and select the feature with strong significant difference (p value < 0.001) as the biomarker for distinguishing DR and RVO.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] (1) The present application first provides a biomarker discovery method for macular edema in diabetic retinopathy and retinal vein occlusion SDOCT images guided by deep learning, which breaks through the prior art research.

[0046] (2) The present application extracts a new set of clinical features, specifically the disruption rate, obscuration rate and gray value of the outer layer of the retina (outer limiting membrane (ELM), elliptical zone (EZ) and retinal pigment epithelium (RPE)), which provides an important reference for subsequent medical research.

[0047] The present application will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of the biomarker discovery method for macular edema in an embodiment.

[0049] Figure 2 is the attention region of the deep model for disease classification in an embodiment, wherein Figure 2 (a) to (d) in are respectively RVO volume data, RVO volume data and attention region, one frame of B-scan of RVO volume data, one frame of B-scan of RVO volume data and attention region; Figure 2 (e) to (h) in are respectively DR volume data, DR volume data and attention region, one frame of B-scan of DR volume data, one frame of B-scan of DR volume data and attention region.

[0050] Figure 3 is a schematic diagram of the SDOCT retinal image layer segmentation result and the outer layer disruption and obscuration of retina in an embodiment, whereinFigure 3 (a) is a schematic diagram of inner and outer retinal layers in (b) is a schematic diagram of the external limiting membrane (ELM), ellipsoid zone (EZ), and retinal pigment epithelium (RPE) in (c) and (d) are schematic diagrams of outer retinal layer (ELM, EZ, and RPE) breaks and masking, respectively. Figure 3 (a) is a schematic diagram of inner and outer retinal layers in (b) is a schematic diagram of the external limiting membrane (ELM), ellipsoid zone (EZ), and retinal pigment epithelium (RPE) in (c) and (d) are schematic diagrams of outer retinal layer (ELM, EZ, and RPE) breaks and masking, respectively. Figure 3 (a) is a schematic diagram of inner and outer retinal layers in (b) is a schematic diagram of the external limiting membrane (ELM), ellipsoid zone (EZ), and retinal pigment epithelium (RPE) in (c) and (d) are schematic diagrams of outer retinal layer (ELM, EZ, and RPE) breaks and masking, respectively.

[0051] Figure 4 is a statistical result graph of the attention area of the deep model in an embodiment, wherein Figure 4 (a) and (b) in are the distribution of the attention area of the deep model in the DR and RVO volume data, respectively.

[0052] Figure 5 is a statistical analysis result graph of the outer retinal layer (ELM, EZ, and RPE) break and masking in an embodiment, wherein Figure 5 (a)-(e) in represent the proportion of samples of the outer retinal layer with breaks or masking in the DR and RVO volume data, and are the proportion of the ELM layer break, the EZ layer break, the ELM layer masking, the EZ layer masking, and the RPE layer masking volume data, respectively, Figure 5 (f)-(j) in are the distribution histograms of the outer layer break rate and the masking rate, and are the histograms of the ELM layer break rate, the EZ layer break rate, the ELM layer masking rate, the EZ layer masking rate, and the RPE layer masking rate, respectively. It should be noted that the break rate of the RPE layer is not displayed because the RPE layer break phenomenon is not observed in the DR and RVO groups. In addition, only the samples with broken or masked outer retinal layer are counted in the histogram.

[0053] Figure 6 is a statistical analysis result graph of the gray value between the ELM layer and the RPE layer in an embodiment, which is displayed in the form of a scatter plot (a), a histogram (b), and a box plot (c), respectively.DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0055] In one embodiment, a deep learning guided SDOCT biomarker discovery method for diabetic retinopathy DR and retinal vein occlusion RVO image macular edema is proposed, which combines Figure 1 , including the following steps:

[0056] Step 1, collect SDOCT retinal images of diabetic retinopathy DR and retinal vein occlusion RVO.

[0057] Here, the existing OCT imaging equipment (Carl Zeiss Meditec) is used to collect the retinal images. As shown in the figure, the size of the SDOCT retinal image data is 1024x512x128, corresponding to the area of 2mm (axial z) x 6mm (horizontal direction x) x 6mm (vertical direction y) near the center of the macula of the retina respectively, and each SDOCT data contains 128 two-dimensional retinal images (B-scan, xz plane) with a size of 1024x512. Figure 2

[0058] Step 2, build and train a deep classification model for automatic classification of SDOCT retinal images of DR and RVO. The specific process includes:

[0059] Step 2-1, build a classification model f θ The model includes an input layer, a residual connection layer and an output layer connected in turn, and contains 18 convolutional layers, 1 fully connected layer and 1 global pooling layer, and the specific parameters of the deep model are shown in Table 1.

[0060] Table 1 Classification model parameter table

[0061]

[0062] Among them, the input layer contains two three-dimensional convolution-batch normalization layers- ReLU activation operations, which increase the number of feature map channels from 1 to 16 and 32 in turn, and the convolution kernel size is 1x7x7 and 3x1x1 (in turn corresponding to the depth, height and width direction of the feature map), and the step is 1x2x2 and 1x1x1;

[0063] The residual connection layer contains four sequentially connected residual blocks, and the number of channels of the output feature map of the residual block is 32, 64, 128 and 256 in turn. Each residual block contains two three-dimensional convolution blocks-batch normalization layers- ReLU activation operations, each three-dimensional convolution block contains two three-dimensional convolution operations of 1x3x3 and 3x1x1, the input of each residual block and the result of the convolution operation are summed, the residual connection operation is realized, and the sum result is taken as the final output of the residual block. The first residual block does not perform down-sampling operation on the feature map (depth, height and width direction), and the last three residual blocks all perform two times down-sampling operation on the feature map, and the down-sampling operation is realized by two three-dimensional convolutions with size 1x3x3 and step 1x2x2, and size 3x1x1 and step 2x1x1;

[0064] The output layer contains a global average pooling operation and a fully connected layer, and outputs the final classification result.

[0065] ​Step 2-2, according to the patient independent principle, five-fold cross-validation data sets are constructed, that is, the patients existing in the training set do not appear in the test set, and the SDOCT volume data are randomly divided into training set and test set in a ratio of 4:1.

[0066] Step 2-3, the area under the receiver operating characteristic curve (AUC) is selected as the evaluation index of the classification performance of the model, and the cross-entropy loss is selected to supervise the training of the classification model, which is specifically represented as:

[0067]

[0068] Wherein, N is the size of each batch, y i represents the label of the input data, DR is 0 class, RVO is 1 class, p i represents the probability that the input data is predicted as DR class.

[0069] Step 2-4, the training set is used to train the classification model until the preset training frequency threshold is reached, and the trained deep classification model is obtained.

[0070] Here, the model with the highest AUC value on the test set is selected as the final model of each fold experiment.

[0071] Here, considering the limitation of hardware factors, in the training stage, each B-scan is down-sampled to 266x138 (four times down-sampling), in order to preserve the three-dimensional information, the present application does not perform down-sampling operation in the vertical direction, and the size of the volume data obtained is 266x138x128.

[0072] Here, in the training stage, data augmentation is performed on the input data in each training round. Preferably, the data augmentation method adopted by the present application is random cropping and random rotation, and the down-sampled SDOCT volume data is randomly cropped to 256x128x128 size, and an angle is randomly selected from-15° to 15° along the xz plane to rotate the volume data.

[0073] The present application performs 100 rounds of model training in each fold, the small batch size is selected as 4, the optimizer is selected as Adam optimizer, and the learning rate is kept as 0.0005. In the test stage, the original SDOCT volume data is directly down-sampled to 256x128x128 size without additional data augmentation.

[0074] Step 3, for the trained deep model, the model is verified in the test set of each fold cross-validation experiment, and the feature visualization technology is used to obtain the attention area of the model in the classification decision.

[0075] The present application adopts gradient class activation map (Grad-CAM) to obtain the attention map of the deep model.

[0076] Specifically: for the deepest image feature map A e R (d,h,w,n) where d, h, w, n represent the depth, height, width and channel number of the feature map respectively, the gradient class activation map M c (c is the class, c e {1, 2}) can be represented as:

[0077]

[0078]

[0079] where l c is the output of the deep model for the c class input, is the pixel point located at (e, a, b) on the i-th channel feature map.

[0080] Upsample the gradient class activation map M c to the original SDOCT volume data size, and superimpose it to the original SDOCT image in the form of pseudo-color map to show the attention map of the deep model, as shown in Figure 2 (b), (d), (f), (h).

[0081] Step 4: Statistics of the model attention area obtained in step 3. Since the class activation map result is related to the model predicted class, and the deep model may incorrectly predict the classification result, only the class activation map statistics of the model correctly predicted classification result are performed in the present application. As shown in Figure 3 (a), according to the internal limiting membrane (ILM), the outer plexiform layer (OPL) and the Bruch membrane (BM), the retinal area is divided into the inner layer of the retina (ILM to OPL area), the outer layer (OPL to BM area) and the whole layer (ILM to BM area), and the distribution of the attention area in the inner layer, the outer layer and the whole layer is counted, and the statistical result is shown in Figure 4 .

[0082] Step 5: According to the statistical result of the deep model attention area obtained in step 4, potential biomarkers that can distinguish DR and RVO are found, including a group of quantitative features such as breakage rate, occlusion rate and average gray scale.

[0083] According to the statistical result, it is found that the deep model mostly focuses on the outer layer of the retina. Accordingly, the present application further counts the breakage rate, occlusion rate and gray scale of the outer layer of the retina (external limiting membrane (ELM), elliptical zone (EZ) and retinal pigment epithelium (RPE)) and the differences of the above features between the two diseases, and the qualitative description of the features is shown in Figure 3 . After quantitatively counting the above features, the non-parametric test method Mann-Whitney U test is adopted to calculate the significant difference p value, so as to reflect the difference of the newly found biomarkers between DR and RVO, and the statistical analysis result is shown inFigure 5 and Figure 6 The p-value is shown in the figure (a p-value less than 0.05 is considered statistically significant, and a p-value less than 0.001 is considered to be highly statistically significant).

[0084] The disruption rate of a specific retinal layer is defined as the ratio of the number of image frames with layer disruption in a single SDOCT volumetric dataset to the total number of frames. Similarly, the occlusion rate of a specific retinal layer is defined as the ratio of the number of image frames with layer occlusion in a single SDOCT volumetric dataset to the total number of frames. The specific definitions of the retinal layer disruption rate, occlusion rate, and average gray value are as follows:

[0085]

[0086]

[0087] Average Grayvalue=mean(grayvalue[ELM:RPE])

[0088] Where disrupted_layer_hum is the number of image frames in a certain retinal layer where a layer is broken, obscured_layer_hum is the number of image frames in a certain retinal layer where a layer is obscured, total_layer_hum is the total number of frames in a single volume, mean represents the average value operation, and grayvalue[ELM:RPE] represents the gray value of the retinal region between the ELM layer and the RPE layer.

[0089] As a specific example, the invention will be further verified and illustrated in one embodiment.

[0090] This invention uses SDOCT retinal body data as input and employs deep learning models and statistical analysis methods to discover biomarkers of macular edema in diabetic retinopathy (DR) and retinal vein occlusion (RVO).

[0091] In this embodiment, the size of the SDOCT retinal volume data acquired by the OCT imaging device is 1024×512×128. Figure 2 In the data, (a) and (e) are SDOCT volume data for RVO and DR, respectively. Figure 2 (c) and (g) in the data are B-scan frames from the two volume data. Figure 3 (a) shows the result of the three-layer segmentation of the retina. From top to bottom, the three layers are the internal limiting membrane (ILM), the external plexiform layer (OPL), and the Bruch's membrane (BM). These three layers divide the retinal region (TR) into the inner retinal layer (IR) and the outer retinal layer (OR).

[0092] The three-dimensional SDOCT volume data is input into the deep classification model for model training. After the model training is completed, the model classification decision basis is visualized by using feature visualization technology. As shown in (b) and (f) of FIG. 6, the classification model focuses on the outer retinal structure, and (d) and (h) of FIG. 6 further show the visualization results of the attention area of the model in the two-dimensional layer. Figure 2 Figure 2 As shown in (b) and (f) of FIG. 6, the classification model focuses on the outer retinal structure, and (d) and (h) of FIG. 6 further show the visualization results of the attention area of the model in the two-dimensional layer. Figure 4

[0093] Based on the attention area of the deep model, the application further statistically analyzes a group of clinical parameters of the retina, such as the breakage rate, the occlusion rate and the gray scale of the outer retinal layer (the external limiting membrane (ELM), the elliptical zone (EZ) and the retinal pigment epithelium (RPE)). The layer segmentation results of the ELM, the EZ and the RPE are shown in (b) of FIG. 7, and the qualitative description of the layer breakage and the layer occlusion are shown in (c) and (d) of FIG. 7. The quantitative analysis comparison of the layer breakage and the layer occlusion is shown in FIG. 8. It can be found that the breakage of the ELM layer and the EZ layer has no significant difference between the DR group and the RVO group, and the p values are 0.246 and 0.734 respectively, while the occlusion of the ELM layer, the EZ layer and the RPE layer has significant difference between the DR group and the RVO group, and the p values are all less than 0.001; the occlusion of the outer three layers is more serious in the RVO group. The average gray scale statistics of the ELM layer to the RPE layer are shown in FIG. 9. It is found that the feature has significant difference between the DR group and the RVO group, and the p value is less than 0.001; the average gray scale is lower in the RVO group than in the DR group. Finally, the occlusion of the ELM layer, the EZ layer and the RPE layer and the average gray scale of the ELM layer to the RPE layer are selected as new biomarkers for distinguishing the DR and the RVO. Figure 3 Figure 3 Figure 5 Figure 6

[0094] In summary, based on the distribution of the attention area of the deep learning model, the application finds new biomarkers and extracts a group of clinical features, which are verified and screened by statistical methods. It is confirmed that the screened features have significant difference between the DR and the RVO, and the feasibility of the biomarker discovery method for the macular edema of the DR and the RVO based on the deep learning model is verified.

[0095] ​​​​​​The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method of SDOCT modality DR and RVO edema biomarker discovery, characterized in that, The method comprises the following steps: Step 1, collecting SDOCT retinal images of diabetic retinopathy (DR) and retinal vein occlusion (RVO); the SDOCT is frequency domain optical coherence tomography; Step 2, constructing and training a deep classification model for automatically classifying the SDOCT retinal images of DR and RVO; Step 3, for the trained deep classification model, using feature visualization technology to obtain the attention area of the model in the classification decision; Step 4, counting the distribution of the attention area in the retina; Step 5, according to the distribution, finding biomarkers for distinguishing DR from RVO; Step 4, counting the distribution of the attention area in the retina, specifically comprising: Filtering the classification results predicted correctly by the deep classification model for class activation map counting; According to the internal limiting membrane (ILM), the outer plexiform layer (OPL) and the Bruch's membrane (BM), the retinal area is divided into inner, outer and whole layers; the inner layer is the ILM to OPL area, the outer layer is the OPL to BM area, and the whole layer is the ILM to BM area; Counting the distribution of the attention area in the inner, outer and whole layers of the retina; Step 5, according to the distribution, finding biomarkers for distinguishing DR from RVO, specifically comprising: Step 5-1, for the inner, outer or whole layer of the retina where the attention area is most distributed, counting the layer fracture rate, layer occlusion rate and average gray level features in the area; The definitions of the retinal layer fracture rate, layer occlusion rate and average gray level features in a certain area are as follows: Average Grayvalue = mean (grayvalue [layer first :layer last ]) wherein, disrupted_layer_num is the image frame number of the layer disruption of the retina of a certain layer, obscured_layer_num is the image frame number of the layer obscuration of the retina of a certain layer, total_layer_num is the total frame number of an SDOCT volume data, mean represents the average operation, and grayvalue[layer first :layer last ] represents the gray value of the retina region between the first layer layer first and the last layer layer last of a certain region. Step 5-2, calculating the significance difference p value of each feature, and selecting the features with p value <0.001 as biomarkers for distinguishing DR from RVO.

2. The SDOCT modality DR and RVO edema biomarker discovery method of claim 1, wherein, The deep classification model in step 2 adopts a deep three-dimensional classification model.

3. The SDOCT modality DR and RVO edema biomarker finding method of claim 2, wherein, Step 2, constructing and training a deep classification model for automatically classifying the SDOCT retinal images of DR and RVO, the specific process comprising: Step 2-1, constructing a deep classification model f θ which includes an input layer, a residual connection layer, and an output layer connected in sequence; The input layer includes two three-dimensional convolutions, batch normalization layers and ReLU activation operations, which are used to increase the number of feature map channels from 1 to 16 and 32, respectively, and the convolution kernel size of the two three-dimensional convolutions is 1x7x7 and 3x1x1, respectively, and the step is 1x2x2 and 1x1x1, respectively; The residual connection layer includes four sequentially connected residual blocks, and the number of channels of the output feature map of each residual block is 32, 64, 128 and 256, respectively; each residual block includes two three-dimensional convolution blocks, batch normalization layers and ReLU activation operations; each three-dimensional convolution block includes two three-dimensional convolution operations with convolution kernel sizes of 1x3x3 and 3x1x1, respectively; the input of each residual block and the result of the convolution operation are summed to realize the residual connection operation, and the sum result is taken as the final output of the residual block; wherein the first residual block does not perform down-sampling operation on the feature map, and the other three residual blocks all perform two times down-sampling operation on the feature map, and the down-sampling operation is realized by two three-dimensional convolutions with sizes of 1x3x3 and steps of 1x2x2, and sizes of 3x1x1 and steps of 2x1x1; The output layer includes a global average pooling operation and a fully connected layer for outputting the final classification result. Step 2-2, based on the image collected in step 1, a five-fold cross-validation data set is constructed, including a training set and a test set. Step 2-3, the area under the receiver operating characteristic curve (AUC) is selected as the evaluation index of the classification performance of the model, and the cross-entropy loss is selected to supervise the training of the classification model, which is specifically represented as L ce : where N is the size of each batch, y i represents the label of the i-th input data, DR is the 0 class, RVO is the 1 class, p i represents the probability that the i-th input data is predicted as the DR class; Step 2-4, training the deep classification model f using the training set θ Training is performed until a preset training number threshold is reached, and a trained deep classification model is obtained.

4. The SDOCT modality DR and RVO edema biomarker finding method of claim 3, wherein, Step 2-4, in the training process, also includes four times down-sampling for each two-dimensional retinal image B-scan, and no down-sampling operation in the vertical direction.

5. The SDOCT modality DR and RVO edema biomarker finding method of claim 4, wherein, Step 2-4, in the training process, also includes data augmentation for input data in each training round.

6. The SDOCT modality DR and RVO edema biomarker discovery method of claim 5, wherein, The data augmentation method includes random cropping and random rotation.

7. The SDOCT modality DR and RVO edema biomarker finding method of claim 3, wherein, Step 3, for the trained deep classification model, the feature visualization technology is used to obtain the attention area of the model in the classification decision, specifically: The attention map of the deep classification model is obtained by gradient class activation map Grad-CAM, and the specific process includes: For the deepest image feature map A e R (d,h,w,n) where d, h, w, n represent the depth, height, width and channel number of the feature map respectively, the gradient class activation map M c is represented as: where c is a class, c e {1, 2}, l c is an output of the deep model for a c-class input, A i is a feature map of the i-th channel, is a pixel point at (e, a, b) in the i-th channel feature map. M is the gradient class activation map c Upsampling to the original SDOCT volume data size, and superimposed to the original SDOCT image in the form of pseudo-color map, showing the attention map of the depth classification model, thus obtaining the attention area of the depth classification model in the classification decision.

8. The SDOCT modality DR and RVO edema biomarker discovery method of claim 1, wherein, Step 5-2 specifically adopts the non-parametric test method Mann-Whitney U test to calculate the significant difference p value.

Images