Methods, systems, and apparatuses for segmentation of choroid layer and choroidal vessels

Abstract

The application discloses a choroid layer and choroid blood vessel segmentation method, system and device, and relates to the technical field of image processing. The method comprises the following steps: acquiring a target OCT choroid image; preprocessing the target OCT choroid image to generate a target choroid dataset; inputting the target choroid dataset into a pre-trained ASPP-SE-U-Net network for prediction; the ASPP-SE-U-Net network comprises a choroid CNN model and a choroid blood vessel CNN model; the step of inputting the target choroid dataset into the pre-trained ASPP-SE-U-Net network for prediction comprises the following steps: generating a choroid segmentation prediction map according to the choroid CNN model, generating a choroid blood vessel segmentation prediction map according to the choroid blood vessel CNN model and the choroid segmentation prediction map, superimposing the choroid segmentation prediction map and the choroid blood vessel segmentation prediction map to generate a choroid layer and choroid blood vessel segmentation map. According to the application, the segmentation result is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a choroid layer and choroid vessel segmentation method, system and device. BACKGROUND

[0002] Choroid is an important vascular layer, which mainly provides oxygen and nutrients for the whole eyeball and has the function of light isolation to make the reflected image clearer. Changes in the choroid are crucial for both clinical and research tasks. In particular, in order to compare with age-matched data or previously measured data, it is often necessary to segment the choroid layer and choroid vessels and then measure the choroidal thickness (CT), choroidal volume (CV) and choroidal vascularity index (CVI) to record the normal aging changes of the eye.

[0003] In recent years, the introduction of optical coherence tomography (OCT) allows non-invasive high-resolution images of the retina and choroid to be captured for such analysis. However, manual image segmentation and analysis of large datasets of optical coherence tomography images by human experts is a time-consuming task, and it is necessary to develop automatic analysis methods.

[0004] At present, choroid automatic layer segmentation methods mainly fall into two categories:

[0005] I. Based on image processing method. The method for retinal surface detection is extended to the choroid, and new criteria are proposed to construct a graph model and search the choroid surface; however, this method is too dependent on experimental assumptions, time-consuming, and limits its application in clinical practice; moreover, this method often needs manual intervention or training and testing on a large amount of data, and mainly focuses on normal eyes, and becomes less accurate and robust in the case of pathological eyes.

[0006] II. Based on deep learning method. Deep convolutional neural networks (DCNN) are used as boundary feature detectors or semantic classifiers to segment the choroid layer and choroid vessels. Since DCNN has strong generalization ability, this method can automatically extract the distinguishing features of the choroid layer and choroid vessels, greatly improving the segmentation performance; however, most of the current DCNNs only focus on the segmentation of the choroid layer, and there is less research on the choroid vessels; moreover, this method requires a large amount of training data, which involves obtaining image data and then making label instructions on the data, which requires a lot of preparation work, and the prediction time is long.

[0007] Therefore, the existing methods cannot adapt to the structural deformation of the choroid and choroid blood vessels and the existence of OCT image noise, and have low segmentation accuracy and insufficient generalization ability. SUMMARY

[0008] The technical problem to be solved by the present application is to provide a choroid layer and choroid blood vessel segmentation method, system and device, which can make the segmentation result more accurate.

[0009] To solve the above technical problems, the present application provides a choroid layer and choroid blood vessel segmentation method, comprising: obtaining a target OCT choroid image; preprocessing the target OCT choroid image to generate a target choroid data set; inputting the target choroid data set into a pre-trained ASPP-SE-U-Net network for prediction, the ASPP-SE-U-Net network comprising a choroid CNN model and a choroid blood vessel CNN model, the step of inputting the target choroid data set into the pre-trained ASPP-SE-U-Net network for prediction comprising: generating a choroid segmentation prediction map according to the choroid CNN model, generating a choroid blood vessel segmentation prediction map according to the choroid blood vessel CNN model and the choroid segmentation prediction map, and superimposing the choroid segmentation prediction map and the choroid blood vessel segmentation prediction map to generate a choroid layer and choroid blood vessel segmentation map.

[0010] As an improvement of the above scheme, the step of generating a choroid segmentation prediction map according to the choroid CNN model comprises: predicting the target choroid data set according to the choroid CNN model to generate a choroid prediction probability map; and generating the choroid segmentation prediction map according to a first preset probability threshold and the choroid prediction probability map.

[0011] As an improvement of the above scheme, the step of generating a choroid blood vessel segmentation prediction map according to the choroid blood vessel CNN model and the choroid segmentation prediction map comprises: cropping the target choroid data set using the choroid segmentation prediction map as a mask; inputting the cropped target choroid data set into the choroid blood vessel CNN model to generate a choroid blood vessel prediction probability map; and generating a choroid blood vessel segmentation prediction map according to a second preset probability threshold and the choroid blood vessel prediction probability map.

[0012] As an improvement of the above scheme, the training method of the ASPP-SE-U-Net network comprises: obtaining a benchmark OCT choroid image; preprocessing the benchmark OCT choroid image to generate a benchmark choroid dataset; extracting a training set from the benchmark choroid dataset, and performing data enhancement processing on the training set; constructing an ASPP-SE-U-Net network in a deep learning tool pytorch framework; inputting the training set into the ASPP-SE-U-Net network, calculating the loss between the output of forward propagation and the choroid label, and updating the model parameters of the ASPP-SE-U-Net network through back propagation; iterating the forward propagation and back propagation operations until the model parameters can represent the choroid features, to generate a choroid CNN model and a choroid blood vessel CNN model.

[0013] As an improvement of the above scheme, the construction step of the ASPP-SE-U-Net network comprises: constructing a U-Net network; adding an SE module to the encoder part of the U-Net network; introducing an ASPP module to the encoder part of the U-Net network; and replacing the convolutional layer of the U-Net network with a residual block.

[0014] As an improvement of the above scheme, the preprocessing step of the target OCT choroid image / benchmark OCT choroid image comprises: cropping the target OCT choroid image / benchmark OCT choroid image to remove the eyeball part of the target OCT choroid image / benchmark OCT choroid image; performing noise reduction processing on the cropped target OCT choroid image / benchmark OCT choroid image according to an anisotropic diffusion filtering method; flattening the target OCT choroid image / benchmark OCT choroid image after noise reduction according to a BM boundary as a reference line; finding the upper choroid boundary from the flattened target OCT choroid image / benchmark OCT choroid image according to the Dijkstra algorithm; covering the retinal part above the upper choroid boundary to generate an OCT choroid image containing only the choroid; cropping the region of interest containing the choroid from the OCT choroid image; and performing segmentation processing on the region of interest.

[0015] Correspondingly, the application also provides a choroid layer and choroid blood vessel segmentation system, comprising: a target acquisition module configured to acquire a target OCT choroid image; a target preprocessing module configured to preprocess the target OCT choroid image to generate a target choroid dataset; a probability prediction module configured to input the target choroid dataset into a pre-trained ASPP-SE-U-Net network for prediction, wherein the ASPP-SE-U-Net network comprises a choroid CNN model and a choroid blood vessel CNN model; the probability prediction module comprises: a choroid prediction unit configured to generate a choroid segmentation prediction map according to the choroid CNN model; a choroid blood vessel prediction unit configured to generate a choroid blood vessel segmentation prediction map according to the choroid blood vessel CNN model and the choroid segmentation prediction map; and a superposition unit configured to superimpose the choroid segmentation prediction map and the choroid blood vessel segmentation prediction map to generate a choroid layer and choroid blood vessel segmentation map.

[0016] As an improvement of the above scheme, the choroid prediction unit comprises: a choroid probability subunit configured to generate a choroid prediction probability map by predicting the target choroid dataset according to the choroid CNN model; and a choroid prediction subunit configured to generate the choroid segmentation prediction map according to a first preset probability threshold and the choroid prediction probability map.

[0017] And / or the choroid blood vessel prediction unit comprises: a cropping subunit configured to crop the target choroid dataset by taking the choroid segmentation prediction map as a mask; a choroid blood vessel probability subunit configured to input the cropped target choroid dataset into the choroid blood vessel CNN model to generate a choroid blood vessel prediction probability map; and a choroid blood vessel prediction subunit configured to generate the choroid blood vessel segmentation prediction map according to a second preset probability threshold and the choroid blood vessel prediction probability map.

[0018] As an improvement of the above-mentioned scheme, the choroid layer and choroid blood vessel segmentation system further comprises: a reference acquisition module, configured to acquire a reference OCT choroid image; a reference preprocessing module, configured to preprocess the reference OCT choroid image to generate a reference choroid dataset; an extraction module, configured to extract a training set from the reference choroid dataset, and perform data enhancement processing on the training set; a construction module, configured to construct an ASPP-SE-U-Net network in a deep learning tool pytorch framework; a training module, configured to input the training set into the ASPP-SE-U-Net network, calculate the loss between the output of forward propagation and the choroid label, and update the model parameters of the ASPP-SE-U-Net network through back propagation; and an iteration module, configured to iterate the forward propagation and back propagation operations until the model parameters can represent the choroid features, so as to generate a choroid CNN model and a choroid blood vessel CNN model.

[0019] Correspondingly, the application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the choroid layer and choroid blood vessel segmentation method when executing the computer program.

[0020] The application has the following beneficial effects:

[0021] The application can automatically segment the OCT choroid image by constructing the choroid CNN model and the choroid blood vessel CNN model in the ASPP-SE-U-Net network, effectively overcoming the time-consuming and subjective shortcomings of manual segmentation, and being more efficient than the prior art.

[0022] Further, the application proposes a new workflow of "segmenting the choroid first, then making a mask using the choroid, cutting the input image using the mask, segmenting the choroid blood vessels from the cut image, and finally merging the choroid layer and the choroid blood vessels", which can simultaneously segment the choroid layer and the choroid blood vessels, and cut the image using the mask made from the segmented choroid feature map, so as to make the choroid layer and choroid blood vessel segmentation more accurate.

[0023] In addition, the application uses an atrous spatial pyramid pooling (ASPP) to increase the receptive field, so that the ASPP-SE-U-Net network can identify the sub-choroid boundary according to global information; and uses an SE module to add weights to the image channels, which not only can better fit the complex correlation between channels, but also can greatly reduce the parameter quantity and calculation quantity. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is an embodiment flowchart of the choroid layer and choroid blood vessel segmentation method of the application.

[0025] Figure 2 is a training method flowchart of the ASPP-SE-U-Net network in the choroid layer and choroid blood vessel segmentation method of the present application;

[0026] Figure 3 is a first embodiment mechanism schematic diagram of the choroid layer and choroid blood vessel segmentation system of the present application;

[0027] Figure 4 is a second embodiment mechanism schematic diagram of the choroid layer and choroid blood vessel segmentation system of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings.

[0029] Reference is made to Figure 1 , Figure 1 shows a flowchart of the choroid layer and choroid blood vessel segmentation method of the present application, which comprises:

[0030] S101, obtaining a target OCT choroid image;

[0031] In actual application, the target OCT choroid image can be obtained by the Topcon fundus tomography system 3D OCT-1.

[0032] S102, preprocessing the target OCT choroid image to generate a target choroid data set;

[0033] Specifically, the step of preprocessing the target OCT choroid image comprises:

[0034] (1) performing cropping processing on the target OCT choroid image to remove the eyeball part in the target OCT choroid image;

[0035] (2) performing noise reduction processing on the cropped target OCT choroid image according to an anisotropic diffusion filtering mode;

[0036] The anisotropic diffusion filtering formula is as follows:

[0037]

[0038] Wherein, I is the cropped target OCT choroid image, t is the iteration number, λ is a constant, E is east, S is south, W is west, and N is north. The four divergence formulas are to find the partial derivative of the current pixel in four directions, and the formula is as follows:

[0039]

[0040]

[0041]

[0042]

[0043] cN is the thermal conductivity coefficient of the north direction, cS is the thermal conductivity coefficient of the south direction, cE is the thermal conductivity coefficient of the east direction, and cW is the thermal conductivity coefficient of the west direction, and the formula is as follows:

[0044]

[0045]

[0046]

[0047]

[0048] wherein k is a constant.

[0049] (3) Flattening the target OCT choroid image after noise reduction with the BM boundary as the reference line;

[0050] The BM boundary, also known as the Bruch's Membrane Edge, is a part of the human eye anatomical structure; it is located between the retinal pigment epithelial cells and the choroid of the fundus, and is a layer of basement membrane structure about 2-4 μm thick, which mainly maintains the material exchange balance between the retina and the choroid.

[0051] In the fundus OCT image, the Bruch's Membrane Edge can be used as a reference line to distinguish the choroid layer and the structure within the retina, which is helpful for image segmentation and analysis.

[0052] (4) According to the Dijkstra algorithm, the upper choroid boundary is searched from the flattened target OCT choroid image;

[0053] (5) The retinal part above the upper choroid boundary is covered to generate an OCT choroid image containing only the choroid;

[0054] (6) The region of interest (ROI) containing the choroid is cropped from the OCT choroid image;

[0055] Specifically, the OCT choroid image is divided into three segments in the vertical direction, and the region of interest containing the choroid in the middle is cropped.

[0056] (7) The region of interest is segmented.

[0057] Specifically, the region of interest is divided into 65 blocks with a size of 256*256 in the horizontal direction with a step of 12 to expand the target choroid data set.

[0058] S103, input the target choroid data set into the pre-trained ASPP-SE-U-Net network for prediction;

[0059] It should be noted that the pre-trained ASPP-SE-U-Net network includes a choroid CNN model and a choroid blood vessel CNN model. The step of inputting the target choroid data set into the pre-trained ASPP-SE-U-Net network for prediction includes:

[0060] (1) generating a choroid segmentation prediction map according to the choroid CNN model;

[0061] Specifically, first, the target choroid data set is predicted according to the choroid CNN model to generate a choroid prediction probability map; then, the choroid segmentation prediction map is generated according to the first preset probability threshold and the choroid prediction probability map.

[0062] Preferably, the first preset probability threshold is 0.5, but it is not limited thereto, and can be adjusted according to actual conditions.

[0063] (2) generating a choroid blood vessel segmentation prediction map according to the choroid blood vessel CNN model and the choroid segmentation prediction map;

[0064] Specifically, first, the target choroid data set is cropped by taking the choroid segmentation prediction map as a mask; then, the cropped target choroid data set is input into the choroid blood vessel CNN model to generate a choroid blood vessel prediction probability map; finally, the choroid blood vessel segmentation prediction map is generated according to the second preset probability threshold and the choroid blood vessel prediction probability map.

[0065] Preferably, the second preset probability threshold is 0.5, but it is not limited thereto, and can be adjusted according to actual conditions.

[0066] (3) superimposing the choroid segmentation prediction map and the choroid blood vessel segmentation prediction map to generate a choroid layer and choroid blood vessel segmentation map.

[0067] It should be noted that the superimposition refers to performing logical operation on the pixel points of the two prediction maps (the choroid segmentation prediction map and the choroid blood vessel segmentation prediction map) one by one to obtain the choroid layer and choroid blood vessel segmentation map.

[0068] Therefore, the application proposes a new workflow of'segmenting the choroid first, then making a mask with the choroid, cutting the input image with the mask, segmenting the choroid blood vessels from the cut image, and finally combining the choroid layer and the choroid blood vessels', which can make the segmentation result more accurate.

[0069] As shown in Figure 2 the training method of the ASPP-SE-U-Net network includes:

[0070] S201, obtaining a reference OCT choroid image;

[0071] Similarly, the reference OCT choroid image can be obtained by the Topcon fundus tomography system 3D OCT-1.

[0072] S202, preprocessing the reference OCT choroid image to generate a reference choroid data set;

[0073] Similarly, the preprocessing step of the reference OCT choroid image includes:

[0074] (1) cutting the reference OCT choroid image to remove the eyeball part in the reference OCT choroid image;

[0075] (2) denoising the cut reference OCT choroid image according to the anisotropic diffusion filtering method;

[0076] (3) flattening the denoised reference OCT choroid image with the BM boundary as the reference line;

[0077] (4) finding the upper choroid boundary from the flattened reference OCT choroid image according to the Dijkstra algorithm;

[0078] (5) covering the retinal part above the upper choroid boundary to generate an OCT choroid image containing only the choroid;

[0079] (6) cutting the region of interest containing the choroid from the OCT choroid image;

[0080] (7) segmenting the region of interest.

[0081] S203, extracting a training set from the reference choroid data set and performing data augmentation processing on the training set;

[0082] It should be noted that the reference choroid data set can be divided into a training set, a validation set and a test set according to the ratio of 8:1:1, but it is not limited thereto, and can be adjusted according to actual conditions.

[0083] In practical applications, the test set can be used as the target choroid data set.

[0084] In combination of step S202 and step S203, the application expands data in the manner of data cropping and data enhancement, so that better results are achieved in the case of small data.

[0085] S204, constructing an ASPP-SE-U-Net network in a deep learning tool pytorch framework;

[0086] Unlike the prior art, the ASPP-SE-U-Net network in the application is improved on the basis of the U-Net network, and specifically, the construction steps of the ASPP-SE-U-Net network include:

[0087] (1) constructing a U-Net network;

[0088] (2) adding an SE (squeeze excitation) module to the encoder part of the U-Net network;

[0089] The flow of the SE module mainly includes the following operations:

[0090] The first step is a squeeze operation: the input image is globally flattened and pooled, and after the squeeze operation, the feature map is compressed into a 1x1xC vector, wherein C is the number of channels of the image.

[0091] The second step is an excitation operation: composed of two fully connected layers, wherein SERatio is a scaling parameter, and the purpose of the scaling parameter is to reduce the number of channels to reduce the amount of calculation; the first fully connected layer has CxSERatio neurons, the input is 1x1xC, and the output is 1x1xCxSERadio; the second fully connected layer has C neurons, the input is 1x1xCxSERadio, and the output is 1x1xC.

[0092] The third step is a scale operation: after obtaining the 1x1xC vector, the channel weight values calculated by the SE module are multiplied with the two-dimensional matrix of the corresponding channel of the original feature map respectively, and the output result is obtained.

[0093] (3) introducing an ASPP (Atrous Spatial Pyramid Pooling) module to the encoder part of the U-Net network;

[0094] The ASPP is composed of a hollow convolution, and the flow of the ASPP module mainly includes the following operations:

[0095] The first step: parallel sampling of the input feature map (Input Feature Map) with different sampling rates of the hollow convolution;

[0096] Second step: merge the results obtained together, expand the number of channels;

[0097] Third step: reduce the number of channels to the expected value through 1x1 convolution.

[0098] (4) Replace the convolution layer of the U-Net network with a residual block.

[0099] The residual block takes x as input and H(x) as output, where F(x) is a residual mapping function; when constructing a deep network, the model is difficult to directly fit the actual mapping H(x), and the residual network converts the problem into fitting the residual mapping function F(x) by introducing a shortcut connection, at this time the actual mapping H(x) is represented as H(x) = F(x) + x.

[0100] S205, input the training set into the ASPP-SE-U-Net network, calculate the loss between the output of the forward propagation and the choroid label, and update the model parameters of the ASPP-SE-U-Net network through the back propagation.

[0101] S206, iterate the forward propagation and back propagation operations until the model parameters can represent the choroid features, to generate the choroid CNN model and the choroid blood vessel CNN model.

[0102] Preferably, the number of iterations is 200, but not limited to this, which can be adjusted according to the actual situation. After 200 iterations, the model parameters can represent the choroid features, so as to obtain the choroid CNN model and the choroid blood vessel CNN model.

[0103] Therefore, the application uses the empty space pyramid pooling (ASPP) to increase the receptive field, so that the ASPP-SE-U-Net network can identify the sub-choroid boundary according to the global information, making the segmentation result more accurate; at the same time, the application uses the SE module to add weight to the image channel, which not only can better fit the complex correlation between channels, but also can greatly reduce the parameter amount and the calculation amount.

[0104] Reference Figure 3 , Figure 3 The specific structure of the choroid layer and choroid blood vessel segmentation system 100 of the application is shown, which includes a target acquisition module 1, a target preprocessing module 2 and a probability prediction module 3; the modules will be described in detail as follows:

[0105] I. Target acquisition module

[0106] The target acquisition module 1 is used to acquire the target OCT choroid image; in actual application, the target OCT choroid image can be acquired by the Topcon fundus tomography system 3D OCT-1.

[0107] II. Target preprocessing module

[0108] The target preprocessing module 2 is configured to preprocess the target OCT choroid image to generate a target choroid dataset;

[0109] Further, the target preprocessing module 2 comprises:

[0110] An eyeball cropping unit configured to perform cropping processing on the target OCT choroid image to remove the eyeball part in the target OCT choroid image;

[0111] A denoising unit configured to perform denoising processing on the cropped target OCT choroid image according to an anisotropic diffusion filtering manner;

[0112] A flattening unit configured to flatten the denoised target OCT choroid image with the BM boundary as a reference line. The BM boundary, also known as Bruch's Membrane Edge, is a part of the human eye anatomy. It is located between the retinal pigment epithelial cells and the choroid in the fundus, and is a layer of basement membrane structure about 2-4 μm thick. Its main function is to maintain the material exchange balance between the retina and the choroid. In the fundus OCT image, the Bruch's Membrane Edge can be used as a reference line to distinguish the choroid layer and the structure inside the retina, which helps to perform image segmentation and analysis.

[0113] A searching unit configured to search for the upper choroid boundary from the flattened target OCT choroid image according to the Dijkstra algorithm;

[0114] A masking unit configured to mask the retina part above the upper choroid boundary to generate an OCT choroid image containing only the choroid;

[0115] A region cropping unit configured to crop a region of interest (ROI) containing the choroid from the OCT choroid image. Specifically, the OCT choroid image is divided into three segments in the vertical direction, and a region of interest containing the choroid in the middle is cropped;

[0116] A segmentation unit configured to perform segmentation processing on the region of interest. Specifically, the region of interest is divided into 65 blocks with a size of 256x256 in the horizontal direction with a step of 12 to expand the target choroid dataset.

[0117] III. Probability prediction module

[0118] The probability prediction module 3 is configured to input the target choroid data set into the pre-trained ASPP-SE-U-Net network for prediction, and the ASPP-SE-U-Net network comprises a choroid CNN model and a choroid vessel CNN model.

[0119] The probability prediction module 3 comprises:

[0120] The choroid prediction unit 31 is configured to generate a choroid segmentation prediction map according to the choroid CNN model.

[0121] The choroid vessel prediction unit 32 is configured to generate a choroid vessel segmentation prediction map according to the choroid vessel CNN model and the choroid segmentation prediction map.

[0122] The superposition unit 33 is configured to superimpose the choroid segmentation prediction map and the choroid vessel segmentation prediction map to generate a choroid layer and choroid vessel segmentation map. It should be noted that the superposition refers to logical operation of pixel points of the two prediction maps (the choroid segmentation prediction map and the choroid vessel segmentation prediction map) one by one to obtain the choroid layer and choroid vessel segmentation map.

[0123] Further, the choroid prediction unit 31 comprises:

[0124] The choroid probability sub-unit is configured to predict the target choroid data set according to the choroid CNN model to generate a choroid prediction probability map.

[0125] The choroid prediction sub-unit is configured to generate a choroid segmentation prediction map according to a first preset probability threshold and the choroid prediction probability map. Preferably, the first preset probability threshold is 0.5, but it is not limited thereto, and can be adjusted according to actual conditions.

[0126] Meanwhile, the choroid vessel prediction unit 32 comprises:

[0127] The cropping sub-unit is configured to crop the target choroid data set by taking the choroid segmentation prediction map as a mask.

[0128] The choroid vessel probability sub-unit is configured to input the cropped target choroid data set into the choroid vessel CNN model to generate a choroid vessel prediction probability map.

[0129] The choroid vessel prediction sub-unit is configured to generate a choroid vessel segmentation prediction map according to a second preset probability threshold and the choroid vessel prediction probability map. Preferably, the second preset probability threshold is 0.5, but it is not limited thereto, and can be adjusted according to actual conditions.

[0130] Therefore, the application proposes a new workflow of "segmenting the choroid first, then making a mask with the choroid, cutting the input image with the mask, segmenting the choroid blood vessels from the cut image, and finally combining the choroid layer and the choroid blood vessels", which can make the segmentation result more accurate.

[0131] As shown in Figure 4 The segmentation system 100 of the choroid layer and the choroid blood vessels further comprises:

[0132] The reference acquisition module 4 is configured to acquire a reference OCT choroid image; similarly, the reference OCT choroid image can be acquired by the Topcon fundus tomography system 3D OCT-1.

[0133] The reference preprocessing module 5 is configured to preprocess the reference OCT choroid image to generate a reference choroid dataset; similarly, the structure of the reference preprocessing module 5 is the same as that of the target preprocessing module 2.

[0134] The extraction module 6 is configured to extract a training set from the reference choroid dataset and perform data enhancement processing on the training set; it should be noted that the reference choroid dataset can be divided into a training set, a validation set and a test set according to a ratio of 8:1:1, but this is not a limitation, and the ratio can be adjusted according to actual conditions.

[0135] The construction module 7 is configured to construct an ASPP-SE-U-Net network in a deep learning tool pytorch framework;

[0136] The training module 8 is configured to input the training set into the ASPP-SE-U-Net network, calculate the loss between the output of the forward propagation and the choroid label, and update the model parameters of the ASPP-SE-U-Net network through the back propagation;

[0137] The iteration module 9 is configured to iterate the forward propagation and the back propagation until the model parameters can represent the choroid features, so as to generate a choroid CNN model and a choroid blood vessel CNN model. Preferably, the number of iterations is 200, but this is not a limitation, and the number of iterations can be adjusted according to actual conditions. After 200 iterations, the model parameters can represent the choroid features, so that the choroid CNN model and the choroid blood vessel CNN model are obtained.

[0138] Unlike the prior art, the ASPP-SE-U-Net network in the application is improved on the basis of the U-Net network, and specifically, the construction steps of the ASPP-SE-U-Net network in the construction module include:

[0139] (1) constructing a U-Net network;

[0140] (2) Adding an SE (squeeze excitation) module in the encoder part of the U-Net network;

[0141] The flow of the SE module mainly includes the following operations: the first step is a squeeze operation: the input image is globally flattened and pooled, and after the squeeze operation, the feature map is compressed into a 1x1xC vector, where C is the number of channels of the image. The second step is an excitation operation: composed of two fully connected layers, where SERatio is a scaling parameter, and the purpose of the scaling parameter is to reduce the number of channels to reduce the amount of calculation; the first fully connected layer has CxSERatio neurons, the input is 1x1xC, and the output is 1x1xCxSERadio; the second fully connected layer has C neurons, the input is 1x1xCxSERadio, and the output is 1x1xC. The third step is a scale operation: after obtaining the 1x1xC vector, the channel weight values calculated by the SE module are multiplied by the two-dimensional matrix of the corresponding channel of the original feature map respectively, and the resulting output is obtained.

[0142] (3) Introducing an ASPP (Atrous Spatial Pyramid Pooling) module in the encoder part of the U-Net network;

[0143] The ASPP is composed of a hollow convolution, and the flow of the ASPP module mainly includes the following operations: the first step: parallel sampling of the input feature map (Input Feature Map) with different sampling rates of the hollow convolution; the second step: combine the results obtained together to expand the number of channels; the third step: reduce the number of channels to the expected value through 1x1 convolution.

[0144] (4) Using a residual block to replace the convolution layer of the U-Net network.

[0145] The residual block takes x as input and H(x) as output, where F(x) is a residual mapping function; when constructing a deep network, the model is difficult to directly fit the actual mapping H(x), and the residual network converts the problem into fitting the residual mapping function F(x) by introducing a shortcut connection, at this time the actual mapping H(x) is represented as H(x)=F(x)+x.

[0146] Therefore, the application expands data in a manner of data pruning and data enhancement, so that better results are obtained in the case of small data; in addition, the application uses an Atrous Spatial Pyramid Pooling (ASPP) to increase a receptive field, so that the ASPP-SE-U-Net network can identify a subchoroidal boundary according to global information, so that the segmentation result is more accurate; meanwhile, the application uses an SE module to add weights to image channels, so that not only the complex correlation between channels can be better fitted, but also the parameter quantity and the calculation quantity can be greatly reduced.

[0147] Correspondingly, the application further discloses a computer device, including a memory and a processor, the memory stores a computer program, wherein the processor executes the computer program to realize the steps of the segmentation method of the choroid layer and choroid blood vessels.

[0148] The above is the preferred embodiment of the application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements also as the protection scope of the present application.

Claims

1. A method for segmenting the choroidal layer and choroidal vessels, characterized in that, include: Acquire target OCT choroidal image; The target OCT choroid image is preprocessed to generate a target choroid dataset; The target choroid dataset is input into a pre-trained ASPP-SE-U-Net network for prediction. The ASPP-SE-U-Net network includes a choroid CNN model and a choroidal vessel CNN model. The step of inputting the target choroid dataset into the pre-trained ASPP-SE-U-Net network for prediction includes: A choroid segmentation prediction map is generated based on the choroid CNN model. A choroidal vessel segmentation prediction map is generated based on the aforementioned choroidal vessel CNN model and choroidal segmentation prediction map. The choroidal segmentation prediction map and the choroidal vessel segmentation prediction map are superimposed to generate the choroidal layer and choroidal vessel segmentation map. The training method of the ASPP-SE-U-Net network includes: acquiring benchmark OCT choroidal images; preprocessing the benchmark OCT choroidal images to generate a benchmark choroidal dataset; extracting a training set from the benchmark choroidal dataset and performing data augmentation on the training set; constructing the ASPP-SE-U-Net network in the deep learning tool PyTorch framework; inputting the training set into the ASPP-SE-U-Net network, calculating the loss between the forward propagation output and the choroidal label, and updating the model parameters of the ASPP-SE-U-Net network through backpropagation; iterating the forward and backpropagation operations until the model parameters can represent the choroidal features, to generate a choroidal CNN model and a choroidal blood vessel CNN model. The preprocessing steps for the target OCT choroid image / reference OCT choroid image include: cropping the target OCT choroid image / reference OCT choroid image to remove the eyeball portion; performing noise reduction processing on the cropped target OCT choroid image / reference OCT choroid image using anisotropic diffusion filtering; flattening the noise-reduced target OCT choroid image / reference OCT choroid image using the BM boundary as a baseline; finding the upper boundary of the choroid in the flattened target OCT choroid image / reference OCT choroid image using Dijkstra's algorithm; covering the retinal portion above the upper boundary of the choroid to generate an OCT choroid image containing only the choroid; cropping the region of interest containing the choroid from the OCT choroid image; and segmenting the region of interest.

2. The method for segmenting the choroidal layer and choroidal vessels as described in claim 1, characterized in that, The step of generating a choroid segmentation prediction map based on the choroid CNN model includes: The target choroid dataset is predicted based on the choroid CNN model to generate a choroid prediction probability map. The choroid segmentation prediction map is generated based on the first preset probability threshold and the choroid prediction probability map.

3. The method for segmenting the choroidal layer and choroidal vessels as described in claim 1, characterized in that, The step of generating the choroidal vessel segmentation prediction map based on the choroidal vessel CNN model and the choroidal segmentation prediction map includes: The choroid segmentation prediction map is used as a mask to crop the target choroid dataset; The cropped target choroid dataset is input into the choroidal blood vessel CNN model to generate a choroidal blood vessel prediction probability map. Based on the second preset probability threshold and the choroidal vessel prediction probability map, a choroidal vessel segmentation prediction map is generated.

4. The method for segmenting the choroidal layer and choroidal vessels as described in claim 1, characterized in that, The construction steps of the ASPP-SE-U-Net network include: Construct a U-Net network; Add an SE module to the encoder section of the U-Net network; An ASPP module is introduced into the encoder section of the U-Net network; Replace the convolutional layers of the U-Net network with residual blocks.

5. A system for segmenting the choroidal layer and choroidal vessels, characterized in that, include: The target acquisition module is used to acquire target OCT choroidal images; The target preprocessing module is used to preprocess the target OCT choroid image to generate a target choroid dataset. The target preprocessing module includes: an eyeball cropping unit for cropping the target OCT choroid image to remove the eyeball portion; a noise reduction unit for performing noise reduction on the cropped target OCT choroid image using an anisotropic diffusion filtering method; a flattening unit for flattening the noise-reduced target OCT choroid image using the BM boundary as a baseline; a search unit for finding the upper boundary of the choroid in the flattened target OCT choroid image using Dijkstra's algorithm; a masking unit for masking the retinal portion above the upper boundary of the choroid to generate an OCT choroid image containing only the choroid; a region cropping unit for cropping the region of interest containing the choroid from the OCT choroid image; and a segmentation unit for segmenting the region of interest. The probability prediction module is used to input the target choroid dataset into a pre-trained ASPP-SE-U-Net network for prediction. The ASPP-SE-U-Net network includes a choroid CNN model and a choroidal vessel CNN model. The probability prediction module includes: a choroid prediction unit, used to generate a choroidal segmentation prediction map based on the choroidal CNN model; a choroidal vessel prediction unit, used to generate a choroidal vessel segmentation prediction map based on the choroidal vessel CNN model and the choroidal segmentation prediction map; and an overlay unit, used to overlay the choroidal segmentation prediction map and the choroidal vessel segmentation prediction map to generate a choroidal layer and a choroidal vessel segmentation map. The benchmark acquisition module is used to acquire benchmark OCT choroidal images; A benchmark preprocessing module is used to preprocess the benchmark OCT choroid image to generate a benchmark choroid dataset. An extraction module is used to extract a training set from the benchmark choroid dataset and perform data augmentation processing on the training set; Build modules are used to construct ASPP-SE-U-Net networks within the PyTorch framework, a deep learning tool. The training module is used to input the training set into the ASPP-SE-U-Net network, calculate the loss between the forward propagation output and the choroid label, and update the model parameters of the ASPP-SE-U-Net network through backpropagation. An iterative module is used to iterate the forward and backward propagation operations until the model parameters can represent the choroidal features, so as to generate a choroidal CNN model and a choroidal vascular CNN model.

6. The segmentation system for the choroidal layer and choroidal vessels as described in claim 5, characterized in that, The choroid prediction unit includes: The choroid probability subunit is used to predict the target choroid dataset based on the choroid CNN model and generate a choroid prediction probability map. The choroid prediction subunit is used to generate the choroid segmentation prediction map based on a first preset probability threshold and the choroid prediction probability map. And / or the choroidal vessel prediction unit includes: The cropping subunit is used to crop the target choroid dataset by using the choroid segmentation prediction map as a mask. The choroidal vessel probability subunit is used to input the cropped target choroidal dataset into the choroidal vessel CNN model to generate a choroidal vessel prediction probability map. The choroidal vessel prediction subunit generates a choroidal vessel segmentation prediction map based on a second preset probability threshold and the choroidal vessel prediction probability map.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

Images