Bruch's membrane segmentation in oct volumes

Abstract

The present invention relates to Bruch's membrane segmentation in OCT volumes. Improving retinal layer segmentation in optical coherence tomography (OCT) data by using OCT angiography (OCTA) data to enhance a target retinal layer within the OCT data that can lack sufficient definition for segmentation. The hybrid of OCT data and OCTA data enhances the OCT data, enhances contrast in the OCT data in areas where the OCT and OCTA data are dissimilar, and attenuates contrast in the OCT data in areas where the OCT and OCTA data are similar. The target retinal layer is segmented in the OCT data based on the enhanced data. Two en face images of the OCTA data that include the target retinal layer are used to check the segmentation of the target retinal layer in the OCT data for deviations. The identified deviations are replaced with approximations based on the locations of the top and bottom retinal layers of one of the en face images.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202180027581.6 entitled "Bruch's membrane segmentation in an OCT body", filed on April 28, 2021. Technical Field

[0002] This invention generally relates to the automatic segmentation of retinal layers in OCT data. More specifically, it relates to the segmentation of retinal layers, such as Bruch's membrane and the choroid-sclera interface, which may be missing from the OCT data in conventional automatic segmentation methods, and to a method for automatically identifying and correcting deviations (errors) in retinal layer segmentation. Background Technology

[0003] OCT is a non-invasive imaging technique that uses light waves to generate cross-sectional images of retinal tissue. For example, OCT allows observation of unique tissue layers of the retina. Typically, an OCT system is an interferometric imaging system that produces a three-dimensional (3D) representation of a sample by determining the scattering spectrum of the sample along the OCT beam by detecting the interference of light reflected from the sample with a reference beam. Each scattering spectrum in the depth direction (e.g., z-axis or axial direction) can be individually reconstructed into an axial scan or A-scan. A cross-sectional two-dimensional (2D) image (B-scan) and a related 3D volume (C-scan or cube scan) can be constructed from multiple A-scans acquired as the OCT beam is scanned / moved through a set of lateral (e.g., x-axis and y-axis) positions on the sample. OCT also allows the construction of a planar frontal view (e.g., frontal) two-dimensional image of a selected portion of a tissue body (e.g., a target tissue blank (subbody) or target tissue layer of the retina). OCT angiography (OCTA) is an extension of OCT and can identify (e.g., provided in image format) the presence or absence of blood flow in tissue layers. OCTA can identify blood flow by recognizing differences (e.g., contrast differences) over time in multiple OCT images of the same retinal region and designating differences that meet predetermined criteria as blood flow. A more in-depth discussion of OCT and OCTA is provided below.

[0004] Typically, for diagnostic purposes, identifying retinal layers within OCT data is beneficial for better visualization of specific tissues. Identifying retinal layers in OCT data allows for the definition of a frontal image centered on a specific portion of the B-scan or better based on the selected retinal layer. Manual segmentation of retinal layers is very time-consuming and can be inconsistent. Therefore, automated retinal layer segmentation tools are important for segmenting retinal layers in OCT data; however, the reliability of these tools is affected by degraded OCT data quality and the presence of pathologies that may alter the typical (e.g., expected) shape of the retinal layers. Therefore, the performance of multi-segmentation methods / tools becomes a crucial determinant when evaluating these tools, especially when structurally altered retinal lesions are present (e.g., caused by multiple retinal diseases).

[0005] Ideally, automated multilayer segmentation methods determine the desired layer boundaries without human intervention. However, they are prone to layer misidentification bias, particularly in eyes with moderate to severe retinal lesions and in low-quality OCT data. In these cases, layer boundaries are often unidentifiable automatically or even manually. Layer misidentification can be confounded by OCT imaging biases primarily caused by morphological complexity and variations in the reflectivity of retinal structures in disease cases, such as weak signal quality, eye movement, and decentration. Accurate multilayer segmentation in these cases exceeds the capabilities of conventional segmentation algorithms.

[0006] Manual multi-layer segmentation by human graders typically requires the grader to identify layers through sketching or by setting points for interpolation or fitting algorithms. Compared to automated methods, manual methods are time-consuming, labor-intensive, and have high intra-grader variability. Sometimes, manual segmentation is impossible due to loss of retinal layers or boundaries.

[0007] Automatic retinal layer segmentation is particularly challenging when attempting to segment retinal layers that are not typically well-defined in OCT data (e.g., structural data), such as Bruch's membrane (BM) or the choroidal capillary layer. The choroidal capillary layer is extremely thin, and it often helps to produce a frontal image for 3D OCTA data imaging to distinguish choroidal capillary layer features. In cases of age-related macular degeneration (AMD) with choroidal warts (depositions of lipoproteins (lipids) accumulating under the retina) or retinal pigment epithelial detachment (PED), segmentation of the retinal pigment epithelium (RPE) is generally unusable for producing a frontal image of the choroidal capillary layer. Therefore, accurate BM segmentation can be crucial for distinguishing choroidal capillary layer features. Unfortunately, BM segmentation can be difficult due to: low contrast in structural OCT data in the BM area, decorrelation tailing problems in OCTA data at the RPE and BM, and / or distorted signal around the BM in pathological cases. In Schottenhamml et al., "OCT-OCTA segmentation: a novel framework and an application to segment Bruch's membrane in the presence ofdrusen", Invest Ophthal Vis Sci. In 2017; 58(8): 645–645, it was proposed to combine OCT structural data and OCT flow data to identify Bruch's membranes. Although the details of their method are not readily apparent, Schottenhamml et al. appear to have used an automated OCT-OCTA map segmentation algorithm to segment Bruch's membranes in the presence of choroidal warts.

[0008] The object of this invention is to provide an automated method / system for providing more reliable segmentation, where reliable segmentation of retinal layers such as Bruch's membrane and choroidal capillary layers has been previously difficult.

[0009] Another object of the present invention is to provide an automated method / system for retinal layer segmentation in the presence of pathology.

[0010] Another object of the present invention is to provide an automatic method / system for identifying deviations in retinal layer segmentation.

[0011] Another object of the present invention is to provide an automated method / system for replacing deviations in retinal layer segmentation with an approximation method. Summary of the Invention

[0012] The above objectives have been achieved in a method / system for the automatic segmentation of Bruch's membrane (BM) and other retinal layers in optical coherence tomography (OCT) data.

[0013] Because they provide different types of information, OCT structural data is fundamentally different from OCTA flow data. Therefore, their respective images can be quite different, especially in the upper retinal layers where OCT typically provides excellent structural information. In these upper layers, OCT and OCTA data can be quite distinct. OCT data generally loses definition in the lower retinal layers. However, the applicant has noted that in these lower retinal layers, OCT data can appear similar to OCTA data. This invention utilizes this observation to better emphasize / define the shift from when OCT data differs significantly from OCTA data to where OCT data becomes more similar to OCTA data. Specifically, the choroid and scleral regions are similar in both structural OCT and flow OCTA images. By noting where OCT data resembles the corresponding OCTA data within the slab, the choroid and scleral regions can be removed from or thinned (or otherwise defined / delineated) within the structural OCT (using the corresponding OCTA data). In this approach, the structural OCT data around Bruch's membrane can be enhanced after the choroidal region has been thinned. Therefore, the enhanced OCT data simplifies the segmentation problem considerably. Although this method is particularly beneficial in the lower retinal layer, it can also be applied to other target retinal layers.

[0014] This invention improves the segmentation of retinal layers in OCT data by using OCTA data to enhance the target retinal layer (e.g., BM, choroid-scleral interface, etc.) or the region surrounding the target retinal layer within the OCT data, where the OCT data might otherwise lack sufficient definition for segmentation. OCT data can be enhanced based on a mixture of OCT and OCTA data. In one example, the contrast of OCT data in regions where OCT and OCTA data are dissimilar (e.g., above the target retinal layer) can be enhanced, while the contrast in regions where OCT and OCTA data are similar (e.g., below the target retinal layer) can be reduced. In another example, the contrast of OCT data around the retinal layer of interest (e.g., Bruch's membrane) can be enhanced by leveraging the similarity and dissimilarity between OCT and OCTA. Enhancing OCT data may include subtracting a proportion (e.g., a weighted) of the mixture of OCT and OCTA data from the OCT data, where the proportion may be based on a ratio of a measure of the joint variability (e.g., statistical covariance) of the OCT and OCTA data to a measure of the data distribution (e.g., statistical variance) of the OCT and OCTA data. Therefore, enhanced OCT data can improve the delineation of the target retinal layer. The target retinal layer in the OCT data can then be segmented based on the enhanced data. Other layers can then be segmented relative to the target retinal layer.

[0015] Although this invention provides improved automatic retinal segmentation, any segmentation method is prone to bias. Therefore, this invention also provides an automatic segmentation bias detection and correction or approximation method. That is, this invention provides a method for identifying retinal layer segmentation failures and replacing them with segmentation approximations. In a specific embodiment, segmentation failures in OCT structural data are automatically identified using an angiography (OCTA) retinal layer blank (e.g., an OCTA frontal image). Two frontal images of the OCTA data, including the target retinal layer (to be checked for failure), are used for bias checking of the segmentation of the target retinal layer in the OCT data. For example, the target retinal layer may be the inner retinal layer (IPL), outer retinal layer (OPL), etc. Alternatively, the first and second frontal images can be defined from the respective blanks from the OCT data.

[0016] The determination of successful or failed segmentation of the target retinal layer is based on a local or global similarity measure between the first and second frontal images. For example, the similarity measure could be based on normalized cross-correlation (NCC) between the two frontal images. Identified local or global biases can be automatically replaced by approximations based on the positions of the top and bottom retinal layers in one of the two frontal images. For example, the target retinal layer could be sandwiched between the top and bottom layers of a blank defining the second frontal image (the blank defining the first frontal image could have the same top layer, but its bottom layer could be defined by the target retinal layer), and the approximation could be based on a weighted combination of the top and bottom layers of the blank defining the second frontal image. The top and bottom layers of the second frontal image can be selected based on confidence measures that these top and bottom layers are bias-free. For example, these top and bottom layers can be selected based on abrupt transitions from bright to dark or from dark to bright in OCT data. The weighted combination can be based on the positions of these top and bottom layers relative to the expected position of the target retinal layer sandwiched between them.

[0017] Other objects and achievements of the invention, as well as a more complete understanding, will become apparent and understood by taking into account the accompanying drawings, the following description, and the appended claims.

[0018] Several publications may be cited or referenced herein to aid in understanding the invention. All publications cited or referenced herein are incorporated herein by reference in their entirety.

[0019] The embodiments disclosed herein are merely examples, and the scope of the invention is not limited thereto. Attached Figure Description

[0020] This patent or patent application includes at least one color drawing. In the drawing, similar reference numerals / text indicate similar parts: Figure 1 An example of seven retinal layers that can typically be automatically segmented using existing multilayer segmentation tools is provided.

[0021] Figure 2 The structural OCT B scan and corresponding OCT B scan of the eye with geographic atrophy (GA) are shown.

[0022] Figure 3 The structural OCT B scan and corresponding OCT B scan of an eye with age-related macular degeneration (AMD) are shown.

[0023] Figure 4 This demonstrates the application of a two-stage segmentation method to enhance data according to the present invention.

[0024] Figure 5ATwo examples of enhanced OCT data used for BM imaging are shown.

[0025] Figure 5B Three examples of BM segmentation with corresponding choroidal capillary layer vascular structure diagrams are shown.

[0026] Figure 6 The mean absolute difference (including 95% confidence interval) between the two readers and between the reader and the BM split is shown, along with R. 2 .

[0027] Figure 7 The image shows a choroidal thickness map (in micrometers) of the right eye generated using manual (left) and automatic (middle) segmentation, with superimposed ETDRS grids located at the center of the fovea, and a structural choroidal vascular outline (right).

[0028] Figure 8 A table is provided that displays information extracted from regression and Bland Altman analysis for each part of the ETDRS grid.

[0029] Figure 9 Exemplary ILM-IPL frontal angiography blank images and exemplary ILM-OPL frontal angiography blank images are shown, defined by the segmentation algorithm output from the automatic segmentation tool.

[0030] Figure 10 Provided display Figure 9 A plot of normalized cross-correlation (NCC) of all rows (B-scan) compared in the frontal image.

[0031] Figure 11 The image shows B-scan and ILM-IPL front panel blank images, where IPL segmentation of the volume data is replaced by the IPL segmentation approximation method.

[0032] Figure 12 Two examples of SRL slabs superimposed on two B-scans using MLS IPL 73a segmentation and IPL approximation 73b are shown, where the MLS IPL segmentation is incorrect.

[0033] Figure 13 The image shown is an example of an image where automatic segmentation at the IS / OS junction failed (FOV=8mm).

[0034] Figure 14 The image shown is one where automatic segmentation failed in the central concave region (FOV=16mm).

[0035] Figure 15An example of an OCT B scan (FOV=12mm) with overlapping segments (from top to bottom) representing the boundaries of multiple retinal layers is shown, which is similar to Figure 19 Those shown.

[0036] Figure 16 The general process of an automated method based on the segmentation extension according to the present invention is shown.

[0037] Figure 17 A semi-automated method based on segmentation extension is shown.

[0038] Figure 18 A generalized frequency-domain optical coherence tomography system suitable for collecting 3D image data of the eye, applicable to the present invention, is shown.

[0039] Figure 19 An exemplary OCT B scan image of a normal retina of the human eye is shown, and several typical retinal layers and boundaries are exemplarily identified.

[0040] Figure 20 An example of a frontal vascular system image is shown.

[0041] Figure 21 An exemplary B-scan image of the vascular structures (OCTA) is shown.

[0042] Figure 22 The example computer system (or computing device or computer) is shown. Detailed Implementation

[0043] Accurate detection of anatomical and pathological structures in optical coherence tomography (OCT) images is crucial for the diagnosis and research of retinal diseases. Manual segmentation of the features of interest in the B-scan of each OCT volume scan requires not only professional graders but is also extremely time-consuming for clinical use. Another problem is the inherent bias among graders, leading to subjective segmentation results. Fully automated methods for segmenting multiple retinal layer boundaries in B-scans can significantly reduce the processing time required for segmentation.

[0044] Using automated multi-layer segmentation methods to segment retinal layers offers several advantages. For example, these tools can save on lengthy preprocessing steps such as noise reduction, resampling and normalization, and OCT cube flattening. Additionally, they allow for the construction of additional information on identified segmented layers, such as referencing unknown layers to one or more known layers, identifying layers by combining their two adjacent layers with annotations, and identifying smaller regions for processing. Automated multi-layer segmentation tools can also facilitate the implementation of other analytical tools, such as multiple thickness maps (macula, RNFL, and ganglion cell thickness) and frontal imaging, such as structural and angiographic frontal images. It can also be used as input for other algorithms, such as foveafinders, OCTA decorrelation tail removal, and CNV finding algorithms.

[0045] Figure 1 Examples of seven retinal layers typically segmented automatically using existing multilayer segmentation tools are provided. While current tools can successfully segment the supraretinal layers automatically, as discussed below, references... Figure 19 There are other retinal layers, such as the Bruch's membrane and choroidal region, that typical automated multisegmentation tools cannot reliably segment or identify. These layers are often the lower layers where the OCT signal may be weaker and tend to have higher levels of artifacts, and / or layers with pathological changes to the typical retinal layer structure of a healthy retina.

[0046] Therefore, the performance of multi-segmentation tools becomes a crucial determinant when structurally altered retinal lesions caused by multiple retinal diseases are present. Existing automated multi-segmentation tools suffer from two main problems. First, they tend to exhibit segmentation bias, particularly in eyes with moderate to severe retinal lesions. Second, most existing methods are computationally extremely expensive and can take minutes to hours to compute retinal segmentation on large OCT data cubes. Segmentation bias can be compounded by OCT imaging biases such as weak signal quality, eye movement and morphological complexity, and reflectivity variations in retinal structures caused by disease. Multi-segmentation in these situations exceeds the capabilities of conventional segmentation algorithms.

[0047] This invention provides a method and system for automatically segmenting retinal layers, such as the Bruch's membrane (BM) and the choroid, which are not typically included in automated segmentation tools. To better segment these layers, this invention enhances the contrast of structural OCT B scans by using appropriate OCT B scans around the BM (or other target retinal layers) without any prior segmentation, by removing or attenuating OCT signals from portions of the BM such as the choroid and sclera.

[0048] Figure 2The structural OCT B scan 11 and corresponding OCTA B scan 13 of an eye with geographic atrophy (GA) are shown. As shown in box 15, the OCT data 11 and OCTA data 13 are blended, and then, as shown in box 17, the resulting blended signal is subtracted from the OCT data 11 to produce enhanced structural data / image 19. The enhanced structural image 19 shows much higher contrast, especially in the GA region, which allows segmentation of the BM even within this pathological region. Due to the low contrast in the OCT around the BM and the decorrelation tails in the OCTA, BM segmentation using OCT or OCTA without normalization would be challenging, and no clear interface would be available around the BM layer.

[0049] Figure 3 The structural OCT B scan 21 and corresponding OCT B scan 23 of an eye with age-related macular degeneration (AMD) are shown. Similarly, as shown in box 25, the OCT data 21 and OCTA data 23 are blended, and they are subtracted from the OCT data 21 (e.g., box 27) to produce an enhanced structural image / data 29. It can be seen that the enhanced structural image 29 shows higher contrast below the retinal pigment epithelial (RPE) retinal layer.

[0050] The generation of enhanced structural images 19 / 29 can constitute all or part of the first stage of a two-(or more)-stage image segmentation method. That is, after the enhanced OCT data is generated in the first stage, any suitable segmentation method / algorithm can then be applied to the enhanced OCT data (e.g., 19 and / or 29) in one or more successive stages to provide automatic (or semi-automatic) segmentation.

[0051] This paper presents several common frameworks and methods for automatic and semi-automatic multi-level segmentation. These are combined as follows: Figures 13 to 17One such automatic segmentation method discussed is relatively fast and can be used for commercial products. This method automatically or manually identifies the starting position (e.g., extending the starting position and / or B-scans) and, for example, extends multi-layer segmentation information to adjacent B-scans and / or the nearest B-scan (e.g., B-scans within a predetermined distance from the current position (e.g., from the starting position) to another B-scan). Another segmentation method suitable for use with the two (or more)-stage image segmentation method of the present invention can use a graph search algorithm based on contrast-enhanced structural OCT data. Regardless of the segmentation method used, each stage can apply segmentation to different resolutions of the enhanced (OCT) data. That is, the enhanced (OCT) data can be downsampled to different resolution levels, and each resolution level can be submitted to a different stage of the segmentation method, wherein the first stage of the segmentation method segments the lowest resolution, and the output of each stage is the starting segmentation for the next stage, and the resolution of the next stage is equal to or higher than that of the previous stage.

[0052] In the exemplary two-(or more)-stage segmentation method of the present invention, in the first stage, the selected image segmentation method used produces a preliminary coarse segmentation result. Thereafter, by using the segmentation baseline of the first stage as the initial segmentation, the second stage of the segmentation method can be initiated by segmenting below the baseline (using any suitable segmentation method) to obtain the final segmentation result. Due to the suitable initialization (e.g., initial segmentation) from the first stage, the second (and subsequent) stages can achieve the desired segmentation result, even for difficult images.

[0053] Figure 4 This demonstrates the application of the two-stage segmentation method of the present invention to augmented data. Embodiments of the present invention may include the following steps: 1) Enhance structural OCT by removing or weakening the choroid region in the structural OCT body using the corresponding OCT body.

[0054]

[0055] V s : Structure OCT body

[0056] V a : OCTA body

[0057] V e : Enhanced structure OCT body

[0058] w 1 , w 2 Weights of OCT and OCT bodies

[0059] f Objective function (e.g., normalized cross-correlation, squared interaction information)

[0060] α Parameters to be optimized

[0061] Solution ,in Cov It is covariance and Var It is the variance, provided that the objective function f is the square of the normalized cross-correlation.

[0062] 2) The first stage of the initial segmentation is achieved by V with high confidence. e The segmentation of each B-scan and the subsequent baseline calculation constitute the second stage of segmentation. The baseline is used for segmentation.

[0063] exist Figure 4 In the image, image 31 shows an enhanced image with high confidence (e.g., V). e The initial segmentation 31a on the ) and the baseline 31b calculated based on the 2-D fitting.

[0064] 3) The second stage consists of a final segmentation performed below the initial segmentation. Figure 4 In the image 33, the enhancement (e.g., V) is shown. e The final segmentation for plotting is shown in Figure 35. This segmentation can then be transferred to the original OCT data (e.g., structural B scan), as shown in Figure 37.

[0065] In embodiments of the present invention, the segmentation method in the first and second stages is a graph search algorithm, but other segmentation methods may also be used.

[0066] As mentioned above, this invention can be used in automated Bruch's membrane segmentation methods in optical coherence tomography (OCT). Accurate Bruch's membrane (BM) segmentation is essential for characterizing possible choroidal capillary layer loss and elevation and dysfunction of retinal pigment epithelial cells (important diagnostic signals of retinal diseases). The BM segmentation method / system of this invention can be applied to OCT.

[0067] The exemplary BM segmentation method of the present invention uses a structure (V) s ) and flow (V) a The OCT volume is used to enhance the BM layer. The enhanced OCT volume (V) is calculated by subtracting a certain proportion of the mixture of structural and flow data from the structural data. e ), such as V e =V s -α(w s V s +wa V a ), where w s and w a This is the weight (e.g., set to 0.5). The scaling factor α can be defined as α = Cov(w s V s +w a V s V s ) / Var(w s V s +w a V a Assume V e and mixture (w s V s +w a V a The similarity (squared normalized cross-correlation) between the segments is minimized. This segmentation method is based on multiresolution and graph search algorithms. The segmentation baseline at each resolution level is used as the starting segment for the next higher resolution segmentation. In an example of the invention, for fast processing, the number of resolution levels is set to 2. The algorithm's performance is evaluated by comparing it with 120 B scans extracted from 40 OCTA cube scans of 3×3mm, 6×6mm, 9×9mm, 12×12mm, and 15×9mm acquired using a 200kHz PLEX® Elite 9000 (ZEISS, Dublin, CA), manually edited by two readers. All scans are a mixture of disease lesions such as DR and AMD.

[0068] Figure 5A The structure (V) is shown. s ) and flow (V) a ) data, for enhanced OCT data of BM imaging (V e Two instances 41a and 41b, and Figure 5B Three examples 43a, 43b, and 43c are shown, illustrating BM segmentation 45a, 45b, and 45c using the corresponding choroidal capillary layer vascular structures 47a, 47b, and 47c. Examples 41a and 41b demonstrate the use of the corresponding OCT (V) structure. s ) and flow (V) a ) Enhanced OCT B-scan (V) e Examples 43a, 43b, and 43c show the segmentation results 45a, 45b, and 45c of the corresponding choroidal capillary layer blanks 47a, 47b, and 47c using OCTA flow data between BM and BM+20μm (e.g., the OCTA blank defined between them).

[0069] Figure 6The mean absolute difference (including 95% confidence interval) between the two readers and between the reader and the subject's BM segmentation is shown, along with R. 2 The mean absolute difference for each scan pattern confirms a strong correlation and high consistency between the reader and the BM segmentation.

[0070] Overall, automated and manual segmentation showed a strong correlation and high consistency. Automated segmentation can be a valuable diagnostic tool for retinal diseases.

[0071] Another exemplary embodiment illustrates a method for segmenting the choroid-scleral interface in optical coherence tomography (OCT). A relatively fast algorithm was developed to segment the choroid-scleral interface. The segmentation of the present invention begins with a B-scan with high contrast around the choroid-scleral boundary. The segmentation is then extended to the entire volume data. The algorithm uses intensity and gradient images as inputs to a map-based approach for segmenting each B-scan in the region of interest. The performance of the algorithm was evaluated using normal SS-OCT volume data of 49 500×500 A-scans within a 12×12 mm area acquired using a PLEX® Elite 9000 SS-OCT (ZEISS, Dublin, CA). Choroidal thickness maps, defined as the distance between the fitted RPE baseline and the choroid-scleral interface, were generated using manual and automatic segmentation. Regression and Bland Altman analyses were performed on each portion of the ETDRS grid to report the performance using this embodiment of the invention. Figure 7 The images show choroidal thickness maps (in micrometers) of the right eye generated using manual (left) and automatic (middle) segmentation, featuring superimposed ETDRS grids centered at the fovea, and a structural choroidal vascular map (right). The ETDRS grid consists of three concentric circles with radii of 2, 4, and 6 mm, centered at the fovea. The images were generated through both manual and automatic segmentation. Figure 7 The thickness diagram of the choroid and the corresponding structural choroidal vessels.

[0072] Figure 8 Tables are provided showing the information extracted from regression and Bland-Altman analyses of each section of the ETDRS mesh. Regression and Bland-Altman analyses of all sections confirm a strong correlation and good consistency between the manual and automated methods. Using an Intel i7 CPU, 2.7 GHz, and 32 GB of memory, the average processing time was less than 4 seconds. Overall, the choroid thickness maps generated by automatic and manual segmentation show a strong correlation and good fit.

[0073] As shown, the present invention provides good results in automated segmentation systems; however, as mentioned above, automated segmentation systems can occasionally produce erroneous results due to a variety of factors beyond the control of the automated segmentation system. For example, segmentation deviations can be caused by weak OCT signal quality, eye movement or morphological complexity, and variations in the reflectivity of retinal structures in disease cases. In view of these problems associated with automated (and manual) segmentation systems, the present invention also proposes a method for identifying retinal layer segmentation failures and replacing them with segmentation approximations.

[0074] In the past, segmentation confidence at each layer's points has been used to determine segmentation quality. Typically, segmentation confidence is determined based on the intensity of the cost image (e.g., gradient image) at a given segmentation point. This method may not work well because a segmentation can jump to adjacent layer segments and still have high confidence.

[0075] In this invention, OCTA flow data can be used to determine the segmentation quality of OCT structural data. Alternatively, OCT data can also be used in this proposal. In an example of this invention, the similarity of the OCTA vascular structure slab can be used as an indication of segmentation failure in a specific layer. Segmentation failures can be locally identified and replaced with approximations.

[0076] This invention employs an angiographic retinal layer preform for automatic identification of segmentation failures. For example, the inner limiting membrane (ILM) and outer retinal layer (OPL) can be used to detect IPL (IPL) segmentation failures to generate ILM-IPL and ILM-OPL angiographic (or structural) preforms, assuming that the IPL and ILM segmentation are reasonably correct. For this purpose, using ILM and OPL segmentation generally performs better than using the retinal nerve fiber layer (RNFL), IPL, and inner nuclear layer (INL) because the changes in ILM from multiple (dark) to RNFL (bright) and from OPL (bright) to avascular areas (dark) are more abrupt. In this instance, if the IPL segmentation works reasonably well, a high degree of local similarity is expected between the ILM-IPL and ILM-OPL angiographic preforms. Local similarity is an indicator of IPL segmentation failure. Preforms can be generated based on the maximum extension within the layer boundaries defined in the OCTA volume data (or other suitable methods for defining the frontal image). For IPL segmentation failures, an IPL approximation method is used instead of IPL segmentation, based on a weighted average of ILM and OPL. The same method can be used to identify OPL segmentation failures (or other target retinal layer failures) by appropriately selecting a reference layer segmentation. For example, if the segmentation of these layers is correct, IPL or IS / OS segmentation can be used as the reference layer segmentation.

[0077] In summary, embodiments of the present invention can automatically identify local segmentation failures at the retinal layer boundaries using angiographic retinal layer blanks. In the following examples, under the assumption that IPL and ILM segmentation are acceptable, ILM and outer OPL layers are used to detect IPL layer segmentation failures to generate ILM-IPL and ILM-OPL angiographic blanks. For this purpose, ILM and OPL segmentation generally perform better than other internal retinal layer segmentations (such as RNFL, IPL, and INL) because the changes in ILM from multiple (dark) to RNFL (bright) and from OPL (bright) to avascular areas (dark) are more abrupt.

[0078] If IPL segmentation works reasonably well, a high degree of local similarity is expected between the ILM-IPL and ILM-OPL angiographic frontal images (or slabs). Local similarity is an indicator of failed IPL segmentation. In an example of this invention, the frontal slab image is generated based on the maximum protrusion within the layer boundaries defined in the OCTA volume data. If IPL segmentation fails, an IPL approximation can be used instead, based on a weighted average of ILM and OPL.

[0079] Figure 9 Exemplary ILM-IPL frontal angiography blank image 51 and exemplary ILM-OPL frontal angiography blank image 53, defined by the segmentation algorithm output from an automatic segmentation tool, are shown. As shown, most of the frontal images 51 and 53 are similar, indicating no segmentation failure. However, a local region 55 in frontal image 51 is dissimilar to a corresponding local region 55 in frontal image 53, indicating IPL segmentation failure in that local region. The dissimilarity may be due to the blank defined by automatic IPL segmentation in image 51, and segmentation failure would cause the blank in image 51 to not conform to the correct shape of IPL (e.g., defining the bottom layer of the blank). The similarity between each row (e.g., corresponding to each B scan) of the ILM-IPL 51 and ILM-OPL 53 angiography blanks can be measured by normalized cross-correlation (NCC) between each row of the two blanks.

[0080] Figure 10 A graph showing the NCC for all rows (B-scans) is provided, numbered from 0 at the top to 1200 at the bottom. As can be seen, the similarity in the top portion (e.g., a continuous local region) of the angiography blank (e.g., from approximately B-scans 50 to 200) is lower than in other portions of the angiography blank. This is an indication of IPL segmentation failure in that top portion. If the NCC of the B-scan is less than a predetermined threshold (e.g., 0.5–0.7 or other suitable value), then IPL segmentation failure is determined to have occurred in that B-scan.

[0081] It can be based on a weighted average of ILM and OPL (e.g., 0.4). ILM+0.6 OPL (Optical Layer Proportion) can be replaced with an IPL segmentation approximation method in the failed region. For example, weighting of ILM (top layer) and OPL (bottom layer) can be based on their positions relative to the expected location of the target retinal layer. Other layers can be identified / approximated / examined based on the IPL approximation method. A weighted average of ILM and IPL approximations (e.g., 0.8) can be used. ILM+0.2 The RNFL segmentation approximation can be calculated using IPL_approx. It can also be based on a weighted average of the OPL and IPL approximations (e.g., 0.6). OPL+0.4 IPL_approx) is used to calculate the INL segmentation approximation method.

[0082] As an alternative, if the variance of the NCC function for all B scans is less than a threshold, the IPL segmentation approximation method can be used to replace the IPL segmentation of the volume data. Figure 11 The image shows B-scan 61 and ILM-IPL frontal blank 63, where IPL segmentation of the volume data is replaced by an IPL segmentation approximation method. In B-scan 61, solid lines and dashed lines represent the segmentation and approximation results for RNFL, IPL, and INL, respectively. Local segmentation failures can also be identified, as described above, by establishing similarity between ILM-IPL and ILM-OPL angiographic blanks within the windowed portion of the B-scan (or a portion of a set of B-scans), rather than across the entire B-scan.

[0083] In summary, in embodiments of the present invention, OCT angiography is used for segmentation failure identification. The method of the present invention can use two or more reference layers for segmentation and identify local segmentation failures.

[0084] In an exemplary embodiment, the present invention provides an automated approximation method for internal retinal layer segmentation in cases of advanced retinal disease during optical coherence tomography (OCTA).

[0085] Typically, automated multilayer segmentation (MLS) methods determine the desired inner layer boundaries. However, they tend to have layer misidentification biases, especially in cases of retinal disease and poor data quality. In these cases, the inner layer boundaries are often not manually identifiable. Example embodiments of the present invention provide an approximate method for automated inner reticular layer (IPL) outer boundary segmentation using OCTA bodies.

[0086] MLS uses internal limiting membrane (ILM)-IPL and ILM-outer reticular layer (OPL) angiographic blanks generated based on ILM segmentation and OPL outer boundary segmentation to detect IPL segmentation failure. This assumes that ILM and OPL segmentation are correct. It is expected that MLS IPL segmentation is not working correctly if these blanks are generated based on maximum extension, resulting in low local similarity measured by normalized cross-correlation (NCC) between ILM-IPL and ILM-OPL angiographic blanks. If the variance of NCC is less than a threshold, it is used as a weighted average of ILM and OPL segmentation. Approximation method Alternate from MLS IPL splitting; otherwise, MLS IPL splitting will be used.

[0087] The performance of the invention at the time of implementation was evaluated using 161 angiographic data points acquired on 3×3mm (76 scans), 6×6mm (67 scans), 8×8mm (2 scans), 12×12mm (7 scans), HD 6×6mm (6 scans), and HD 8×8mm (3 scans) plates acquired using a CIRRUS™ HD-OCT 6000 (ZEISS, Dublin, CA) equipped with AngioPlex® OCT angiography. The data included a mixture of retinal diseases. Clinical graders evaluated each surface retinal layer (SRL) blank generated using the new algorithm as either successful or unsuccessful.

[0088] Figure 12 Two examples of SRL blanks 71 and 72 before and after correction are shown (e.g., original blanks 71A / 72A and corrected blanks 71B / 72B using biased MLS segmentation). Both blanks 71 and 72 show MLS IPL segmentation 73a (e.g., cyan dashed lines in a color image or dashed lines in a monochrome image) and IPL approximation 73b (e.g., solid cyan lines in a color image or dashed lines in a monochrome image) superimposed on two B scans, where the MLS IPL segmentation is incorrect. As shown in the corrected blanks 71B and 72B, the IPL approximation yielded a more accurate representation of SRL for these cases. 34 (21%) MLS IPL segments in the scans were replaced by the IPL approximation. The success rates without using the IPL approximation and with the approximation were 79% and 96%, respectively. Therefore, exemplary embodiments of the present invention produce acceptable SRL blanks when MLS IPL segmentation performance suffers from severe retinal disease or poor image quality.

[0089] Currently, this paper provides alternative exemplary segmentation methods described above, which automatically or manually identify the starting position (e.g., extending the starting position and / or B-scan) and extend multi-layer segmentation information. For illustrative purposes, Figure 13The image shown is an example of automatic segmentation failure at the IS / OS junction (FOV=8mm), which also affected RPE segmentation. By manually correcting the IS / OS segmentation and then extending it, the RPE can be automatically and correctly re-segmented. Similarly, Figure 14 The image shown is an example of automatic segmentation failure in the fovea region (FOV=16mm), which affects the inner retinal layer as well as IS / OS and RPE segmentation. Additionally, manual correction and extension in the fovea region can correct segmentation in that area.

[0090] This invention provides an automated method for extending the B-scan, which has optimal contrast in the inner / outer retina, across the entire OCT volume based on multi-layer segmentation. This contrasts with previous extension methods, which were semi-automated and limited to extension along single-layer boundaries. This invention allows for the use of a semi-automated method similar to the automated method of this invention, except that the initial B-scan is selected and / or partially edited by a human expert. In the semi-automated method, subgroups of layer boundaries from the initial B-scan can be edited. Then, the automated portion of the algorithm of this invention can still segment the remaining unedited layer boundaries before extension.

[0091] The automated method of this invention is relatively fast, making it suitable for use in commercial applications. The automated method of this invention is based on the idea of ​​multi-layer segmentation extension (e.g., extension based on multiple layer boundaries simultaneously). This automated method naturally begins with a portion of a retinal scan that has a healthy structure as a healthy condition, or from enhanced OCT data (image) or other suitable high-quality portions of an image (OCT data). Segmentation begins with a healthy portion of the retina (B-scan), and this extension algorithm is relatively fast and robust due to the smooth transition to adjacent B-scans.

[0092] This document describes the general concept of automatic segmentation and extension. A preprocessing method suitable for use with this invention is described in U.S. Patent 10,169,864, assigned to the same assignee as this invention. However, the actual segmentation and extension workflows are different.

[0093] Figure 15 An example of an OCT B scan (FOV=12mm) with overlapping segmentation (from top to bottom) is shown, which reveals multiple retinal layer boundaries (similar to...). Figure 19 The ones shown include the inner limiting membrane (ILM), the outer boundary of the retinal nerve fiber layer (RNFL or NFL), the outer boundary of the inner retinal layer (IPL), the outer boundary of the inner nuclear layer (INL), the outer boundary of the outer retinal layer (OPL), the IS / OS junction, and the retinal pigment epithelial cells (RPE).

[0094] Layer boundaries with positive axis gradients (dark-to-light transitions) generally include the boundary between the vitreous body and the ILM. The upper boundary of the bright line is related to the IS / OS, and the lower boundary of the bright line is related to the INL. Layers with negative axis gradients (light-to-dark transitions) generally include the outer boundaries of the RNFL, IPL, and OPL (and BM).

[0095] For example, layer boundaries with positive axis gradients can be segmented simultaneously or sequentially in a B-scan. These segments can be used as baselines for segmenting adjacent B-scans. Additionally, these segments can define regions of interest in adjacent B-scans for segmenting layer boundaries with positive or negative axis gradients. Layer boundaries with negative axis gradients can be segmented simultaneously or sequentially in a B-scan.

[0096] Figure 16 The general process of an automated method for segmentation extension based on the present invention is shown. First, initial segmentation is performed in step 91. In this step, a subgroup of B-scans from the OCT volume having the highest contrast around the layer or group of layers of interest is automatically selected. As described in U.S. Patent 9,778,021 (as assigned to the same assignee as the present invention), a measure of separability can be used to determine the local contrast of the B-scans. The subgroups of B-scans can be segmented independently or dependently using a graph-based method or other fast methods known in the art (if they are spatially adjacent to each other). In step 93, a starting B-scan is selected, as well as a starting segmentation to be extended. A confidence metric for the segmentation can be used to determine which B-scan segmentation from the subgroup of B-scans has the highest confidence. Preferably, a confidence value is assigned to each segmentation point, which may be a value acquired from the total cost function (image) at the segmentation point. The total cost image can also be used for graph-based segmentation to search for layers of interest. In a third step 95, the segmentation is extended from the B-scan selected in the previous steps. The segmentation extends to the adjacent B-scans that started from a certain point. In this process, the dividing lines of the B-scan become the baselines for the same layer segmentation of adjacent B-scans. It is assumed that the segmentation variation between two adjacent B-scans is defined (e.g., restricted) within a (predetermined) range, which is used to extract the region of interest within the B-scan to be segmented (e.g., the segmentation is assumed to be limited to the region of interest). For example, the range can be approximately 1 to 5 times the interval between two adjacent B-scans. For example, for a 12-micron interval between two adjacent B-scans, the range can be 10 to 30 microns. Alternatively, since each dividing line is constrained by two adjacent dividing lines, the range (e.g., the region of interest) can be the region between two adjacent dividing lines. For example, a first B-scan_1 can be segmented, and then a second B-scan_2 can be segmented using the segmentation of B-scan_1 as a reference, thereby defining the OCT region of interest for layer segmentation. Once B-scan_2 is segmented, it becomes the reference for a third B-scan_3.

[0097] Figure 17 A semi-automated method based on segmentation and extension is shown. Semi-automated segmentation and extension for layer boundaries or groups of layer boundaries will be implemented. Figure 16 Steps 91 and 93 in the automated method shown are replaced with human expert input. In box 101, the segmentation line in the selected B-scan where segmentation failure is observed (e.g., determined by the automated method) is correct. In box 103, the segmentation line is in a manner similar to... Figure 16 The segmentation line extends in the manner of step 95 in the automatic method. As shown in box 105, the segmentation line extends throughout the entire OCT volume (if no segmentation determined by the automatic method is available), or stops at the layer boundary if the deviation between the automatic segmentation and the current segmentation is less than a threshold (e.g., a 5-micron threshold). Then, boxes 101 to 105 are repeated until all segmentation lines in the OCT volume are correct, as shown in box 107.

[0098] The following provides a description of various hardware and architectures suitable for this invention.

[0099] Optical coherence tomography imaging system

[0100] Generally, optical coherence tomography (OCT) uses low-coherence light to produce two-dimensional (2D) and three-dimensional (3D) internal views of biological tissues. OCT enables in vivo imaging of retinal structures. OCT angiography (OCTA) produces flow information, such as blood flow from blood vessels within the retina. Examples of OCT systems are provided in U.S. Patents 6,741,359 and 9,706,915, and examples of OCTA systems can be found in U.S. Patents 9,700,206 and 9,759,544, all of which are incorporated herein by reference in their entirety. Exemplary OCT / OCTA systems are provided herein.

[0101] Figure 18A generalized frequency-domain optical coherence tomography (FD-OCT) system suitable for collecting 3D image data of the eye is shown. The FD-OCT system OCT_1 includes a light source LtSrc1. Typical light sources include (but are not limited to) broadband light sources with short time coherence lengths or scanning laser sources. The beam from the light source LtSrc1 is typically guided by an optical fiber Fbr1 to illuminate a sample, for example, the eye E; a typical sample is tissue in the human eye. For example, the light source LrSrc1 can be a broadband light source with a short time coherence length in the case of spectral domain OCT (SD-OCT), or a wavelength-tunable laser source in the case of scanning source OCT (SS-OCT). Typically, the light is scanned by a scanner Scnr1 located between the output of the optical fiber Fbr1 and the sample E, thereby laterally scanning the beam (dash Bm) over the sample area to be imaged. The beam from the scanner Scnr1 can pass through a scanning lens SL and an ophthalmic lens OL and be focused onto the sample E to be imaged. The scanning lens SL can receive the beam from the scanner SnNR1 at multiple incident angles and produce substantially parallel light, which the ophthalmic lens OL can then focus onto the sample. Examples of the invention illustrate the need to scan a beam in two lateral directions (e.g., the x and y axes in the Cartesian plane) to scan the desired field of view (FOV). An example of this would be a point-field OCT, which uses a point-field beam to scan the entire sample. Thus, the scanner SnNR1 is exemplarily shown to include two sub-scanners: a first sub-scanner XSCN for scanning the point-field beam across the entire sample in a first direction (e.g., the horizontal x-direction); and a second sub-scanner YSCN for scanning the point-field beam across the sample in an intersecting second direction (e.g., the vertical y-direction). If the scanning beam is a line-field beam (e.g., a line-field OCT), which can sample an entire line portion of the sample each time, then only one scanner may be needed to scan the entire sample with the line-field beam to cover the desired FOV. If the scanning beam is a full-field beam (e.g., full-field OCT), then a scanner is not required, and the full-field beam can be applied to the entire desired FOV together.

[0102] Regardless of the type of beam used, light scattered from the sample (e.g., sample light) is collected. In an example of the invention, the scattered light returning from the sample is collected into the same optical fiber Fbr1 used to guide the light for illumination. A reference light originating from the same light source LtSrc1 propagates along a separate optical path, in this case including optical fiber Fbr2 and a retroreflector RR1 with adjustable optical delay. Those skilled in the art will recognize that a transmission reference optical path can also be used and that the adjustable delay can be placed in the reference arm of the sample or interferometer. For example, in an optical fiber coupler Cplr1, the collected sample light is combined with the reference light to form an optical interference in an OCT photodetector Dtctr1 (e.g., a photodetector matrix, digital camera, etc.). Although a single fiber port into detector Dtctr1 is shown, those skilled in the art will recognize that various interferometer designs can be used for balanced or unbalanced detection of interference signals. The output from detector Dtctr1 is provided to a processor (e.g., an internal or external computing device) Cmp1, which converts the observed interference into sample depth information. Depth information can be stored in memory associated with processor Cmp1 and displayed on a display (e.g., computer / electronic display / screen) Scn1. The processing and storage functions can be located within the OCT instrument, and the functions can be offloaded to (e.g., implemented thereon) an external processor (e.g., an external computing device) that can transmit the acquired data. Figure 22 An example of a computing device (or computer system) is shown. This unit may be dedicated to data processing or performing other very general tasks, rather than being dedicated to an OCT device. For example, the processor (computing device) Cmp1 may include a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a system-on-a-chip (SoC), a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), or a combination thereof, which may perform some or all of the processing steps in a serial and / or parallel manner together with one or more main processors and / or one or more external computing devices.

[0103] The sample and reference arms in the interferometer can be composed of bulk optics, fiber optics, or hybrid bulk optics systems, and can have different architectures, such as Michelson, Mach-Zehnder, or common-path-based designs known to those skilled in the art. The beam used herein should be interpreted as any carefully guided optical path. Instead of a mechanically scanning beam, the optical field can illuminate a one-dimensional or two-dimensional region of the retina to generate OCT data (e.g., see U.S. Patent 9,332,902; D. Hillmann et al., “Holoscopy – Holographic Optical Coherence Tomography,”). Optics Letters , 36(13): 2390 2011; Y. Nakamura et al., “High-Speed ​​Three DimensionalHuman Retinal Imaging by Line Field Spectral Domain Optical CoherenceTomography,” Optics Express , 15(12):7103 2007; Blazkiewicz et al., “Signal-To-Noise Ratio Study of Full-Field Fourier-Domain Optical Coherence Tomography,” Applied Optics , 44(36):7722 (2005)). In time-domain systems, the reference arm needs to have an adjustable optical delay to generate interference. Balanced detection systems are commonly used in TD-OCT and SS-OCT systems, while spectrometers are used for the detection port of SD-OCT systems. The invention described herein can be applied to any type of OCT system. Various aspects of the invention can be applied to any type of OCT system or other types of ophthalmic diagnostic systems and / or multiple ophthalmic diagnostic systems, including but not limited to fundus imaging systems, visual field testing devices, and scanning laser polarimeters.

[0104] In Fourier domain optical coherence tomography (FD-OCT), each measurement is a real-valued spectral interferogram (Sj(k)). The real-valued spectral data typically undergoes several post-processing steps, including background subtraction and dispersion correction. The Fourier transform of the processed interferogram results in a complex-valued OCT signal output Aj(z) = |Aj|ei. The absolute value |Aj| of this complex-valued OCT signal reveals the scattering intensity spectrum at different path lengths, thus scattering is a function of depth (z-direction) in the sample. Similarly, phase... j can also be extracted from complex-valued OCT signals. The scattering spectrum as a function of depth is called an axial scan (A-scan). A set of A-scans measured at adjacent locations in a sample produces a cross-sectional image (tomogram or B-scan) of the sample. A series of B-scans acquired at different tangential locations on the sample constitutes a data volume or cube. For a specific data volume, the term fast axis refers to the scanning direction along a single B-scan, while slow axis refers to the axis along which multiple B-scans are collected. The term "cluster scan" can refer to a single cell or data block generated by repeated acquisition at the same (or substantially the same) location (or region) for the purpose of analyzing motion contrast, which can be used to identify blood flow. A cluster scan can consist of multiple A-scans or B-scans acquired at approximately the same location on the sample at relatively short time intervals. Because the scans in a cluster scan belong to the same region, the static structure remains relatively unchanged between scans within the cluster scan, while the motion contrast between scans that meet predetermined criteria can be identified as blood flow.

[0105] Various methods for generating B-scans are known in the art, including but not limited to: along the horizontal or x-direction, along the vertical or y-direction, along the diagonal of x and y, or in a circular or spiral pattern. A B-scan can be in the xz dimension, but can be any cross-sectional image including the z-dimensional dimension. Figure 19 This image shows an example of a normal human retina via OCT B scan. An OCT B scan of the retina provides a view of the retinal tissue structure. For illustrative purposes, Figure 19 Several typical retinal layers and their boundaries were identified. The identified retinal boundary layers include (from top to bottom): Inner Limiting Membrane (ILM) Layer 1, Retinal Nerve Fiber Layer (RNFL or NFL) Layer 2, Ganglion Cell Layer (GCL) Layer 3, Inner Reticulum Layer (IPL) Layer 4, Inner Nuclear Layer (INL) Layer 5, Outer Reticulum Layer (OPL) Layer 6, Outer Nuclear Layer (ONL) Layer 7, Junction between the Outer Segment (OS) and Inner Segment (IS) of the photoreceptor (represented by reference symbol Layer 8), Outer or Outer Limiting Membrane (ELM or OLM) Layer 9, Retinal Pigment Epithelial Cells (RPE) Layer 10, and Bruch's Membrane (BM) Layer 11.

[0106] In OCT angiography or functional OCT, analytical algorithms can be applied to OCT data collected at the same or substantially the same sampling locations of the sample at different times (e.g., cluster scans) to analyze motion or flow (e.g., see U.S. Patent Publications Nos. 2005 / 0171438, 2012 / 0307014, 2010 / 0027857, 2012 / 0277579 and 6,549,801, the entire contents of which are incorporated herein by reference). OCT systems can use any of a variety of OCT angiography processing algorithms (e.g., motion contrast algorithms) to identify blood flow. For example, motion contrast algorithms can be applied to intensity information derived from image data (intensity-based algorithms), phase information derived from image data (phase-based algorithms), or complex image data (complexity-based algorithms). A frontal image is a 2D projection of 3D OCT data (e.g., by averaging the intensity of each individual A-scan such that each A-scan defines pixels in the 2D projection). Similarly, a frontal vascular system image is an image displaying motion-contrast signals, where the data dimension corresponding to depth (e.g., along the z-direction of the A-scan) is displayed as a single representative value (e.g., a pixel in a 2D projected image), typically displayed by summing or integrating all or individual portions of the data (e.g., see U.S. Patent No. 7,301,644, the entire contents of which are incorporated herein by reference). An OCT system providing angiographic imaging capabilities may be referred to as an OCT angiography (OCTA) system.

[0107] Figure 20 An example of a frontal vascular system image is shown. After processing the data using any motion contrast technique known in the art to enhance motion contrast, a range of pixels corresponding to a given tissue depth from the surface of the internal limiting membrane (ILM) in the retina can be summed to produce a frontal (e.g., orthographic) image of the vascular system. Figure 21 An exemplary B-scan image of vascular structures (OCT) is shown. As illustrated, structural information may not be explicitly defined because blood flow can cross multiple retinal layers, making them less clearly defined than in a structural OCT B-scan. Figure 19As shown. Nevertheless, OCTA provides a non-invasive technique for imaging the microvascular system of the retina and choroid, which can be important for the diagnosis and / or monitoring of a variety of pathologies. For example, OCTA can be used to identify diabetic retinopathy by recognizing microaneurysms, neovascular complexes, and quantifying avascular and non-perfused areas of the fovea. Furthermore, OCTA has been shown to be in good agreement with fluorescein angiography (FA), a more routine but more invasive technique that requires the injection of dye to observe vascular flow in the retina. Additionally, in dry age-related macular degeneration, OCTA has been used to monitor a general reduction in choroidal capillary flow. Similarly, in wet age-related macular degeneration, OCTA can provide qualitative and quantitative analysis of the choroidal neovascular membrane. OCTA has also been used to investigate vascular occlusion, such as non-perfused areas, and to evaluate the integrity of surface and deep plexuses.

[0108] Computing device / system

[0109] Figure 22 An example computer system (or computing device or computer apparatus) is shown. In some embodiments, one or more computer systems may provide the functionality described or shown herein and / or perform one or more steps of one or more methods described or shown herein. The computer system may take any suitable physical form. For example, the computer system may be an embedded computer system, a system-on-a-chip (SOC), a single-board computer system (SBC) (e.g., a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, a grid of computer systems, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, the computer system may reside in a cloud, which may include one or more cloud components in one or more networks.

[0110] In some embodiments, the computer system may include a processor Cpnt1, memory Cpnt2, storage Cpnt3, input / output (I / O) interface Cpnt4, communication interface Cpnt5, and bus Cpnt6. The computer system may optionally also include a display Cpnt7, such as a computer monitor or screen.

[0111] Processor Cpnt1 includes hardware for executing instructions, such as those that make up a computer program. For example, processor Cpnt1 may be a central processing unit (CPU) or a general-purpose graphics processor (GPGPU). Processor Cpnt1 may retrieve (or fetch) instructions from internal registers, internal caches, memory Cpnt2, or memory Cpnt3, decode and execute the instructions, and write one or more results to internal registers, internal caches, memory Cpnt2, or memory Cpnt3. In a particular implementation, processor Cpnt1 may include one or more internal caches for data, instructions, or addresses. Processor Cpnt1 may include one or more instruction caches and one or more data caches to maintain a data table. Instructions in the instruction cache may be copies of instructions in memory Cpnt2 or memory Cpnt3, and the instruction cache may accelerate the retrieval of those instructions by processor Cpnt1. Processor Cpnt1 may include any suitable number of internal registers and may include one or more arithmetic logic units (ALUs). Processor Cpnt1 may be a multi-core processor; or may include one or more processors Cpnt1. Although this invention discloses and shows a particular processor, this invention discloses considerations for any suitable processor.

[0112] Memory Cpnt2 may include main memory for storing instructions that enable processor Cpnt1 to execute or hold temporary data during processing. For example, a computer system may load instructions or data (e.g., data tables) from memory Cpnt3 or from another source (such as another computer system) into memory Cpnt2. Processor Cpnt1 may load instructions and data from memory Cpnt2 into one or more internal registers or internal caches. To execute instructions, processor Cpnt1 may fetch instructions from internal registers or internal caches and decode them. During or after instruction execution, processor Cpnt1 may write one or more results (which may be intermediate or final results) to internal registers, internal caches, memory Cpnt2, or memory Cpnt3. Bus Cpnt6 may include one or more memory buses (each of which may include an address bus and a data bus) and may couple processor Cpnt1 to memory Cpnt2 and / or memory Cpnt3. Optionally, one or more memory management units (MMUs) facilitate data transfer between processor Cpnt1 and memory Cpnt2. Memory Cpnt2 (which may be fast volatile memory) may include random access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM). Memory Cpnt3 may include long-term or high-capacity memory for data or instructions. Memory Cpnt3 may be internal or external to the computer system and includes one or more of the following: disk drives (e.g., hard disk drives, HDDs, or solid-state drives SSDs), flash memory, ROM, EPROM, optical disks, magneto-optical disks, magnetic tape, Universal Serial Bus (USB) accessible drives, or other types of non-volatile memory.

[0113] The I / O interface Cpnt4 can be software, hardware, or a combination of both, and includes one or more interfaces (e.g., serial or parallel communication ports) for communicating with I / O devices, enabling communication with a person (e.g., a user). For example, I / O devices may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, camera, stylus, tablet, touchscreen, trackball, camcorder, another suitable I / O device, or a combination of two or more of these.

[0114] The communication interface Cpnt5 provides a network interface for communicating with other systems or networks. The communication interface Cpnt5 may include a Bluetooth interface or other types of packet-based communication. For example, the communication interface Cpnt5 may include a network interface controller (NIC) and / or a wireless NIC or wireless adapter for communicating with a wireless network. The communication interface Cpnt5 can provide communication with Wi-Fi networks, ad hoc networks, personal area networks (PANs), wireless PANs (e.g., Bluetooth WPANs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), cellular telephone networks (e.g., Global System for Mobile Communications (GSM) networks), the Internet, or a combination of two or more of these.

[0115] The Cpnt6 bus can provide communication links between the aforementioned components of a computing system. For example, the Cpnt6 bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand bus, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCIe bus, a Serial Advanced Technology Accessory (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these.

[0116] Although the present invention discloses and illustrates a particular computer system having a particular number of particular components in a particular arrangement, the present invention discloses contemplation of any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0117] In this document, where appropriate, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical disc drives, floppy disks, floppy disk drives (FDDs), magnetic tape, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, a computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0118] Although the invention has been described in conjunction with several specific embodiments, it will be apparent to those skilled in the art that many other alternatives, modifications, and alterations will be apparent from the above description. Therefore, the invention described herein is intended to cover all such alternatives, modifications, applications, and alterations that may fall within the spirit and scope of the appended claims.

Claims

1. A method for segmenting the retinal layer in an optical coherence tomography (OCT) image, the method comprising: OCT data of the eye are collected using an OCT system; The OCT data is segmented into individual retinal layers; For the target retinal layer: A first frontal image is generated based on a first slab, wherein the target retinal layer is positioned facing the bottom of the first slab; A second frontal image is generated based on a second slab, wherein the target retinal layer is located between the top and bottom layers of the second slab. Based on the similarity metric between the first frontal image and the second frontal image, the segmentation of the target retinal layer is designated as either a failure or a success.

2. The method according to claim 1, wherein the first frontal image and the second frontal image belong to a slab from the OCT data or from OCT angiography data generated using the OCT data.

3. The method according to claim 1 or 2, wherein the similarity measure is based on normalized cross-correlation (NCC) between frontal images.

4. The method according to any one of claims 1 to 3, wherein for a segmentation response designated as a failure, at least a portion of the segmentation is replaced with an approximation based on the top and bottom layers of the second slab.

5. The method according to claim 4, wherein determining the approximation method comprises: Weights are applied to the top and bottom layers of the second slab based on their positions relative to the expected location of the target retinal layer.

6. The method of claim 4, wherein the similarity measure includes a local similarity measure that identifies local segmentation failures, and the local segmentation failures are replaced by an approximation method.

7. The method of claim 6, wherein a local similarity metric is determined on a B-scan via a B-scan basis.

8. The method according to any one of claims 1 to 7, wherein the first slab and the second slab have the same top layer.

9. The method of claim 8, wherein the top and bottom layers of the second slab are selected based on the degree of abrupt change from light to dark or from dark to light.

10. The method of claim 8, wherein the target retinal layer is the bottom layer of the first slab, and the bottom layer of the second slab is below the target retinal layer.

11. The method of claim 10, wherein the top layer is an inner limiting membrane (ILM), the target retinal layer is an inner retinal layer (IPL), and the bottom layer of the second slab is an outer retinal layer (OPL).

12. The method of claim 10, wherein the top layer is an inner limiting membrane (ILM), the target retinal layer is an outer retinal layer (OPL), and the bottom layer of the second blank is a junction between an outer segment (OS) and an inner segment (IS).

Images