Techniques for localizing features in retinal fundus images and / or measurements

By generating graphs with nodes and edges from retinal fundus images and applying cost functions, the method improves the efficiency and accuracy of feature localization in retinal fundus imaging, facilitating precise health assessments.

JP2024542719AInactive Publication Date: 2024-11-15TESSERACT HEALTH INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024532675
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-01
Filing Date
2022-11-30
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Conventional imaging and measurement systems for retinal fundus images are inefficient and inaccurate in locating features such as retinal layers, leading to potential errors in health assessments.

Method used

A method involving generating graphs from retinal fundus images using nodes and edges, including auxiliary nodes and edges, to determine feature locations efficiently by applying cost functions and weighting values, allowing for accurate and computationally efficient boundary detection.

Benefits of technology

Enhances the computational efficiency and accuracy of locating retinal fundus features, enabling precise determination of retinal layer boundaries and improving health assessments based on retinal fundus images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024542719000001_ABST
    Figure 2024542719000001_ABST
Patent Text Reader

Abstract

Described herein are techniques for imaging and / or measuring a subject's eye, including the subject's retinal fundus. In some embodiments, one or more processors may be used to generate graphs from images and / or measurements (e.g., optical coherence tomography images and / or measurements) that may be useful for locating features within the images and / or measurements. For example, the images and / or measurements may include the subject's retinal fundus. The features may include one or more layers and / or boundaries between layers of the subject's retinal fundus in the images and / or measurements.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This disclosure relates to techniques for imaging and / or measuring a subject's eye, including the subject's retina fundus. This application claims the benefit of priority to U.S. Provisional Application No. 63 / 284,791, entitled "FEATURE LOCATION TECHNIQUES FOR RETINA FUNDUS IMAGES AND / OR MEASUREMENTS," filed December 1, 2021, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] There is room for improvement in techniques for imaging and / or measuring a subject's eye. Summary of the Invention

[0003] Some aspects of the present disclosure relate to methods comprising generating a graph from images and / or measurements of a retina of a subject, where generating the graph includes at least one processor generating a plurality of nodes corresponding to a plurality of pixels of the images and / or measurements and a plurality of edges connecting the plurality of nodes, at least one auxiliary node, and an auxiliary edge connecting the auxiliary node to a first node of the plurality of nodes.

[0004] In some embodiments, the auxiliary edge is a first auxiliary edge, and generating the graph further includes the at least one processor generating a second auxiliary edge connecting the at least one auxiliary node to a second node of the plurality of nodes, the first and second nodes corresponding to first and second pixels, respectively, in a first column of the image and / or measurement results.

[0005] In some embodiments, the at least one auxiliary node includes a first auxiliary node that is a start node of the graph, and a second auxiliary node that is an end node of the graph.

[0006] In some embodiments, the auxiliary edge is a first auxiliary edge, and the first node corresponds to a first pixel of a plurality of pixels in a first column of the images and / or measurements, and generating the graph further includes the at least one processor generating a second auxiliary edge connecting the first auxiliary node to a second node of the plurality of nodes corresponding to a second pixel of the plurality of pixels in the first column, a third auxiliary edge connecting the second auxiliary node to a third node of the plurality of nodes corresponding to a third pixel of the plurality of pixels in a second column of the images and / or measurements, and a fourth auxiliary edge connecting the second auxiliary node to a fourth node of the plurality of nodes corresponding to a fourth pixel of the plurality of pixels in the second column.

[0007] In some embodiments, the method may further include the at least one processor using the graph to identify a location of a boundary between a first layer and a second layer of the subject's retinal fundus in the images and / or measurements.

[0008] In some embodiments, the at least one auxiliary node comprises a start node and / or an end node of the graph, and locating the boundary comprises determining a plurality of paths from the start node to the at least one auxiliary node and / or from the at least one auxiliary node to the end node and selecting a path from among the plurality of paths.

[0009] In some embodiments, generating the graph further comprises assigning weighting values ​​to at least some of the plurality of nodes and / or edges, generating auxiliary edges comprises assigning pre-set weighting values ​​to the auxiliary nodes and / or edges, and selecting the path from among the plurality of paths comprises performing a cost function using the weighting values ​​and the pre-set weighting values, and determining that the path has and / or shares the lowest cost among the plurality of paths.

[0010] In some embodiments, the weighting values ​​are assigned to the nodes based on pixel intensities of pixels corresponding to the nodes, and / or to the edges based on pixel intensities of pixels corresponding to pairs of the nodes connected by the edges.

[0011] In some embodiments, the weighting values ​​are assigned to the nodes based on the frequency and / or phase of pixels corresponding to the nodes, and / or the weighting values ​​are assigned to the edges based on the frequency and / or phase of pixels corresponding to pairs of the nodes connected by the edges.

[0012] In some embodiments, performing the cost function includes determining a cost for each of the plurality of paths, the cost of each path being based at least in part on the weighting values ​​and / or pre-defined weighting values ​​assigned to nodes and / or edges in each path.

[0013] In some embodiments, the pre-set weighting value has a minimum cost. Some aspects of the present disclosure relate to methods comprising generating a graph from images and / or measurements of a retina of a subject, where generating the graph includes at least one processor generating a plurality of nodes corresponding to a plurality of pixels of the images and / or measurements and a plurality of edges connecting the plurality of nodes, selecting a start node and / or an end node of the graph from the plurality of nodes, and generating at least one auxiliary edge connecting the start node and / or the end node to a first node of the plurality of nodes.

[0014] In some embodiments, the method may further comprise the at least one processor assigning weighting values ​​to at least some of the plurality of nodes and / or edges, and assigning a preset weighting value to the at least one auxiliary edge and / or a start node and / or an end node.

[0015] In some embodiments, the weighting values ​​are assigned to the nodes based on pixel intensities of pixels corresponding to the nodes, and / or to the edges based on pixel intensities of pixels corresponding to pairs of the nodes connected by the edges.

[0016] In some embodiments, the weighting values ​​are assigned to the nodes based on derivatives corresponding to the nodes, and / or assigned to the edges based on derivatives of pixels corresponding to pairs of the nodes connected by the edges.

[0017] In some embodiments, the start node is selected to correspond to a first corner pixel of the image and / or measurement and the end node is selected to correspond to a second corner pixel of the image and / or measurement, the first and second corner pixels being located in different columns of the image and / or measurement.

[0018] In some embodiments, generating the at least one auxiliary edge includes generating a first plurality of auxiliary edges connecting the start node to each of a first plurality of outer perimeter nodes of the plurality of nodes corresponding to a plurality of pixels in a first column of the image and / or measurement results, and generating a second plurality of auxiliary edges connecting the end node to each of a second plurality of outer perimeter nodes of the plurality of nodes corresponding to a plurality of pixels in a second column of the image and / or measurement results.

[0019] In some embodiments, the method may further include the at least one processor using the graph to identify a location of a boundary between a first layer and a second layer of the subject's retinal fundus in the images and / or measurements.

[0020] In some embodiments, locating the boundary includes determining a plurality of paths from the start node to the end node via the auxiliary edges and selecting a path from among the plurality of paths.

[0021] In some embodiments, selecting the path includes performing a cost function based on the weighting values ​​and the pre-set weighting values, and determining that the path has and / or shares a minimum cost among the plurality of paths.

[0022] In some embodiments, the pre-set weighting value has a minimum cost. In some embodiments, executing the cost function includes minimizing the cost function.

[0023] The above summary is not intended to be limiting. Aspects of the present disclosure can be implemented alone or in combination with other aspects according to various embodiments. The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component illustrated in various figures is represented by a like numeral. For purposes of clarity, not every component is labeled in every drawing. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a block diagram of a cloud-connected system for processing images in accordance with some embodiments of the techniques described herein. [Diagram 2] FIG. 2 is an example image including multiple pixels, according to some embodiments. [Diagram 3] FIG. 3 is an exemplary graph including nodes corresponding to pixels of the image of FIG. 2 and edges connecting the nodes, according to some embodiments. [Figure 4A] FIG. 4A is an exemplary graph including nodes corresponding to pixels of the image of FIG. 2, pairs of auxiliary nodes, and edges connecting the nodes of the graph, according to some embodiments. [Figure 4B] FIG. 4B is the graph of FIG. 4A with paths traversing the graph shown, according to some embodiments. [Figure 5A] FIG. 5A is an alternative exemplary graph including nodes corresponding to pixels of the image of FIG. 2 and edges connecting the nodes, according to some embodiments. [Figure 5B] FIG. 5B is the graph of FIG. 5A with a path shown traversing a portion of the graph, according to some embodiments. [Figure 6A] FIG. 6A is the example image of FIG. 2 with a path traversing a portion of the image, according to some embodiments. [Figure 6B]FIG. 6B is the example image of FIG. 2 with first and second subsets of pixels shown in the image, according to some embodiments. [Figure 6C] FIG. 6C is the example image of FIG. 2 with a second path that further traverses a portion of the image, according to some embodiments. [Figure 7] FIG. 7 is an exemplary image of a subject's retinal fundus, according to some embodiments. [Figure 8] FIG. 8 is an example differential image that may be generated using the image of FIG. 7 according to some embodiments. [Figure 9] FIG. 9 is another example image of a subject's retinal fundus, according to some embodiments. [Figure 10] FIG. 10 is an example image that may be produced by shifting pixels in columns of the image of FIG. 9 according to some embodiments. [Figure 11] FIG. 11 is yet another exemplary image of a subject's retinal fundus, according to some embodiments. [Figure 12] FIG. 12 is an example positive derivative image of the image of FIG. 11 according to some embodiments. [Figure 13] FIG. 13 is the image of FIG. 11 with paths shown traversing the internal limiting membrane (ILM) boundary and the inner segment-outer segment (IS-OS) boundary, respectively, of a subject's retinal fundus, in accordance with some embodiments. [Figure 14] FIG. 14 is an example negative derivative image of the image of FIG. 11 according to some embodiments. [Figure 15] FIG. 15 is the image of FIG. 11 showing paths traveled through the ILM boundary, the IS-OS boundary, and the Bruch's membrane (BM) boundary of a subject's retinal fundus, respectively, in accordance with some embodiments. [Figure 16] FIG. 16 is the image of FIG. 11 showing a subset of pixels above a threshold pixel intensity level according to some embodiments. [Figure 17] FIG. 17 is the image of FIG. 11 showing one of the subsets of pixels shown in FIG. 16, according to some embodiments. [Figure 18] FIG. 18 is the image of FIG. 11 showing a subset of pixels corresponding to the retinal nerve fiber layer-ganglion cell layer (RNFL-GCL) boundary of a subject's retinal fundus, in accordance with some embodiments. [Figure 19] FIG. 19 is the image of FIG. 11 showing a subset of pixels below a threshold pixel intensity level according to some embodiments. [Figure 20] FIG. 20 is an example positive derivative image of the image of FIG. 11 within the subset of pixels shown in FIG. 19, according to some embodiments. [Figure 21] FIG. 21 is the image of FIG. 11 showing a path traveled across the inner nuclear layer-outer plexiform layer (INL-OPL) boundary of a subject's retina fundus, in accordance with some embodiments. [Figure 22] FIG. 22 is an example negative derivative image of the image of FIG. 11 within the subset of pixels shown in FIG. 19, according to some embodiments. [Figure 23] FIG. 23 is an image of FIG. 11 showing paths traveled across the inner plexiform layer-inner nuclear layer (IPL-INL) boundary and the outer plexiform layer-outer nuclear layer (OPL-ONL) boundary of a subject's retinal fundus, respectively, in accordance with some embodiments. [Figure 24] FIG. 24 is a flowchart of an exemplary method for generating a graph from images and / or measurements, according to some embodiments. [Diagram 25] FIG. 25 is a flowchart of an alternative exemplary method for generating a graph from images and / or measurements, according to some embodiments. [Figure 26] FIG. 26 is a flowchart of an exemplary method for locating one or more features of a subject's retinal fundus in images and / or measurements of the subject's retinal fundus, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] [I. Introduction] The present inventors have recognized and understood that the eye of a subject (e.g., a person) provides a window into the body that can be used to determine whether the subject has an eye disease, as well as to determine the subject's overall health. In particular, the fundus retina can provide useful information through imaging for use in various health assessments. However, conventional systems for imaging, measuring, and / or processing images and / or measurements of the fundus are limited in many ways.

[0026] The inventors have recognized that conventional imaging and / or measurement systems do not accurately locate certain features of a subject's retinal fundus in images and / or measurements. For example, imaging and / or measurement systems cannot accurately locate boundaries between retinal layers, much less in a computationally efficient manner. In a clinical setting, once images and / or measurements are obtained, a clinician may need to inspect each image and / or measurement to locate features such as boundaries between retinal layers by segmentation in each image. In addition to being time consuming, this process may be imperfect due to practical limitations of human vision, resulting in inaccurate measurements. These measurements may then be used to determine the subject's health, but may be inaccurate and / or erroneous. Similarly, existing systems for locating features of a subject's retinal fundus in images and / or measurements are not accurate or computationally efficient in doing so.

[0027] To solve the above problems, the present inventors have developed improved techniques and methods for generating graphs from images and / or measurements (e.g., optical coherence tomography images and / or measurements) by one or more processors, which may be useful for locating features in the images and / or measurements. For example, the images and / or measurements may include a subject's retinal fundus. The features may include one or more layers and / or boundaries between layers of the subject's retinal fundus in the images and / or measurements.

[0028] In some embodiments, generating the graph may include generating nodes corresponding to pixels of the image and / or measurements and edges connecting the nodes. For example, nodes may be generated for some or all pixels of the image and / or measurements. In some embodiments, generating the graph may further include generating at least one auxiliary node. For example, the auxiliary nodes may be generated in addition to nodes of the graph corresponding to pixels of the image and / or measurements. The auxiliary nodes may be start and / or end nodes of the graph. In some embodiments, generating the graph may further include generating auxiliary edges connecting a first auxiliary node to a first node of the graph. For example, the auxiliary edges may be generated in addition to any edges generated connecting nodes of the graph corresponding to pixels.

[0029] The inventors have recognized that the generation of auxiliary node(s) and / or auxiliary edge(s) can increase the computational efficiency of locating features using a graph. For example, the feature location techniques described herein can include determining one or more paths that traverse the nodes and edges of a graph. The use of the auxiliary nodes and / or edges can allow for computationally efficient selection of an appropriate path for locating features (e.g., using a cost function). In this example, the use of the auxiliary nodes and / or edges can more efficiently determine which node corresponding to one or more pixels of an image and / or measurement should be the second node and / or the penultimate node in the selected path.

[0030] In some embodiments, a first auxiliary node may be generated as a start node and a second auxiliary node may be generated as an end node. The inventors further recognized that path determination may be more computationally efficient if the auxiliary node is a start node or an end node. In some embodiments, auxiliary edges may be generated that connect the auxiliary node(s) to some or all nodes that correspond to pixels in the same column of the graph, such as a perimeter column. For example, one of the nodes that correspond to pixels in the perimeter column may be the second or next to last node in a path that starts or ends on the perimeter column side of the image and / or measurement.

[0031] In some embodiments, weighting values ​​may be assigned to the nodes and / or edges of the graph, and preset weighting values, such as a minimum value, may be assigned to the auxiliary nodes and / or edges. For example, using the graph to locate the retinal fundus feature may include performing a cost function based on the weighting values ​​and / or the preset weighting values. The inventors have recognized that using preset weighting values, such as a minimum value (e.g., a local minimum and / or a global minimum), may make the selection of a path that indicates the location of the feature more computationally efficient.

[0032] In some examples, executing the cost function may include minimizing the cost function and selecting the path may include selecting a path of connected nodes and / or edges having a minimum cost. In some examples, executing the cost function (e.g., when an inverted or negated cost function is used) may include maximizing the cost function such that searching for a path may include searching for a path of connected nodes and edges having a maximum cost.

[0033] The inventors further recognized that generating one or more auxiliary edges when the start and / or end nodes of the graph correspond to pixels in the image and / or measurement may make feature localization using the generated graph more computationally efficient. In accordance with other techniques described herein, in some embodiments, generating the graph may include generating a plurality of nodes corresponding to a plurality of pixels of the image and / or measurement and a plurality of edges connecting the plurality of nodes, selecting a start and / or end node of the graph from among the plurality of nodes, and generating at least one auxiliary edge connecting the start and / or end node to another node of the graph. For example, the start and / or end node may be selected as a node corresponding to a corner pixel of the image and / or measurement. In this example, the auxiliary edge(s) may connect the corner pixel(s) to nodes corresponding to other pixels in the same column of the image and / or measurement, such as a perimeter column that includes the corner pixel. In some embodiments, the start and end nodes may correspond to opposite corner pixels of the image and / or measurement.

[0034] The techniques described herein for locating retinal fundus features within images and / or measurements are more computationally efficient than previous techniques, for example, the techniques described herein require fewer edges when generating a graph of the images and / or measurements, which can increase the efficiency of determining and / or selecting paths through the graph that correspond to the features.

[0035] The inventors have further developed other techniques, further described herein, that can be used alone or in combination with the techniques described above to more accurately and computationally efficiently locate one or more features of the subject's retinal fundus in an image and / or measurement. Such techniques may include, for example, first locating a first feature (e.g., a first retinal layer boundary) of the subject's retinal fundus in an image and / or measurement, and then using the location of the first feature to locate a second feature of the subject's retinal fundus in the same image and / or measurement in a derivative of the image and / or measurement and / or in a subset of pixels of the image and / or measurement. The inventors have recognized that in some cases, the first and second features may have a known juxtaposition (e.g., one is expected to be above the other, or vice versa) and / or may have relative pixel intensity levels in the image and / or measurement that can advantageously locate the second feature more efficiently and / or more accurately after locating the first feature.

[0036] In some embodiments, the techniques described herein may be implemented using a system comprising at least one processor and memory configured to receive images and / or measurements via a communications network. Alternatively or additionally, in some embodiments, the techniques described herein may be implemented on-board and / or on images and / or measurements acquired by an imaging device and / or a measuring device. In some embodiments, the imaging device and / or measuring device described herein may be suitable for use by a subject with or without the assistance of a provider, clinician, or technician. In some embodiments, the imaging device and / or measuring device described herein and / or associated systems may be configured to determine a health status of a subject based on acquired images and / or measurements.

[0037] The techniques described herein may be implemented alone or in combination with any other techniques described herein, and although at times the specification may refer only to images, the embodiments described herein are not so limited, and aspects described herein for images may also apply to measurements.

[0038] II. An Exemplary System for Generating Graphs from Retinal Images As noted above, the inventors have developed techniques for generating graphs from images of the retina. In some embodiments, such techniques may be implemented using the exemplary systems described herein. Although reference is made below to images, the embodiments described herein are not so limited, and aspects described herein for images also apply to measurements.

[0039] FIGURE 1 is a block diagram of an exemplary system 100 for generating a graph from images and / or measurements of a subject's retina, according to some embodiments. FIGURE 1 illustrates system 100 including an imaging device 130 and a computer 140, which may be coupled to one another via a communications network 160. In some embodiments, imaging device 130 may be configured to capture images of the subject's retina and provide the images to computer 140 via communications network 160. In some embodiments, computer 140 may be configured to receive the images and generate a graph from the images. In some embodiments, imaging device 130 may alternatively or additionally be configured to generate a graph from the images.

[0040] In some embodiments, the imaging device 130 may be configured to capture an image of the subject's retina and provide the image to the computer 140 via the communication network 160. As shown in FIG. 1, the imaging device 130 may include an imaging element 132, a processor 134, and a memory 136. In some embodiments, the imaging element 132 may be configured to capture an image of the subject's eye, such as the subject's retinal fundus. For example, in some embodiments, the imaging element 132 may include an illumination source component configured to illuminate the subject's eye, a sample component configured to focus and / or relay the illumination light to the subject's eye, and a detection component configured to capture light reflected and / or emitted from the subject's eye in response to the illumination. In some embodiments, the imaging element 132 may include a fixation component configured to display a fixation target on the subject's eye to guide the subject's eye to a desired position and / or orientation. According to various embodiments, the imaging element 132 may be an optical coherence tomography (OCT) device, a white light device, a fluorescent device, or an infrared (IR) device. In some embodiments, imaging device 130 may include multiple imaging elements 132, such as any or each of the imaging elements described above, although the embodiments described herein are not so limited.

[0041] In some embodiments, the processor 134 may alternatively or additionally be configured to transmit captured images to the computer 140 via the communication network 160. In some embodiments, the imaging device 130 may include a stand-alone network controller configured to communicate over the communication network 160. Alternatively, the network controller may be integrated with the processor 134. In some embodiments, the imaging device 130 may include one or more displays for providing information to a user of the imaging device 130 via a user interface displayed on the display. In some embodiments, the imaging device 130 may be portable. For example, the imaging device 130 may be configured to perform imaging of the eye using power stored in a rechargeable battery.

[0042] In some embodiments, computer 140 may be configured to obtain images and / or measurements of the subject's retinal fundus from imaging device 130 and generate a graph from the images and / or measurements. For example, computer 150 may be configured to use the graph to identify a location of one or more features of the subject's retinal fundus, such as a boundary between a first layer and a second layer of the subject's retinal fundus. As shown in FIG. 1, computer 140 may include a storage medium 142 and a processor 144.

[0043] In some embodiments, the processor 144 may be configured to generate a graph from the images and / or measurements of the subject's retina. For example, the processor 144 may be configured to generate a plurality of nodes corresponding to a plurality of pixels of the images and / or measurements. In this example, the processor 144 may be configured to generate a node for each pixel of the images and / or measurements, or for only a subset of the images and / or measurements. In some embodiments, the processor 144 may be configured to generate a plurality of edges connecting the plurality of nodes. For example, when connected by an edge, the processor 144 may be configured to traverse the nodes of the graph along the edges. In this example, the processor 144 may be configured to generate edges connecting each node or only a subset of the generated nodes.

[0044] In some embodiments, the processor 144 may be configured to generate an auxiliary node as a start node and / or an end node of the graph and generate a first edge from the auxiliary node to a second node of the graph. For example, the second node may be included among a plurality of generated nodes corresponding to a plurality of pixels of the image and / or the measurement. The processor 144 may also be configured to generate the auxiliary node in addition to the plurality of generated nodes corresponding to a plurality of pixels of the image and / or the measurement. In some embodiments, the processor 144 may further be configured to generate a second edge from the start node and / or the end node to a third node of the graph. For example, the second and third nodes of the graph may be perimeter nodes corresponding to pixels along a perimeter of the image and / or the measurement, such as within the same column of the image and / or the measurement. Alternatively or additionally, in some embodiments, the processor 144 may be configured to generate a graph from the image by selecting a start and / or end node from the nodes corresponding to pixels of the image and / or the measurement, with or without generation of the auxiliary node. For example, the processor 144 may be configured to generate auxiliary edges connecting the start node and / or the end node to another node that corresponds to a pixel of the image and / or the measurement.

[0045] In some embodiments, the processor 144 may be configured to use the graph to locate at least one feature of the subject's retinal fundus in the image and / or measurement. For example, the processor 144 may be configured to locate a boundary between a first layer and a second layer of the subject's retinal fundus. In some embodiments, the processor 144 may be configured to determine multiple paths from a start node to an end node of the graph. For example, the processor 144 may be configured to traverse the graph from the start node to the end node via different paths that include one or more other nodes of the graph. In some embodiments, the processor 144 may be configured to assign a cost to each path based on a cost function. For example, the processor 144 may be configured to assign a cost based on derivatives of the nodes included in the path (e.g., based on differences in derivatives of the nodes). Alternatively or additionally, the processor 144 may be configured to assign a higher cost to longer paths (e.g., paths that traverse more nodes than other paths). In some embodiments, the processor 144 may be configured to select one path from among the multiple paths. For example, the processor 144 may be configured to select, from among a plurality of paths, a path that has a lowest cost and / or shares.

[0046] In some embodiments, computer 140 may be further configured to precondition the images and / or measurements to identify the location of the subject's retinal fundus features. For example, in some embodiments, processor 144 may be configured to generate a derivative of the images and / or measurements and generate a graph using the derivative of the images and / or measurements. For example, processor 144 of computer 140 may be configured to apply a filter to the images and / or measurements to generate the derivative before generating the graph. Alternatively or additionally, in some embodiments, processor 144 may be configured to shift pixels in columns of the images and / or measurements before generating the graph. For example, processor 144 may be configured to shift pixels such that one or more pixels corresponding to features of the images and / or measurements are aligned within at least one row of pixels (e.g., with features included in only one or two rows of pixels). Further, alternatively or additionally, processor 144 may be configured to select a subset of pixels of the images and / or measurements that identify the location of the subject's retinal fundus feature(s). For example, the processor 144 may be configured to apply a pixel property threshold, such as a pixel intensity threshold, to the image and / or measurements to identify features in only a subset of pixels above (or below) the threshold. Alternatively or additionally, the processor 144 may be configured to select a subset of pixels that identifies the location of a feature(s) based on a previously determined location of one or more features within the image and / or measurements.

[0047] According to various embodiments, communication network 160 may be a local area network (LAN), a cellular network, a Bluetooth network, the Internet, or any other network. For example, computer 140 may be located remotely relative to imaging device 130, such as in a different room than imaging device 130, and communication network 160 may be a LAN. In some embodiments, computer 140 may be located in a different geographic area than imaging device 130 and communicate via the Internet.

[0048] According to various embodiments, multiple devices may be included in place of or in addition to imaging device 130. For example, an intermediate device may be included in system 100 to communicate between imaging device 130 and computer 140. Alternatively or additionally, multiple computers may be included in place of or in addition to computer 140 to perform the various tasks attributed to computer 140 described herein.

[0049] It should be noted that in some embodiments, the systems described herein may not include imaging and / or measurement devices, as at least some of the techniques described herein may be performed using images and / or measurements obtained from other systems.

[0050] III. Exemplary Techniques for Locating Retinal Fundus Features in an Image As described herein, the inventors have developed techniques for generating a graph from images and / or measurements of a subject's retinal fundus and using the generated graph to identify the location of one or more features of the subject's retinal fundus. In some embodiments, the techniques described herein may be implemented using the system of FIG. 1, such as using an imaging device and / or one or more processors of a computer.

[0051] 2 is an exemplary image 200 including a plurality of pixels, of which pixels 201 and 202 are labeled, according to some embodiments. In some embodiments, one or more processors of the system 100, such as the processor 134 of the imager 130 and / or the processor 144 of the computer 140, may be configured to generate a graph using the image 200. In some embodiments, the image 200 may be captured using an imager, such as the imager 132 of the imager 130. For example, the image 200 may be an OCT image, a white light image, a fluorescent image, or an IR image. In some embodiments, the image 200 may include a retinal fundus of the subject. For example, one or more processors described herein may be configured to locate one or more features of the retinal fundus of the subject in the image 200.

[0052] In some embodiments, pixels of image 200 may have pixel intensity values ​​(e.g., ranging from 0 to 255) that can control the brightness of the pixel. For example, in FIG. 2, pixel 202 may have a lower pixel intensity value than pixel 201 when pixel 202 is shown to have a higher brightness than pixel 201. In some embodiments, the pixel intensity values ​​of the pixels of image 200 may indicate the presence of one or more retinal fundus features shown in image 200. For example, the pixel intensity values ​​of the retinal fundus image may correspond to the intensity of backscattered light received at an imaging device that captured image 200, and the intensity of the backscattered light may vary depending on the feature being imaged.

[0053] FIG. 3 is an example graph 300 including multiple nodes corresponding to multiple pixels of image 200 and multiple edges connecting the multiple nodes, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate graph 300 using image 200, such as by generating nodes corresponding to some or all of the pixels of image 200. For example, in FIG. 3, node 301 of graph 300 may correspond to pixel 201 of image 200, and node 302 of graph 300 may correspond to pixel 202 of image 200. In some embodiments, the processor(s) may further be configured to generate edges connecting some or all of the nodes of graph 300. For example, FIG. 3 shows edge 311 connecting nodes 301 and 302. In some embodiments, the processor(s) may be configured to store graph 300 including the nodes and edges in a storage medium, such as storage medium 142 of computer 140.

[0054] In some embodiments, the processor(s) may be configured to store (e.g., in a storage medium) values ​​associated with some or all of the nodes of the graph 300 based, for example, on pixel intensity values ​​of pixels to which the nodes correspond. For example, the processor(s) may be configured to store, in association with the node 301, a pixel intensity value of the pixel 201. Alternatively or additionally, the processor may be configured to store, in association with the node 301, a derivative of the pixel intensity of the image 200 at the pixel 201. In either example, the processor(s) may be configured to use the stored values ​​associated with each node to calculate a cost associated with traversing one or more paths through the graph 300. Alternatively or additionally, in some embodiments, the processor(s) may be configured to store values ​​associated with some or all of the edges of the graph 300 based, for example, on pixel intensity values ​​of pixels corresponding to the nodes connected by the respective edges. For example, the processor(s) may be configured to store, in association with the edge 311, a value based on a derivative of the pixel intensity of the image 200 at each pixel 201, 202. In this example, the processor(s) may be configured to calculate the cost associated with traversing one or more paths through the graph 300 using the stored values ​​associated with each edge.

[0055] In some embodiments, the stored values ​​associated with each node and / or the edges connecting pairs of nodes may be weighted. In some examples, the stored value associated with each edge may be a calculation of a cost function based on the values ​​of the nodes that form the edge. For example, the cost function may be 2-(g a +g b )+W min The processor can then compute the cost function 2-(g a +g b )+W min A weighting value W equal to the value of ab may be configured to store in association with the edge 311. Here, g a,g b is the derivative of the pixel intensity at pixels a, b corresponding to the nodes connected by the edge, and W min may be a weight of a preset value. For example, the preset value may be predetermined and / or calculated based on pixel intensity values ​​of image 200, rather than based on particular pixel intensity values ​​of pixels corresponding to two nodes connected by an edge. In this example, the preset value may be the minimum value of all other edges, or may be less than it.

[0056] 4A is an example graph 400 including a plurality of nodes corresponding to a plurality of pixels of image 200, a pair of auxiliary nodes 401, 402, and auxiliary edges connecting the auxiliary nodes to nodes corresponding to image 200, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate graph 400 using graph 300 by generating auxiliary nodes 401, 402 and a plurality of edges connecting auxiliary nodes 401, 402 to at least some nodes on the periphery of the graph. For example, as shown in FIG. 4A, auxiliary node 401 is connected to nodes 303, 304 in periphery column 403 of the graph via auxiliary edges 405, 406, respectively, and auxiliary node 402 is connected to nodes 301, 302 in periphery column 404 of the graph via auxiliary edges 407, 408, respectively.

[0057] In some embodiments, auxiliary node 401 and / or auxiliary node 402 may be a starting node and / or an ending node of graph 400. For example, processor(s) may be configured to determine one or more paths that traverse the nodes and edges of graph 400 that begin and / or end at auxiliary node 401 and / or auxiliary node 402. In some embodiments, processor(s) may be configured to calculate a cost of each path based on, for example, a cost associated with each edge and / or node traversed by the path. For example, the cost associated with each edge and / or node may be based on pixel intensities and / or derivatives of pixel intensities at the respective edges and / or nodes, such that a minimum and / or maximum cost path may indicate the location of one or more retinal fundus features in image 200. In some embodiments, auxiliary edges connecting auxiliary node 401 and / or auxiliary node 402 to other nodes of graph 400 may be weighted with the same pre-set value, such as a minimum value. For example, the minimum value may provide a minimum cost for traversing each auxiliary edge. According to various embodiments, the minimum cost may be a global minimum and / or a local minimum for a local node (eg, corresponding to a particular subset of pixels).

[0058] 4B is a graph 400 having a display path 450 that traverses the graph 400, according to some embodiments. As shown in FIG. 4B, the display path 450 may traverse nodes 303, 305, 306, 307, 308, and 309 of the graph 400. In some embodiments, one or more processors described herein may be configured to determine the path 450 by starting at supplemental node 401 and / or supplemental node 402 and traversing the nodes of the graph 400 until the other of supplemental nodes 401, 402 is reached.

[0059] In some embodiments, the processor(s) may be configured to determine multiple paths traversing the graph 400 and select path 450 from among the multiple paths. For example, the processor(s) may be configured to calculate a cost for each path based on which nodes and / or edges are traversed by the respective paths and determine that path 450 has and / or shares the smallest cost. In the example of FIG. 4B , the processor(s) may be configured to determine path 450 by starting at auxiliary node 401 and continuing to node 303 via auxiliary edge 405 (e.g., at the smallest cost). The processor(s) may be configured to determine that continuing from node 303 to node 305 has the lowest cost of all nodes connected to node 303 by a single edge, such as based on node 305 having the lowest pixel intensity value of all nodes connected to node 303 by a single edge. Similarly, the processor(s) may be configured to determine that continuing from node 305 to node 306 has the lowest cost of all nodes connected to node 305 by a single edge (e.g., excluding node 303). When the processor(s) reaches node 309, the processor may be configured to determine that continuing to auxiliary node 402 via auxiliary edge 408 has the lowest cost of all nodes connected to node 309 by a single edge (e.g., because the auxiliary edges connected to auxiliary nodes 401, 402 may have the lowest cost). In some embodiments, the processor may use a Dijkstra algorithm, a Bellman-Ford algorithm, a Warshall-Floyd algorithm, an A-class algorithm, or a sparse-space algorithm. * The route 450 may be configured to be selected using an algorithm such as the Johnson method and / or the algorithm.

[0060] 5A is an alternative exemplary graph 500 including multiple nodes corresponding to multiple pixels of image 200 and multiple edges connecting the multiple nodes, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate graph 500 using graph 300, such as by selecting corner nodes 303, 309 as the start and / or end nodes of graph 500, and generating multiple auxiliary edges connecting node 303 to each node in perimeter column 403 and multiple auxiliary edges connecting node 309 to each node in perimeter column 404. For example, the processor(s) may be configured to select corner node 303 and / or corner node 309 as the start and / or end nodes for determining multiple paths traversing graph 500, as described herein for graph 400. In FIG. 5A, auxiliary edge 505 is shown connecting node 309 to node 301, and auxiliary edge 506 is shown connecting node 309 to node 302. In some embodiments, the processor(s) may be configured to assign a pre-set weighting value (e.g., a minimum value) to the auxiliary edges 505, 506 as described herein for the auxiliary edges of the graph 400.

[0061] According to various embodiments, any corner node of graph 300 may be selected as a start node and / or an end node for determining multiple paths. In some embodiments, the processor(s) may be configured to determine one or more paths traversing graph 500 between node 303 and node 309 in the manner described herein for graph 400.

[0062] FIG. 5B is a graph 500 with a displayed path 450 that traverses a portion of the graph 500, according to some embodiments. As shown in FIG. 5B, the processor may be configured to determine and / or select the same path 450 using nodes 303, 309 as the start and end nodes, as in the example of FIG. 4A and FIG. 4B with auxiliary start and end nodes. In the illustrated example, the processor may be configured to select the path 450 based on determining that traveling other nodes in the column 403 (FIG. 5A) connected to node 303 via auxiliary edges having a preset value exceeds the cost of traveling the path 450. For other exemplary images, the processor may be configured to select a path that traverses one or more auxiliary edges from node 303 to a node in the column 403.

[0063] FIG. 6A is an image 200 having a path 601 traversing a portion of the image 200, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate a version of the image 200 showing the path 601 in response to determining and / or selecting the path 601 using a graph (e.g., graph 300, graph 400, or graph 500) from the image 200. In some embodiments, the path 601 may indicate a location of one or more retinal fundus features in the image 200, such as a boundary between a first layer of the subject's retinal fundus and a second layer of the subject's retinal fundus. In some embodiments, the processor(s) may be configured to determine and / or select multiple paths to generate a version of the image 200 showing each path. For example, various paths may correspond to features of the subject's retinal fundus shown in the image 200.

[0064] FIG. 6B is an image 200 having first and second subsets 600a, 600b of pixels shown in the image 200, according to some embodiments. In some embodiments, one or more processors described herein may be configured to divide the image 200 into multiple subsets of pixels, such as the subsets 600a, 600b shown in FIG. 6B. For example, one or each of the subsets 600A, 600B of pixels may include pixels corresponding to one or more retinal fundus features. In FIG. 6B, a first subset 600a is shown with a path 601 traversing the pixels of the subset 600a, which may correspond to a first feature of the human fundus shown in the image 200. In some embodiments, at least some pixels of the second subset 600B may correspond to a second feature of the human retinal fundus. The inventors have recognized that dividing the pixels of the image into subsets before locating at least some features in the image may facilitate locating features in different areas of the image.

[0065] In some embodiments, the processor(s) may be configured to divide pixels of the image 200 into subsets 600a, 600b after locating features of the image 200 indicated by the path 601. For example, the processor(s) may be configured to sort pixels traversed by the path 601 into subset 600a along with pixels adjacent to the traversed pixels on one side of the path 601. In this example, the processor(s) may be configured to sort pixels on the other side of the path 601 into subset 600b. In this example, since the path 601 has been obtained by processing the entire image 200 to locate a first feature in subset 600a, dividing the image between subsets 600a and 600b may focus further processing of the image 200 in subset 600b to locate additional features.

[0066] Alternatively or additionally, in some embodiments, the processor may be configured to divide the pixels of the image 200 into subsets 600a, 600b based on pixel characteristics such as pixel intensity, frequency, and / or phase. For example, the processor(s) may be configured to sort into each subset adjacent pixels above a threshold pixel intensity level and / or adjacent pixels within a threshold pixel intensity level of each other. In this example, dividing the image into subsets of pixels based on pixel characteristics may facilitate determining the location of features at expected locations relative to each other (e.g., determining the location of adjacent retinal layer boundaries in a retinal fundus image) and / or distinguishing features located in different subsets based on the relative characteristics of the subsets (e.g., relative pixel intensity). For example, the processor may be configured to obtain multiple clusters by applying one or more vector quantization techniques (e.g., K-Means clustering) and select clusters having higher (or lower) cluster means (e.g., corresponding to pixel intensity values). The processor may then be configured to identify the location of one or more features (e.g., the retinal pigment epithelium and / or the retinal nerve fiber layer) within the selected cluster by applying a polynomial fit.

[0067] 6C is an example image 200 with a second display path 602 that further traverses a portion of the image 200, according to some embodiments. For example, the second path 602 may correspond to a second feature located using the graph generation and route determination methods described herein for the path 601. In this example, the processor(s) may be configured to generate a graph using pixels of the subset 600b to determine the second path 602. FIG. 7 is an example image 700 of a subject's retinal fundus, according to some embodiments. In some embodiments, the image 700 may be captured using an imaging element (e.g., imaging element 132) described herein.

[0068] In some embodiments, image 700 may show one or more features of the subject's retinal fundus. For example, in FIG. 7, image 700 shows layers 701-714 of the subject's retinal fundus, as well as a region of vitreous humor 715 adjacent layer 701 and the subject's sclera 716 adjacent layer 714. In some embodiments, layer 701 may be the subject's internal limiting membrane (ILM) layer, layer 702 may be the subject's retinal nerve fiber layer (RNFL), layer 703 may be the subject's ganglion cell layer (GCL), layer 704 may be the subject's inner plexiform layer (IPL), layer 705 may be the subject's inner nuclear layer (INL), layer 706 may be the subject's outer plexiform layer (OPL), layer 707 may be the subject's outer nuclear layer (ONL), and layer 708 may be the subject's sclera (ONL). 708 may be the subject's outer limiting membrane (ELM) layer, layer 709 may be the subject's outer segment (OS) of the photoreceptor (PR) layer, layer 710 may be the subject's inner segment (IS) of the PR layer, layer 711 may be the subject's retinal pigment epithelium (RPE) layer, layer 712 may be the subject's Bruch's membrane (BM) layer, layer 713 may be the subject's choriocapillaris (CC) layer, and / or layer 714 may be the subject's choroidal stroma (CS) layer. According to various embodiments, image 700 may show any or each layer of the subject's fundus retina.

[0069] In some embodiments, one or more processors described herein may be configured to identify the location of one or more features of the subject's retinal fundus shown in image 700. For example, the processor(s) may be configured to generate a graph from image 700 as described herein for graphs 300, 400, and / or 500, and determine one or more paths (e.g., path 450) traversing the graph. In this example, the processor(s) may be configured to select one or more paths and generate a version of image 700 showing the path(s) traversing image 700, which may indicate the location(s) of the feature(s). In the example of FIG. 7, the processor(s) may be configured to identify the location of features, such as any or each of layers 701-714, and / or the location of features, such as the boundaries between any or each of layers 701-716.

[0070] In some embodiments, one or more processors described herein may be configured to generate a derivative of the image 700 and generate a graph using the derivative of the image 700.

[0071] FIG. 8 is an exemplary derivative image 800 that may be generated using image 700, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate the derivative image 800 using image 700. For example, the processor may be configured to generate the derivative image 800 by applying a filter to image 700. For example, the filter may be configured to output, for some or each pixel of image 700, a derivative of the pixel intensity of image 700 at the respective pixel. In the example of FIG. 8, the derivative image 800 is a positive derivative image because the pixel intensity of the pixel of image 800 indicates an increase in the pixel intensity of the corresponding pixel of image 700 in direction 803. In some embodiments, the processor(s) may be configured to generate the derivative image 800 using a convolution filter, such as using a Sobel, Laplacian, Prewitt, and Roberts operator. In some embodiments, the processor(s) may be configured to generate a graph from the differential image 800.

[0072] The inventors have recognized that differentiation of an image of the subject's retinal fundus may highlight the location of certain features of the subject's retinal fundus in the image. For example, in differential image 800, portions 801, 802 of differential image 800 that may correspond to layers 701, 708 shown in image 700 have higher pixel intensity values ​​than those in image 700. In some embodiments, the processor(s) may be configured to generate a graph from a positive differential image, such as differential image 800, and determine one or more paths across the graph to identify the location of a boundary between the subject's ILM and a region of vitreous humor adjacent to the ILM, and / or an IS-OS boundary, in image 700. For example, a portion of the retinal fundus image between the ILM layer and the vitreous humor region and / or an IS-OS boundary may be more prominent in the differential image than in the retinal fundus image. In some embodiments, the processor(s) may alternatively or additionally be configured to generate a negative derivative image of image 700, generate a graph from the negative derivative image, and determine one or more paths traversing the graph, such as to identify the location of the BM layer within image 700. For example, a negative derivative image of a retinal fundus image may make the BM layer more prominent.

[0073] FIG. 9 is another exemplary image 900 of a subject's retinal fundus, according to some embodiments. In some embodiments, the image 900 (e.g., an OCT image) may show one or more features of the subject's retinal fundus. In some embodiments, one or more processors described herein may be configured to identify the location of one or more retinal fundus features within the image 900, such as using the methods described herein in connection with FIGS. 2-8. In FIG. 9, a curve 901 shows the location of the subject's fundus retinal features. For example, the curve 901 may show the subject's RPE layer.

[0074] In some embodiments, one or more processors described herein may be configured to shift pixels of the image 900 within columns of the image 900 after locating at least one feature within the image 900. For example, the processor(s) may be configured to locate a feature represented by curve 901 and shift pixels within columns of the image 900 until the curve 901 forms a line across one or more rows of pixels of the image 900. The inventors have recognized that shifting the pixels of the image after locating a retinal fundus feature (e.g., the RPE layer) may better position the pixels of the image for feature location.

[0075] 10 is an example image 1000 that may be generated by shifting pixels in columns of image 900, according to some embodiments. As shown in FIG. 10, the pixels in the columns of image 1000 are shifted relative to image 900 such that pixels exhibiting a curve 901 form one or more rows exhibiting a substantially flat line 902. In some embodiments, the processor(s) may be configured to locate one or more features of image 1000 using the methods described herein.

[0076] The inventors have also recognized that the methods described above may be advantageously combined to locate retinal fundus features in an image. One exemplary process incorporating the methods described above is described herein with reference to Figures 11-19.

[0077] 11 is yet another exemplary image 1100 of a subject's retinal fundus, according to some embodiments. As shown in FIG. 11, image 1100 (e.g., an OCT image) may show features 1101-1109, 1111-1112 of the subject's retinal fundus. For example, feature 1101 may be an area of ​​vitreous humor, feature 1102 may be the subject's ILM, feature 1103 may be the subject's RNFL, feature 1104 may be the subject's GCL, feature 1105 may be the subject's IPL, feature 1106 may be the subject's INL, feature 1107 may be the subject's OPL, feature 1108 may be the subject's ONL, layer 1109 may be the subject's OS photoreceptor layer, feature 1110 may be the subject's IS photoreceptor layer, feature 1111 may be the subject's RPE, and feature 1112 may be the subject's BM. In some embodiments, pixels of an image 1100 as shown in FIG. 11 may be shifted within columns of the image 1100 as described herein in connection with FIGS.

[0078] 12 is a positive derivative image 1200 that may be generated from image 1100, according to some embodiments. In some embodiments, one or more processors described herein may be configured to generate the derivative image 1200 to increase pixel intensity values ​​of pixels corresponding to a boundary 1201 (e.g., the ILM-vitreous boundary) between features 1101 and 1102 and / or a boundary 1202 (e.g., the IS-OS boundary) between features 1109 and 1110. In some embodiments, the processor(s) may be configured to generate one or more graphs (e.g., including generating one or more auxiliary nodes) from the positive derivative image 1200 and determine one or more paths traversing the graph to identify the location of boundary 1201 and / or boundary 1202, as described herein in conjunction with FIGS. 7 and 8.

[0079] 13 is an image 1100 with indicated paths 1121, 1122 that travel a boundary 1201 between features 1101 and 1102 and a boundary 1202 between features 1109 and 1110, respectively, of a subject's retinal fundus, according to some embodiments. In some embodiments, one or more processors described herein may be configured to determine the paths 1121 and / or 1122 from the differential image 1200 using methods described herein. For example, the processor may be configured to locate one feature (e.g., boundary 1201), divide the pixels of image 1100 and / or differential image 1200 into subsets of pixels, and locate another feature (e.g., boundary 1202) in a subset different from the subset containing the originally located feature, as described herein in connection with FIGS. 6A-6C.

[0080] In some embodiments, the processor(s) described herein may alternatively or additionally be configured to locate retinal fundus features in image 1100 by generating a negative derivative of image 1100. For example, the processor(s) may be configured to generate a negative derivative image after locating feature 1122 in image 1100. Feature 1122 may be used to segment the negative derivative image to facilitate locating additional features in image 1100.

[0081] FIG. 14 is a negative derivative image 1400 that may be generated from image 1100. As shown in FIG. 14, boundary 1412 may have higher (or lower) pixel intensity values ​​than image 1100. In some embodiments, boundary 1412 may correspond to a boundary between feature 1111 and feature 1112 (e.g., the RPE-BM boundary). In FIG. 14, a path 1122 from FIG. 13 showing boundary 1202 is superimposed on negative derivative image 1400. For example, processor(s) may be configured to divide pixels of negative derivative image 1400 into a subset of pixels on either side of path 1122. In some embodiments, processor(s) may be configured to select a subset of pixels on a side of path 1122 that includes boundary 1412, generate a graph from negative derivative image 1400, and determine one or more paths that traverse the graph to identify the location of boundary 1412.

[0082] FIG. 15 is an image 1100 with a display path 1123 that further moves the boundary 1412 of the subject's retinal fundus, according to some embodiments. FIG. 16 is an image 1100 further illustrating subsets 1603, 1610, 1611 of pixels above a threshold pixel intensity level, according to some embodiments. As shown in FIG. 16, the pixels of subset 1603 may correspond to feature 1103, and the pixels of subset 1610 may correspond to feature 1110, and the pixels of subset 1611 may correspond to feature 1111. In some embodiments, one or more processors described herein may be configured to identify contiguous subsets of pixels 1603, 1610, and / or 1611 as being above a threshold pixel intensity level. In the example of FIG. 16, subsets of pixels other than subsets 1603, 1610, 1611 may include pixels having pixel intensity levels below the threshold. According to various embodiments, pixels having a threshold pixel intensity level may be grouped with pixels above the threshold pixel intensity level and / or pixels below the threshold pixel intensity level, although the embodiments described herein are not so limited.

[0083] 16, a path 1122 indicating the boundary 1202 is superimposed on the image 1100. For example, the processor(s) may be configured to further divide the subsets 1603, 1610, 1611 on either side of the boundary 1202.

[0084] 17 is an image 1100 showing one of the subsets 1603 of pixels above a threshold pixel intensity level of FIG. 16, according to some embodiments. In some embodiments, the processor(s) may be configured to select one or more of the subsets 1603, 1610, 1611 of neighboring pixels from the image 1100 to identify a location of a feature of the subject's retinal fundus. For example, the processor(s) may be configured to select the subset 1603 based on the subset 1603 being above the boundary 1202 (e.g., outside the subject's retinal fundus in the depth direction). In some embodiments, the processor(s) may be configured to identify a location of a feature at a boundary between the features 1103 and 1104 (e.g., the RNFL-GCL boundary) within the selected pixel subset 1603.

[0085] 18 is an image 1100 having a displayed path 1124 moving through a boundary between features 1103 and 1104 of a subject's retinal fundus, according to some embodiments. In some embodiments, one or more processors described herein may be configured to determine the path 1124 by generating a graph using pixels of the subset 1603. Alternatively or additionally, in some embodiments, the processor(s) may be configured to determine the path 1124 as the bottom (e.g., innermost of the retinal fundus in the depth direction at which the image 1100 was captured) boundary of the subset 1603.

[0086] 19 is an image 1100 showing subsets 1905, 1906, 1907, 1908 of pixels below a threshold pixel intensity level, according to some embodiments. In some embodiments, subsets 1905, 1906, 1907, 1908 may correspond to features 1105, 1106, 1107, 1108 of FIG. 11. In some embodiments, the processor(s) may be configured to apply the same pixel intensity thresholds to obtain subsets 1905, 1906, 1907, 1908 as to obtain subsets 1603, 1610, 1611 in FIG. Alternatively or additionally, the processor(s) may be configured to apply different pixel intensity thresholds to obtain subsets 1603, 1610, 1611.

[0087] FIG. 20 is an exemplary positive derivative image 2000 that may be generated using subsets 1905, 1906, 1907, 1908 of pixels in image 1100 shown in FIG. 19, according to some embodiments. The inventors have recognized that by generating a derivative image using only one or more selected subsets of the image's pixels, portions of the image within the selected subsets may be further enhanced. For example, in FIG. 20, regions 2001 corresponding to features 1105, 1107 may be further enhanced than those in positive derivative image 1200. Also in FIG. 20, a path 1124 is shown traversing the derivative image 2000. In some embodiments, the processor(s) may be configured to select region 2001 in derivative image 2000 to identify the location of the boundary between features 1106 and 1107 based on path 1124. For example, path 1124 may indicate the location of feature 1105 in derivative image 2000. The processor(s) may be configured to select the region 2001 based on the location of the feature 1105.

[0088] According to some embodiments, the subset may be a subset of contiguous pixels having above a threshold pixel intensity level. The one or more processors may be configured to determine whether the set of contiguous pixels includes a pixel having a pixel intensity level higher than the threshold pixel intensity level.

[0089] FIG. 21 is an image 1100 with a display path 1125 that moves across the boundary between features 1106 and 1107, according to some embodiments. FIG. 22 is an exemplary negative derivative image 2200 that may be generated using subsets 1905, 1906, 1907, 1908 of pixels in image 1100 shown in FIG. 19, according to some embodiments. In FIG. 22, boundary 2201 (e.g., IPL-INL boundary) between features 1105 and 1106, and boundary 2202 (e.g., OPL-ONL boundary) between features 1107 and 1108 may be enhanced more than those in negative derivative image 1400. Also in FIG. 22, path 1125 is shown traversing derivative image 2200. In some embodiments, processor(s) may be configured to select subsets of pixels of derivative image 2200 on either side of path 1125 to identify the locations of boundaries 2201, 2202 within the respective selected subsets.

[0090] FIG. 23 is an image 1100 with display paths 1126, 1127 that travel across a boundary 2201 between features 1105 and 1106, and a boundary 2202 between features 1107 and 1108, in accordance with some embodiments.

[0091] 24 is a flowchart of an example method 2400 of generating a graph from images and / or measurements, according to some embodiments. In some embodiments, method 2400 may be performed using one or more processors of a system described herein (e.g., system 100) and / or a non-transitory storage medium may store instructions that, when executed by one or more processors, cause the processors to perform method 2400.

[0092] 24, the illustrated method 2400 includes generating, in step 2401, a number of nodes and / or a number of edges of a graph corresponding to a number of pixels of the image and / or measurements, generating auxiliary nodes of the graph in step 2402, and generating auxiliary edges connecting the auxiliary nodes to one or more nodes of the graph in step 2403. In some embodiments, the images and / or measurements may be images and / or measurements of a fundus of a subject, as described herein in connection with FIGS. 7, 9, and 11.

[0093] In some embodiments, generating the plurality of nodes and / or the plurality of edges of the graph in step 2401 may include a processor(s) generating nodes for some or all pixels of the image and / or measurements and a plurality of edges connecting the nodes to each other, as described herein in connection with FIG. 3. For example, the nodes and edges generated in step 2401 may include at least one column of nodes connected to each other by edges. In some embodiments, method 2400 may further include assigning weighting values ​​to some or all of the plurality of edges generated in step 2401, as described herein in connection with FIG. 3.

[0094] In some embodiments, generating auxiliary nodes in step 2402 may include the processor(s) adding auxiliary nodes to nodes that correspond to pixels of the image and / or measurements, as described herein in connection with FIG. 4A. For example, the auxiliary nodes may not correspond to pixels of any of the images and / or measurements and may be start or end nodes of the graph. In some embodiments, step 2402 may include the processor(s) generating multiple auxiliary nodes, such as start and end nodes of the graph.

[0095] In some embodiments, generating auxiliary edges in step 2403 may include the processor(s) connecting an auxiliary node to at least one node of the graph generated in step 2401, as described herein in connection with FIG. 4A. For example, the processor(s) may connect the auxiliary node to one or more adjacent nodes of a graph that may include multiple nodes corresponding to pixels in a column of the image and / or measurements.

[0096] In some embodiments, the method 2400 may further include locating a boundary between a first layer and a second layer of the subject's retina fundus using the graph, as described herein in connection with Figures 11-23. For example, the method 2400 may include determining a plurality of paths traversing the graph (e.g., from a start node, an auxiliary node, to an end node, an auxiliary node) and selecting a path from among the plurality of paths. In this example, the selected path may correspond to a boundary between a first layer and a second layer of the subject's retina fundus in the images and / or measurements.

[0097] In some embodiments, method 2400 may further include shifting pixels within columns of the image and / or measurements prior to generating the graph in step 2401. Alternatively or additionally, pixels of the image and / or measurements used to generate the graph may be pre-shifted within columns of pixels prior to performing method 2400, although embodiments described herein are not so limited.

[0098] In some embodiments, method 2400 may further include generating derivatives (e.g., positive and / or negative derivatives) of the images and / or measurements and generating a graph using the derivatives, as described herein in connection with Figures 9 and 10. Alternatively or additionally, the images and / or measurements used to generate the graph may be the derivatives of another image and / or measurement generated prior to performing method 2400, although embodiments described herein are not so limited.

[0099] In some embodiments, method 2400 may further include dividing pixels of the image and / or measurements into subsets and selecting the subset of pixels for generating a graph in step 2401 and / or for identifying the location of features of the subject's retinal fundus (e.g., boundaries between layers) within the graph as described herein, including in connection with Figures 16-23.

[0100] 25 is a flowchart of an alternative exemplary method 2500 of generating a graph from images and / or measurements, according to some embodiments. In some embodiments, the method 2500 may be performed in a manner described herein with respect to the method 2400. For example, the method 2500 may be performed using one or more processors described herein, and / or a non-transitory storage medium may store instructions that, when executed by one or more processors, cause the processor to perform the method 2500. Alternatively or additionally, the images and / or measurements may be images and / or measurements of a fundus of a subject.

[0101] In FIG. 25 , the illustrated method 2500 includes, in step 2501, generating a number of nodes and a number of edges of a graph from images and / or measurements, in step 2502, selecting a start node and / or an end node from the number of nodes of the graph, and in step 2503, generating an auxiliary edge connecting the start node or the end node to another node of the graph.

[0102] In some embodiments, generating a graph from the images and / or measurements in step 2501 may be performed in the manner described herein with respect to step 2401 of method 2400.

[0103] In some embodiments, selecting a start node and / or an end node from the plurality of nodes of the graph in step 2402 may include the processor(s) selecting a corner node corresponding to a corner pixel of the image and / or measurement as the start node and / or the end node. In some embodiments, the processor(s) may select a first node corresponding to a first corner pixel in a first column of the image and / or measurement as the start node and a second node corresponding to a second corner pixel in a second column of the image and / or measurement as the end node.

[0104] In some embodiments, generating an auxiliary edge connecting the start node or the end node to another node of the graph in step 2503 may include the processor(s) generating an auxiliary edge connecting the start node to another node corresponding to a pixel in the same column as the pixel corresponding to the start node, as described herein in connection with FIG. 5A. Alternatively or additionally, the processor(s) may generate an auxiliary edge connecting the end node to another node corresponding to a pixel in the same column as the pixel corresponding to the end node, as described herein in connection with FIG. 5A. For example, the processor(s) may generate one or more auxiliary edges connecting the start node to some or all nodes corresponding to pixels in the same column as the pixel corresponding to the start node, and / or generate one or more auxiliary edges connecting the end node to some or all nodes corresponding to pixels in the same column as the pixel corresponding to the end node.

[0105] In some embodiments, method 2500 may further include assigning weight values ​​to some or all of the edges generated in step 2501 and / or assigning pre-set weight values ​​to the auxiliary edges generated in step 2503, as described herein in connection with Figures 5A and 5B. In some embodiments, method 2500 may further include locating features of the subject's retina fundus within the images and / or measurements, such as by determining and / or selecting one or more paths that traverse the nodes of the graph, as described herein in connection with Figure 5B.

[0106] In some embodiments, the method 2500 may further include other steps of the method 2400 described herein in connection with FIG. 26 is an example method 2600 for locating one or more features of a subject's retinal fundus in images and / or measurements of the subject's retinal fundus, according to some embodiments. In some embodiments, the method 2600 may be performed using one or more processors described herein and / or a non-transitory computer-readable medium may store encoded instructions that, when executed by one or more processors, cause the processor to perform the method 2600.

[0107] As shown in FIG. 26 , method 2600 may include, in step 2601, shifting pixels in one or more columns of images and / or measurements; in step 2602, generating a first differential image and / or measurement from the images and / or measurements to identify the locations of one or more first features; in step 2603, generating a second differential image and / or measurement from the images and / or measurements to identify the locations of one or more second features; and in step 2604, selecting a subset of the images and / or measurements to identify the locations of one or more third features.

[0108] In some embodiments, shifting pixels within one or more columns of the image and / or measurements in step 2601 may include the processor(s) shifting the pixels until pixels of the image and / or measurements corresponding to at least one feature of the subject's retinal fundus (e.g., RPE) form a line along one or more rows of the image and / or measurements, as described herein in connection with Figures 9 and 10.

[0109] In some embodiments, generating a first derivative image and / or measurement from the image and / or measurement to locate the first feature in step 2602 may include the processor(s) generating positive and / or negative derivative images as described herein in connection with Figures 7 and 8. For example, the processor(s) may generate a positive derivative image that further highlights the first feature (e.g., the ILM-vitreous boundary and / or the IS-OS boundary) in the image and / or measurement. In some embodiments, step 2602 may further include locating the first feature as described herein in connection with method 2400 and / or method 2500.

[0110] In some embodiments, generating second derivative images and / or measurements from the images and / or measurements to locate the second feature in step 2603 may include the processor(s) generating positive and / or negative derivative images as described herein with respect to step 2602. For example, the processor(s) may generate negative derivative images that further highlight the second feature (e.g., the RPE-BM boundary) in the images and / or measurements. In some embodiments, the processor(s) may determine the location(s) of the second feature(s) using the location(s) of the first feature(s) located in step 2602 as described herein with respect to FIG. 14 and FIG. 15.

[0111] In some embodiments, selecting a subset of the images and / or measurements for locating a third feature in step 2604 may include the processor(s) applying a threshold (e.g., a pixel intensity threshold) to pixels of the images and / or measurements to select one or more subsets of pixels of the images and / or measurements that are above or below the threshold, as described herein in connection with FIG. 16. For example, pixels at a threshold level may be sorted with pixels above or below the threshold level, according to various embodiments. In some embodiments, step 2604 may further include locating a third feature(s) (e.g., the RNFL-GCL boundary) within the selected subset(s), as described herein in connection with FIG. 17. For example, the processor(s) may use the location of the first and / or second features (e.g., the IS-OS boundary) located in step 2602 and / or step 2603 to select a subset of pixels for locating a third feature, as described herein in connection with FIG. 17.

[0112] In some embodiments, step 2604 may alternatively or additionally include generating differential images and / or measurements of the same or another selected subset(s) as described herein with respect to steps 2602, 2603 to identify the location of a third feature (e.g., INL-OPL, IPL-INL, and / or OPL-ONL), as described herein with respect to Figures 19-23.

[0113] In some embodiments, method 2600 may further include some or all of the steps of method 2400 and / or method 2500 described in connection with FIG. 24 and FIG.

[0114] [IV.Application] The present inventors have developed improved imaging and measurement techniques that can be implemented using the imaging devices described herein. According to various embodiments, such imaging and measurement techniques can be used to process images.

[0115] The inventors have recognized that various health conditions may be indicated by the condition of a person's retinal fundus in one or more images captured according to the techniques described herein. For example, diabetic retinopathy may be indicated by small bulges or microaneurysms protruding from the vessel walls of smaller blood vessels, allowing fluid and blood to leak into the retina. Larger retinal vessels may also begin to expand and become irregular in diameter. Nerve fibers in the retina may also begin to swell. The central part of the retina (the macula) may also begin to swell, as in macular edema. Damaged blood vessels may close, causing the growth of new abnormal blood vessels in the retina. Glaucomatous optic neuropathy or glaucoma may be indicated by thinning of the parapapillary retinal nerve fiber layer (RNFL) and optic disc cupping, as a result of axon and secondary retinal ganglion cell loss. RNFL layer may be measured, for example, as an average value in different eye segments around the optic disc. The inventors have recognized that RNFL defects, as shown by OCT, are one of the earliest signs of glaucoma.Age-related macular degeneration (AMD) can also be shown by macula detachment and / or lifting, macular pigmentation disorder (e.g., yellowish material under the pigment epithelium layer in the central retina), and / or drusen (e.g., macular drusen, peripheral drusen, and / or granular pattern drusen).AMD can also be shown by hyperpigmentation, nummular atrophy, and / or geographic atrophy, such as a well-defined circular area of ​​subretinal fluid.

[0116] Stargardt's disease may be indicated by the death of photoreceptor cells in the central retina. Macular edema may be indicated by grooves in the area surrounding the fovea. Macular holes may be indicated by holes in the macula. Diabetic macular edema (DME) may be indicated by fluid accumulation in the retina due to damaged vascular leakage. Bulbous edema may be indicated by obscuration of the unfocused light path. Retinal detachment may be indicated by severe optic disc destruction and / or separation from the underlying pigment epithelium. Retinal degeneration may be indicated by deterioration of the retina. Age-related macular degeneration (AMD) may be indicated by thinning of the entire retina, especially the RPE layer. Wet AMD may also result in leakage in the retina. Central serous retinopathy (CSR) may be indicated by elevation of the sensory retina in the macula and / or localized detachment from the pigment epithelium. Choroidal melanoma may be indicated by malignant tumors originating from pigment cells originating in the choroid. Cataracts may be indicated by an opaque lens and may cause blurred fluorescence lifetime and / or two-dimensional retinal fundus images. Macular telangiectasia may be indicated by a ring of dramatically increased fluorescence lifetime in the macula and small blood vessels deteriorating at and around the fovea. Alzheimer's and Parkinson's disease may be indicated by thinning of the RNFL. Diabetic retinopathy, glaucoma, and other such diseases may result in blindness or severe visual impairment if not properly examined and treated. In another example, optic neuropathy, optic atrophy, and / or choroidal folds may be indicated in images captured using the techniques described herein. Optic neuropathy and / or optic atrophy may be caused by damage within the eye, such as glaucoma, optic neuritis, and / or papilledema, damage along the path of the optic nerve to the brain, such as tumors, neurodegenerative disorders, and / or trauma, and / or congenital disorders, such as Leber's Hereditary Optic Atrophy (LHOA) and Autosomal Dominant Optic Atrophy (ADOA). For example, compressive optic atrophy may be indicated by or associated with external signs, such as pituitary adenomas, intracranial meningiomas, aneurysms, craniopharyngiomas, mucous cysts, papillomas, and / or metastases, and / or optic nerve gliomas, optic nerve sheath (ONS) meningiomas, and / or lymphomas.Optic atrophy may be indicated by macular thinning with preserved foveal thickness. Vascular and / or ischemic optic atrophy may be indicated by or associated with sector disc pallor, non-arteritic anterior ischemic optic neuropathy (NAION), arteritic ischemic optic neuropathy (AION), severe optic atrophy with gliosis, giant cell arteritis, central retinal artery occlusion (CRAO), carotid artery occlusion, and / or diabetes. Neoplastic optic atrophy may be indicated by or associated with lymphoma, leukemia, tumor, and / or glioma. Inflammatory optic atrophy may be indicated by demyelination, such as sarcoid, systemic lupus erythematosus (SLE), Behçet's disease, multiple sclerosis (MS) also known as Devic's disease, and / or neuromyelitis optica spectrum disorder (NMOSD), allergic vasculitis (AN), and / or Churg-Strauss syndrome. Infectious optic atrophy may be indicated by the presence of a viral, bacterial, and / or fungal infection. Radiation optic neuropathy may also be indicated.

[0117] Further, in some embodiments, the imaging device may be configured to detect a concussion, at least in part, by tracking the movement of the person's eyes over a sequence of images. For example, the iris sensor, white light imaging components, and / or other imaging components described herein may be configured to track the movement of the person's eyes for various indicators of concussion. Toxic optic atrophy and / or nutritional optic atrophy may be indicated in association with ethambutol, amiodarone, methanol, vitamin B12 deficiency, and / or thyroid eye disease. Metabolic optic atrophy may be indicated by or associated with diabetes. Hereditary optic atrophy may be indicated by or associated with ADOA and / or LHOA. Traumatic optic neuropathy may be indicated by or associated with optic nerve trauma, ONS hematoma, and / or fracture.

[0118] Thus, in some embodiments, a person's predisposition to various medical conditions may be determined based on one or more images of the person's retinal fundus captured in accordance with the techniques described herein. For example, if one or more of the above-mentioned indicators of a particular medical condition (e.g., macular detachment and / or lifting of AMD) are detected in the captured image(s), the person may have a predisposition to that medical condition.

[0119] The inventors have also recognized that several health conditions may be detected using the fluorescence imaging techniques described herein. For example, macular holes may be detected using excitation light wavelengths of 340-500 nm to excite the retinal pigment epithelium (RPE) and / or macular pigment in the subject's eye with a fluorescence emission wavelength of 540 nm and / or 430-460 nm. Fluorescence from the RPE may be primarily due to lipofuscin from the RPE lysomes. Retinal artery occlusion may be detected by exciting flavin adenine dinucleotide (FAD), RPE, and / or nicotinamide adenine dinucleotide (NADH) in the subject's eye with a fluorescence emission wavelength between 520-570 nm using an excitation wavelength of 445 nm. AMD in drusen may be detected using excitation light wavelengths of 340-500 nm to excite the RPE in the subject's eye with a fluorescence emission wavelength of 540 nm and / or 430-460 nm. AMD, including geographic atrophy, may be detected using an excitation light wavelength of 445 nm to excite the RPE and elastin in the subject's eye with a fluorescence emission wavelength of 520-570 nm. Neovascular AMD may be detected by exciting the subject's choroid and / or inner retinal layers. Diabetic retinopathy may be detected using an excitation light wavelength of 448 nm to excite FAD in the subject's eye with a fluorescence emission wavelength of 590-560 nm. Central serous chorioretinopathy (CSCR) may be detected using an excitation light wavelength of 445 nm to excite the RPE and elastin in the subject's eye with a fluorescence emission wavelength of 520-570 nm. Stargardt's disease may be detected using an excitation light wavelength of 340-500 nm to excite the RPE in the subject's eye with a fluorescence emission wavelength of 540 nm and / or 430-460 nm. Choroideremia can be detected using excitation light wavelengths of 340-500 nm to excite the RPE in the subject's eye with fluorescent emission wavelengths of 540 nm and / or 430-460 nm.

[0120] The present inventors have also developed techniques for using captured images of a person's retinal fundus to diagnose various health problems in the person, for example, in some embodiments, any of the health conditions described above may be diagnosed.

[0121] In some embodiments, the imaging techniques described herein may be used for health status determinations, which may include determinations regarding cardiac health, cardiovascular disease and / or cardiovascular risk, anemia, retinal toxicity, body mass index, water weight, hydration status, muscle mass, age, smoking habits, blood oxygen levels, heart rate, white blood cell count, red blood cell count, and / or other such health attributes. For example, in some embodiments, a light source having a bandwidth of at least 40 nm may be configured with sufficient imaging resolution to capture red blood cells having a diameter of 6 μm and white blood cells having a diameter of at least 15 μm. Thus, the imaging techniques described herein may be configured to facilitate sorting and counting of red blood cells and white blood cells, estimating their respective densities in blood, and / or other such determinations.

[0122] In some embodiments, the imaging techniques described herein may facilitate tracking of blood cell movement to measure blood flow velocity. In some embodiments, the imaging techniques described herein may facilitate tracking of blood vessel width to provide estimates of blood pressure changes and perfusion. For example, an imaging device described herein configured to resolve red and white blood cells using a two-dimensional (2D) spatial scan completed within 1 μs may be configured to capture blood cell movement at 1 meter / second. In some embodiments, a light source that may be included in the device described herein, such as a superluminescent diode, LED, and / or laser, may be configured to emit sub-microsecond light pulses such that an image may be captured in less than 1 microsecond. Using the spectral scanning techniques described herein, an entire cross section of the scanned line (e.g., lateral) versus depth may be captured in sub-microseconds. In some embodiments, a two-dimensional (2D) sensor described herein may be configured to capture such images for slow internal or external reading and subsequent analysis. In some embodiments, a 3D sensor may be used. The embodiments described below overcome the challenge of obtaining multiple high-quality scans within 1 microsecond.

[0123] In some embodiments, the imaging device described herein may be configured to scan lines aligned along the vascular direction. For example, the scan may be rotated and positioned after identifying the vascular configuration of the subject's retinal fundus and selecting larger vessels for observation. In some embodiments, vessels that are small and only one cell can pass through them in sequence may be selected so that the selected vessels fit within a single scan line. In some embodiments, limiting the target imaging area to a smaller section of the subject's eye may reduce the collection area for the imaging sensor. In some embodiments, using a portion of the imaging sensor may facilitate increasing the imaging frame rate to tens of KH. In some embodiments, the imaging device described herein may be configured to perform high speed scans over a small area of ​​the subject's eye while reducing spectral spread interference. For example, each scan line may use a different portion of the imaging sensor array. Thus, multiple depth scans may be captured simultaneously, with each scan being captured by a separate portion of the imaging sensor array. In some embodiments, each scan may be expanded to provide a wider spacing on the imaging sensor array, such as a wider than spread spectrum, such that each depth scan may be measured independently.

[0124] Although several aspects and embodiments of the technology described herein have been described, those skilled in the art may conceive of various changes, modifications, and improvements. Such changes, modifications, and improvements are intended to be within the spirit and scope of the technology described herein. For example, those skilled in the art may readily envision various other means and / or structures for performing the functions, obtaining the results, and / or obtaining one or more of the advantages described herein, and each of such changes and / or variations is considered to be within the scope of the embodiments described herein. Those skilled in the art may recognize, or be able to ascertain using routine experimentation, many equivalents to the specific embodiments described herein. Thus, the embodiments described above are presented by way of example only, and the embodiments of the invention may be practiced otherwise than as specifically described, within the scope of the appended claims and their equivalents. Also, any combination of two or more features, systems, articles, materials, kits, and / or methods described herein is included within the scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0125] The above-described embodiments may be implemented in any of many ways. One or more aspects and embodiments of the present disclosure involving the execution of a process or method may utilize program instructions executable by a device (e.g., a computer, a processor, or other device) to execute or control the execution of the process or method. In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memories, circuitry in field programmable gate arrays or other semiconductor devices, or other tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, execute a method that realizes one or more of the various embodiments described above. The one or more computer-readable media may be transportable such that the one or more programs stored thereon can be loaded into one or more different computers or other processors to implement various of the aspects described above. In some embodiments, the computer-readable medium may be a non-transitory medium.

[0126] The terms "image" and "measurement" as used herein and in the claims mean image and / or measurement, i.e. image and measurement, image, or measurement, unless expressly indicated otherwise. The terms "image" and "measurement" mean image and / or measurement.

[0127] The above-described embodiments may be implemented in any of many ways. One or more aspects and embodiments of the present disclosure involving the execution of a process or method may utilize program instructions executable by a device (e.g., a computer, a processor, or other device) to execute or control the execution of the process or method. In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memories, circuitry in field programmable gate arrays or other semiconductor devices, or other tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, execute a method that realizes one or more of the various embodiments described above. The one or more computer-readable media may be transportable such that the one or more programs stored thereon can be loaded into one or more different computers or other processors to implement various of the aspects described above. In some embodiments, the computer-readable medium may be a non-transitory medium.

[0128] The terms "program" or "software" are used herein in a general sense to refer to any type of computer code or set of computer-executable instructions that may be employed to program a computer or other processor to implement the various aspects described above. Also, according to one aspect, one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.

[0129] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0130] Also, the data structures may be stored in the computer-readable medium in any suitable form. For ease of explanation, the data structures may be depicted as having fields that are related through their location in the data structure. Such relationships may similarly be achieved by assigning storage for the fields locations in the computer-readable medium that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information in fields of the data structure, including through the use of pointers, tags, or other mechanisms that establish relationships between data elements.

[0131] When implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0132] Further, the computer may be embodied in any of a number of forms, such as, by way of non-limiting examples, a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. A computer may also be incorporated within devices that are not generally considered computers, but which have suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device.

[0133] A computer may also have one or more input and output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output, and a speaker or other sound generating device for audible presentation of output. Examples of input devices that may be used for a user interface include keyboards and pointing devices such as mice, touchpads, and digitizing tablets. As another example, a computer may receive input information via voice recognition or in other audible formats.

[0134] Such computers may be interconnected by one or more networks of any suitable form, including local or wide area networks, such as enterprise networks, and intelligent networks (INs) or the Internet. Such networks may be based on any suitable technology and operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0135] The acts performed as part of the method may be ordered in any suitable manner, and thus, although shown in the exemplary embodiment as sequential acts, embodiments may be constructed in which acts are performed in an order different from that shown, which may include performing some acts simultaneously.

[0136] All definitions provided and used herein control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms. As used in this specification and claims, "a" means "at least one," unless expressly indicated otherwise.

[0137] The term "and / or" as used herein and in the claims means "either or both" of the elements so conjoined, i.e., elements that are conjunctive in some cases and disjunctive in other cases. A plurality of elements listed with "and / or" is to be interpreted in the same manner as "one or more" of the elements so conjoined. Other elements other than the elements specifically identified by the term "and / or" may optionally be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to "A and / or B", when used in conjunction with open-ended language such as "comprising", may refer in one embodiment to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), etc.

[0138] As used herein and in the claims, the phrase "at least one" in reference to a list of one or more elements means at least one element selected from any one or more of the elements in the list of elements, but does not necessarily include at least one of each and every element specifically listed in the list of elements, and does not necessarily exclude any combination of elements in the list of elements. This definition also allows for elements other than those specifically identified in the list of elements to which the phrase "at least one" refers, whether related or unrelated to those specifically identified elements, may optionally be present. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B," or, equivalently, "at least one of A and / or B") can refer in one embodiment to including at least one (optionally more than one) A and no B (and optionally including elements other than B), in another embodiment to including at least one (optionally more than one) B and no A (and optionally including elements other than A), in yet another embodiment to including at least one (optionally more than one) A and including at least one (optionally more than one) B (and optionally including other elements), etc.

[0139] Also, the phraseology and terminology used herein are for purposes of description and should not be regarded as limiting. The use of "including," "comprising," "having," "containing," "accompanying," and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof, as well as additional items.

[0140] In the claims, as well as in the above specification, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "accompanying," "holding," "consisting of," and the like, are open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively.

Claims

1. 1. A method comprising: generating a graph from the images and / or measurements of the retina of the subject, the generating of the graph comprising: a plurality of nodes corresponding to a plurality of pixels of the image and / or measurement results, and a plurality of edges connecting the plurality of nodes; at least one auxiliary node; and an auxiliary edge connecting the auxiliary node to a first node of the plurality of nodes.

2. the auxiliary edge is a first auxiliary edge, generating the graph further includes the at least one processor generating a second auxiliary edge connecting the at least one auxiliary node to a second node of the plurality of nodes; The method of claim 1 , wherein the first and second nodes correspond to first and second pixels, respectively, in a first column of the image and / or measurement.

3. The method of claim 1 , wherein the at least one auxiliary node includes a first auxiliary node that is a start node of the graph and a second auxiliary node that is an end node of the graph.

4. the auxiliary edge is a first auxiliary edge and the first node corresponds to a first pixel of a plurality of pixels in a first column of the image and / or measurement; Generating the graph includes the step of: a second auxiliary edge connecting the first auxiliary node to a second node of the plurality of nodes corresponding to a second pixel of the plurality of pixels in the first column; a third auxiliary edge connecting the second auxiliary node to a third node of the plurality of nodes corresponding to a third pixel of a plurality of pixels in a second column of the image and / or measurement; a fourth auxiliary edge connecting the second auxiliary node to a fourth node of the plurality of nodes corresponding to a fourth pixel of the plurality of pixels in the second column; The method of claim 3 , further comprising generating:

5. 2. The method of claim 1, further comprising: the at least one processor using the graph to identify a location of a boundary between a first layer and a second layer of the subject's retinal fundus in the images and / or measurements.

6. the at least one auxiliary node includes a start node and / or an end node of the graph; 6. The method of claim 5, wherein locating the boundary comprises determining a plurality of paths from the start node to the at least one auxiliary node and / or from the at least one auxiliary node to the end node and selecting a path from among the plurality of paths.

7. generating the graph further comprises assigning weighting values ​​to at least some of the plurality of nodes and / or edges; generating the auxiliary edges includes assigning predefined weighting values ​​to the auxiliary nodes and / or edges; 7. The method of claim 6, wherein selecting the path from among the plurality of paths comprises: performing a cost function using the weighting values ​​and the pre-set weighting values; and determining that the path has and / or shares a lowest cost among the plurality of paths.

8. 8. The method of claim 7, wherein the weighting values ​​are assigned to the nodes based on pixel intensities of pixels corresponding to the nodes and / or to the edges based on pixel intensities of pixels corresponding to pairs of the nodes connected by the edges.

9. 8. The method of claim 7, wherein the weighting values ​​are assigned to the nodes based on the frequency and / or phase of pixels corresponding to the nodes and / or the weighting values ​​are assigned to the edges based on the frequency and / or phase of pixels corresponding to pairs of the nodes connected by the edges.

10. 8. The method of claim 7, wherein performing the cost function comprises determining a cost for each of the plurality of paths, the cost of each path being based at least in part on the weighting values ​​and / or preset weighting values ​​assigned to nodes and / or edges in each path.

11. The method of claim 10 , wherein the pre-set weighting value has a minimum cost.

12. 2. The method of claim 1, further comprising shifting one or more pixels of the images and / or measurements relative to one another before generating the graph, the one or more pixels corresponding to features of the images and / or measurements.

13. the image and / or measurement comprises a plurality of pixels arranged in rows and columns; The method further comprises modifying the images and / or measurements prior to generating the graph, the modifying comprising: identifying pixels of the plurality of pixels that correspond to features of the image and / or measurements; and 2. The method of claim 1, comprising shifting one or more of the identified pixels such that the identified pixels are located along a same row or a same column of the image and / or measurement.

14. The method of claim 13 , wherein the features of the image and / or measurements include curves, and shifting the one or more pixels includes converting the curves into straight lines.

15. The method of claim 13 , wherein the features of the image and / or measurements include a boundary between a first layer and a second layer of the subject's retinal fundus.

16. 16. The method of claim 15, wherein the boundary between the first layer and the second layer of the subject's retinal fundus comprises a retinal pigment epithelium (RPE) layer.

17. The method of claim 13 , wherein generating the graph from the images and / or measurements of the subject's retinal fundus comprises generating the graph from the corrected images and / or measurements.

18. 1. A method comprising: generating a graph from the images and / or measurements of the retina of the subject, the generating of the graph comprising: generating a plurality of nodes corresponding to a plurality of pixels of the image and / or measurements, and a plurality of edges connecting the plurality of nodes; selecting a start node and / or an end node of the graph from the plurality of nodes; and generating at least one auxiliary edge connecting the start node and / or the end node to a first node of the plurality of nodes; A method comprising:

19. 20. The method of claim 18, further comprising: the at least one processor assigning weighting values ​​to at least some of the plurality of nodes and / or edges, and assigning a preset weighting value to the at least one auxiliary edge and / or a start node and / or an end node.

20. 20. The method of claim 19, wherein the weighting values ​​are assigned to the nodes based on pixel intensities of pixels corresponding to the nodes and / or to the edges based on pixel intensities of pixels corresponding to pairs of the nodes connected by the edges.

21. 20. The method of claim 19, wherein the weighting values ​​are assigned to the nodes based on derivatives corresponding to the nodes and / or to the edges based on derivatives of pixels corresponding to pairs of the nodes connected by the edges.

22. 20. The method of claim 18, wherein the start node is selected to correspond to a first corner pixel of the image and / or measurement and the end node is selected to correspond to a second corner pixel of the image and / or measurement, the first and second corner pixels being located in different columns of the image and / or measurement.

23. generating the at least one auxiliary edge generating a first plurality of auxiliary edges connecting the starting node to respective ones of a first plurality of peripheral nodes of the plurality of nodes corresponding to a plurality of pixels in a first column of the image and / or measurements; and generating a second plurality of auxiliary edges connecting the end node to respective ones of a second plurality of peripheral nodes of the plurality of nodes corresponding to a plurality of pixels in a second column of the image and / or measurements; 22. The method of claim 21 , comprising:

24. 20. The method of claim 19, further comprising: the at least one processor using the graph to identify a location of a boundary between a first layer and a second layer of the subject's retinal fundus in the images and / or measurements.

25. 25. The method of claim 24, wherein locating the boundary comprises determining a plurality of paths from the start node to the end node via the auxiliary edges and selecting a path from among the plurality of paths.

26. 26. The method of claim 25, wherein selecting the path comprises: executing a cost function based on the weighting values ​​and the pre-set weighting values; and determining that the path has and / or shares a minimum cost among the plurality of paths.

27. 27. The method of claim 26, wherein the pre-set weighting value has a minimum cost.

28. 27. The method of claim 26, wherein executing the cost function comprises minimizing the cost function.

29. 20. The method of claim 18, further comprising shifting one or more pixels of the images and / or measurements relative to one another before generating the graph, the one or more pixels corresponding to features of the images and / or measurements.

30. the image and / or measurements include a plurality of pixels arranged in rows and columns, the method comprising: and modifying the images and / or measurements prior to generating the graph, the modifying further comprising: identifying pixels of the plurality of pixels that correspond to features of the image and / or measurements; and shifting one or more of the identified pixels such that the identified pixels are located along the same row or column of the image and / or measurement; 20. The method of claim 18, comprising:

31. 31. The method of claim 30, wherein the features of the image and / or measurements include curves, and shifting the one or more pixels includes converting the curves to straight lines.

32. 31. The method of claim 30, wherein the features of the images and / or measurements include a boundary between a first layer and a second layer of the subject's retinal fundus.

33. 33. The method of claim 32, wherein the boundary between the first layer and the second layer of the subject's retinal fundus comprises a retinal pigment epithelium (RPE) layer.

34. 31. The method of claim 30, wherein generating the graph from the images and / or measurements of the subject's retinal fundus comprises generating the graph from the corrected images and / or measurements.