RAMAN ophthalmic imager guided by hyperspectral images for Alzheimer's disease pathology

Abstract

A non-invasive ophthalmic light-based detection device for detecting pathologies associated with Alzheimer's disease (AD) in the retina of an eye. The device uses two imaging modalities, one of which directs a specific area to be interrogated by the second imaging modality. A hyperspectral reflectance imaging unit detects light reflected and / or backscattered from the retina from a broadband light source, which is used to determine one or more regions of interest. A Raman spectroscopy unit detects light from a laser that is re-emitted by the retina through the phenomenon of Raman scattering, which is targeted to the region of interest determined based on the hyperspectral reflectance information. The detection information from the hyperspectral reflectance imaging unit and the Raman spectroscopy unit is used to determine the presence of the one or more pathologies associated with AD. The detection device allows identification of high-risk groups, diagnosis and tracking of patient response to treatment.

Description

[0001] Cross-references

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 590,836, filed on November 27, 2017, entitled “LIGHT-BASED OCULAR SCANNERFOR DETECTION OF ALZHEIMER'S DISEASE PATHOLOGIES,” the contents of which are incorporated herein by reference in the detailed description below. Technical Field

[0003] Example embodiments generally relate to ophthalmic light-based diagnostic detectors for detecting, localizing, and quantifying pathology associated with Alzheimer's disease in the eye. Background Art

[0004] Alzheimer's disease (AD) is a fatal neurodegenerative disorder. Disease confirmation is typically performed post-mortem. Some existing conventional diagnostic systems involve highly invasive procedures, imaging equipment that is prohibitive due to cost or complexity, or the use of harmful radioactive tracers.

[0005] Some conventional biomarker methods are used to identify pathologies associated with AD and are considered auxiliary measures that can help clinicians detect AD at an early stage and distinguish its symptoms from other forms of dementia. These technologies often assess brain deposition of amyloid or downstream neuronal damage, and include, for example, cerebrospinal fluid (CSF) measurements for beta-amyloid (Aβ) and phosphorylated Taus (a component of neurofibrillary tangles (NFTs)), positron emission tomography (PET) imaging for beta-amyloid or fluorodeoxyglucose (FDG) uptake (metabolic breakdown of the parietal and temporal lobes), and magnetic resonance imaging (MRI) for brain atrophy. However, many of these technologies are highly invasive, slow (e.g., requiring external laboratory verification), expensive, complex, difficult to obtain, or beyond the training of many clinicians, and are insufficient to identify early or asymptomatic stages of AD.

[0006] One objective is to provide a non-invasive, light-based detection system that can be easily operated and accessed by clinicians to screen patient populations for early detection of pathology associated with AD, diagnose, and track patient response to preventive or therapeutic interventions. The objective is to perform detection without exogenous fluorescent agents, dyes, or tracers.

[0007] The system is designed to detect specific characteristics of the chemical composition of parts of the eye to more specifically identify pathologies associated with AD. Summary of the Invention

[0008] Example embodiments relate to a non-invasive ophthalmic light-based detection device for detecting pathologies associated with AD in the eye. The device can be used for optical detection of portions of the fundus, such as the retina. The device is a light-based tool that provides an accessible and non-invasive procedure for identifying individuals at high risk for AD, diagnosing, and tracking the effectiveness of treatments and interventions. The device uses two imaging modalities, wherein a first imaging modality guides the operation of a second imaging modality. Using the first imaging modality, the device detects light reflected and / or scattered off the retina from a broadband light source to determine the location and size of one or more regions of interest (ROIs) requiring further interrogation. Using the second imaging modality, the device detects light re-emitted by a Raman scattering process initiated by a laser directed onto each ROI; this enables the device to detect Raman spectral information, detecting counts of one or more specific wavenumber shifts that are characteristic of the chemical composition of one or more pathologies associated with AD with high specificity.

[0009] The device is a sensitive and specific non-invasive tool for detecting one or more pathologies associated with AD and can be used for pre-screening, diagnosis, and tracking the effectiveness of treatments and interventions. Conventional optical methods for non-invasive detection suffer from a lack of specificity and sensitivity, or may rely on exogenous fluorescent agents, dyes, or tracers.

[0010] Two imaging modalities are used sequentially to determine the presence of AD-associated pathologies indicative of AD. For the first imaging modality, in an exemplary embodiment, a light source (e.g., a broadband lamp or monochromatic, patterned light) is used to acquire wide-field reflectance-based images of the subject's retina. The first imaging modality allows for the detection of abnormal regions, which may be protein oligomers or aggregates, based on their physical properties, and the identification of the location and size of one or more ROIs, which are then further interrogated using a second imaging modality using a second light source (e.g., a monochromatic laser). In an exemplary embodiment, a monochromatic laser probes each ROI to determine whether one or more specific wavenumber shifts characteristic of the chemical components of these AD-associated pathologies are achieved using Raman spectroscopy. Raman spectroscopy is a highly specific method for detecting protein aggregates or other features characteristic of AD or its precursors. In Raman spectroscopy, the target of interest (e.g., protein aggregates or other features) responds to the monochromatic laser by re-emitting (Raman scattering) light characteristic of the chemical component. This Raman scattered light is collected by the device and spectroscopically analyzed to detect the chemical signature of the pathology associated with AD.

[0011] The device does not rely on exogenous fluorescent agents, dyes or radioactive tracers. It is completely non-invasive and utilizes two distinct imaging modalities that work synergistically to produce high sensitivity and specificity in the detection of pathologies associated with AD, such as Tau protein, soluble and / or insoluble beta-amyloid species, amyloid precursor protein (APP), and surrounding neural and glial cell pathology and vascular features.

[0012] In some examples, the device uses a machine learning algorithm to operate the device and classify optical information obtained from the fundus of the subject (including the retina). The device allows for rapid, non-invasive pre-screening, diagnosis, and tracking of treatment and intervention effects (positive or negative responses) in high-risk populations for AD. While many current non-invasive optical methods for AD detection in the retina rely on the use of exogenous fluorescent agents, the device utilizes endogenous optical contrast and Raman resonance in the eye to perform highly specific detection of pathologies associated with AD without the use of exogenous fluorescent agents.

[0013] In some examples, the machine learning algorithm is implemented by the device in two steps: first, regions of interest are identified based on hyperspectral reflectance information, which is used to direct the laser of the Raman spectroscopy unit to those ROIs; second, pathologies associated with AD are classified based on the Raman spectra returned from interrogation of these specific ROIs and the hyperspectral reflectance information. In summary, these two spectroscopy modalities and the machine learning algorithm together produce a highly sensitive, highly specific, non-invasive device for pre-screening high-risk individuals for AD, diagnosing, and tracking the effectiveness of treatments and interventions.

[0014] In some examples, the machine learning algorithm is trained using verified training data. Verified training data can be obtained by comparing adjacent slices of ex vivo tissue samples from subjects known to have AD. One slice of the subject's tissue is analyzed using hyperspectral imaging and Raman spectroscopy, and then the adjacent slice is stained and verified by histology using microscopy or other imaging modalities. When AD pathology is verified in one slice using histology, the adjacent slice can be analyzed at the corresponding location using hyperspectral imaging and Raman spectroscopy, so that it can be used as verified training data for the machine learning algorithm.

[0015] A non-invasive in vivo ophthalmic light-based detection device for detecting one or more pathologies associated with AD from an eye of a subject, comprising: a hyperspectral reflectance imaging unit comprising a broadband light source and a hyperspectral camera; a Raman spectroscopy unit comprising a laser and a spectrometer; a memory; and one or more processors configured to execute instructions stored in the memory to: control the hyperspectral reflectance imaging unit to illuminate a wide field of view of the fundus using the broadband light source and detect the resulting reflected and / or backscattered light from the eye using the hyperspectral camera to determine hyperspectral reflectance information, determine one or more ROIs as potential pathologies associated with AD based on the hyperspectral reflectance information, control the Raman spectroscopy unit to illuminate each of the one or more ROIs using the laser and detect Raman scattered light from the eye generated by the laser and detected using the spectroscopy unit for determining Raman spectral information, and classify the subject as having one or more pathologies associated with AD using the hyperspectral reflectance information and the Raman spectroscopy information, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of the following: tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0016] Another example embodiment is a method for non-invasively detecting one or more pathologies associated with AD in vivo from an eye of a subject, comprising: controlling a hyperspectral reflectance imaging unit to use a broadband light source to illuminate a wide field of view of the fundus; using a hyperspectral camera to detect light from the eye generated by the broadband light source to determine hyperspectral reflectance information; using one or more processors to determine the location of one or more ROIs as potential pathologies associated with AD based on the hyperspectral reflectance information; controlling a Raman spectroscopy unit to use a laser to illuminate each of the one or more ROIs; using the spectroscopy unit to detect Raman scattered light from the eye generated by the laser to determine Raman spectral information; and using the one or more processors to classify the subject as having one or more pathologies associated with AD using the hyperspectral reflectance information and the Raman spectroscopy information, the one or more pathologies associated with AD including protein aggregates, the protein aggregates including at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0017] Another example embodiment is a computer program product with a machine learning training process, the computer program product including instructions stored in a non-transitory computer-readable medium, the instructions, when executed by a computer, causing the computer to perform non-invasive in vivo detection of one or more pathologies associated with Alzheimer's disease (AD) from an eye of a subject, the machine learning training process comprising: using one or more processors, training the computer program using verified training data, the verified training data obtained by the following operations: cutting an ex vivo tissue sample from the subject into tissue sections, placing the tissue sections on slides, staining a first tissue section of the first slide, providing a second slide having a second tissue section adjacent to the first tissue section in the tissue sample and not stained, verifying using histology that the stained first tissue section has one or more pathologies associated with AD, performing at least one imaging modality on the second slide to obtain imaging information, and classifying the imaging information into one or more pathologies associated with AD, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of the following: tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0018] Another example embodiment is a method for performing machine learning training on a computer program stored in a memory, the computer program, when executed by a computer, causing the computer to perform non-invasive in vivo detection of one or more pathologies associated with AD from an eye of a subject, the method comprising: using one or more processors, training the computer program using verified training data, the verified training data being obtained by: cutting an ex vivo tissue sample from the subject into tissue sections, placing the tissue sections on slides, staining a first tissue section of the first slide, providing a second slide having an unstained second tissue section adjacent to the first tissue section in the tissue sample, verifying using histology that the stained first tissue section has one or more pathologies associated with AD, performing at least one imaging modality on the second slide to obtain detection information, and classifying the detection information into one or more pathologies associated with AD, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of the following: tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein; and storing the trained computer program in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Reference will now be made, by way of example, to the accompanying drawings which illustrate example embodiments, in which:

[0020] Figure 1A non-invasive ophthalmic light-based eye detection apparatus for detecting AD pathology in the eye according to example embodiments is illustrated in schematic form.

[0021] Figure 2 Pictured Figure 1 Schematic side view of the interface unit of the device.

[0022] Figure 3 Pictured Figure 1 Schematic top view of the Raman spectroscopy unit of the device.

[0023] Figure 4 Shown are Raman images of unstained, formalin-fixed, paraffin-embedded (FFPE) brain tissue from a post-mortem AD patient.

[0024] Figure 5 Shown with Figure 4 The pixels of the Raman map correspond to the broadband Raman spectra of the AD plaques.

[0025] Figure 6 Shown from Figure 4 Broadband Raman spectrum of a pixel containing the Raman map of background tissue.

[0026] Figure 7 Another Raman image of unstained FFPE brain tissue from a postmortem AD patient is shown.

[0027] Figure 8 Shown with Figure 7 The pixels of the Raman map correspond to the broadband Raman spectrum.

[0028] Figure 9 Shown from Figure 7 Broadband Raman spectrum of a pixel containing the Raman map of background tissue.

[0029] Figure 10 Illustrated is a hyperspectral imaging map of patient tissue used to identify regions of interest for subsequent Raman spectroscopy according to an example embodiment.

[0030] Figure 11 A flow chart of a method for detecting AD pathology in an eye is illustrated according to an example embodiment.

[0031] Figure 12 Illustrated is a diagram for determining a Figure 1 A flow chart of a method for training data for a machine learning algorithm of a device.

[0032] Figure 13 A system for detecting AD pathology in an eye according to an example embodiment is illustrated.

[0033] Figure 14 Polarization microscope images are shown, including Figure 4 The Raman map corresponds to the patch (e.g., stained red spot).

[0034] Figure 15 Two different magnifications (10x, left, and 40x, right) of polarization microscope images of stained slides are shown, showing the Figure 7 (Mirror image) Raman map of plaques next to blood vessels.

[0035] Figure 16 A hyperspectral image of an unstained slide (left) is shown, which contains the same Figure 15 The same blood vessels seen in , and their polarization microscope image (right, mirror image).

[0036] Figure 17A The image is taken by the Raman spectroscopy unit. Figure 16 White light image of the same unstained slide showing that the Raman spectroscopy unit can be used to locate Figure 16 of the same blood vessels.

[0037] Figure 17B Pictured Figure 17A White light image showing the area that has been Raman mapped.

[0038] Like reference numerals may be used in different drawings to identify like components. DETAILED DESCRIPTION

[0039] Figure 1 A non-invasive ophthalmic in vivo light-based detection device 100 for detecting pathologies associated with AD in an eye is illustrated in accordance with an example embodiment. The device 100 can be used to perform optical detection of a portion of the fundus, such as the retina. The device 100 is a point-of-care (POC) tool that provides an accessible and non-invasive procedure for identifying high-risk populations for AD. The device 100 detects light reflected from the fundus from a broadband light source. The device 100 can also detect Raman scattered light emitted from the fundus in response to interrogation by a monochromatic laser, so that the device 100 detects the presence of one or more pathologies associated with AD with high specificity. This allows for the identification of high-risk populations based on the presence of one or more pathologies associated with AD.

[0040] Device 100 includes a Raman base station 1 and an interface unit 2 that interfaces with a subject under study. The subject can be, for example, a human or an animal. Device 100 includes a hyperspectral reflectance imaging unit and a Raman spectroscopy unit. The hyperspectral reflectance imaging unit is located in interface unit 2. The Raman spectroscopy unit is defined by the components of Raman base station 1 and interface unit 2.

[0041] The subject is positioned in front of the interface unit 2, against a rubber eye mask 4, with their chin resting on a chin rest 5. The Raman base station 1 and the interface unit 2 are connected via an optical fiber 3, which is used to deliver monochromatic laser light from the Raman base station 1 to the interface unit 2 in a narrow beam arrangement. In one example, the laser light can be 532 nm coherent light, or in another example, the laser light can be 785 nm. In other examples, other laser wavelengths can be used. Through the same interface unit 2, the optical fiber 3 also collects light re-emitted by specific areas or parts of the subject's eye in response to laser excitation due to Raman processing and delivers this re-emitted light back to the Raman base station 1 for detection by a suitable photodetector (e.g., a spectrometer). Based on the teachings herein, it will be clear to one of ordinary skill in the art that in other example embodiments, the Raman base station 1 and the interface unit 2 can be combined into a single device or further separated. A computer 6 is used to interact (control and communicate) with the Raman base station 1 and the interface unit 2. The computer includes a memory 30 and a processor 32. Cable 7 relays information back and forth between Raman base station 1 and computer 6, and coaxial cable 8 relays information back and forth between interface unit 2 and computer 6. Computer 6 processes the received information using a machine learning algorithm, which will be described in more detail herein. Computer 6 sends the output of the machine learning algorithm or other control information to interface unit 2 via cable 9, which uses the received information to direct laser light from fiber optic cable 3 to a designated area or portion of the subject's eye. In an example, computer 6 may include one or more image analysis-specific chips (e.g., a graphics processing unit or GPU) that can decompose the received imaging information and process the imaging information partially or fully in real time.

[0042] Figure 2The interface unit 2 is illustrated in more detail. The interface unit 2 may include one or more controllers or processors (not shown) for controlling the operation of the interface unit 2 and for communicating with the computer 6 and the Raman base station 1. The target of interest may be placed in front of a rubber eye mask 4. In the case of in vivo imaging, the subject will place their eyes against the eye mask 4 and their chin on the chin rest 5. This serves to align the subject with the optical path of the interface unit 2. The interface unit 2 may also be used for in vivo imaging of animals, ex vivo imaging of tissues, or any other suitable target, where a table may be attached to the interface unit to position the target in a suitable location (e.g., the focal plane of the optical system). Other components for supporting and positioning the target may be used in other examples.

[0043] The interface unit 2 may include a fundus camera or the like capable of performing wide field imaging of the fundus of the subject. A light sensor 10 capable of detecting and distinguishing light of different wavelengths is used to capture the image. The light sensor may take the form of a hyperspectral camera, a multispectral camera, a red, green, and blue color camera, or a monochrome camera. In the example, the visible and near infrared spectrum (400 nm) is covered. -1 A broadband light source 11 (100 nm) is used to illuminate the subject's retina. The broadband light source 11 passes through two beam splitters 12 and 13 and is directed onto the retina via a focusing element, such as a lens assembly 14. It will be appreciated that in other example embodiments, other focusing and beam shaping elements may be present to adjust the light distribution on the subject's eye. Once directed onto the subject's eye, at least some of the broadband light is reflected and / or backscattered from the retina or other areas of the eye. A portion of this light travels back to the interface unit 2, where it is collected by the lens assembly 14 and directed by the beam splitter 13 to the hyperspectral camera 10. Other suitable configurations for the location of the hyperspectral camera 10 and the geometry for collecting the reflected and / or backscattered light will be apparent to one of ordinary skill in the art.

[0044] The entire field of view indicated by the lens assembly 14 is detected by the light sensor 10 in a single capture. For example, in the case of a hyperspectral camera 10, all wavelength information is detected simultaneously across the entire field of view. Wide-field hyperspectral reflectance imaging units contrast with raster scanning across rows or columns of the entire field of view, or detecting one wavelength band at a time (e.g., multispectral imaging), or line hyperspectral cameras or those requiring coherent illumination light (e.g., optical coherence tomography).

[0045] In this example, the central area of ​​the subject's retina is the imaging target that fills the entire field of view. In other examples, other areas of the fundus can be used as imaging targets.

[0046] The coaxial cable 8 transmits the hyperspectral information detected by the hyperspectral camera 10 to the computer 6 for real-time processing. As described in more detail herein, the machine learning algorithm of the computer 6 uses this hyperspectral information to determine the location and size of one or more ROIs based on previously acquired training data. In some examples, the size of each ROI can be defined as a circular area (e.g., represented by a radius or diameter) or a rectangular area (e.g., indicated by M×N pixels) centered at the location. Once one or more ROIs have been identified, the device 100 can perform another imaging modality, such as Raman spectroscopy using a Raman spectroscopy unit. A second light source, such as a monochromatic laser 18 ( Figure 3 ), housed within the Raman spectroscopy unit. Light from a monochromatic laser 18 is directed toward the appropriate ROI via mirrors 15, 16, and 17. Mirrors 15, 16, and 17 are controlled by an electromechanical motor via an interface unit 2 to direct the focused laser light onto the appropriate ROI of the subject's retina, as previously identified from the hyperspectral information obtained by the hyperspectral reflectance imaging unit. Cable 9 carries signals from computer 6 to interface unit 2 to control the angles of mirrors 15, 16, and 17. The laser light interacts with the retina at the ROI and, through the Raman phenomenon, is Raman scattered with one or more specific wavenumber shifts characteristic of the chemical composition of the tissue being interrogated. This re-emitted light is collected via lens assembly 14 and then transmitted through beam splitter 13, where it is reflected by beam splitter 12 and mirrors 15, 16, and 17 before being coupled back into optical fiber 3. The optical fiber then transmits this re-emitted Raman light back to the Raman base station 1 for detection.

[0047] Raman spectroscopy can be performed on each identified ROI to identify the presence of one or more wavenumber shifts characteristic of one or more specified chemical components. Using Raman spectroscopy with mechanical mirrors 15, 16, 17, lens assembly 14, and / or apertures, computer 6 can obtain spectral information for the ROI, which can include one or more specific pixels within the tissue environment. In some examples, counts at specific wavelengths are detected, and one or more wavenumber shifts are calculated from these counts by calculating the difference from the known wavelength of monochromatic laser 18.

[0048] In an example, Raman spectroscopy information for each identified ROI having a position and size can be detected in a single capture by a Raman spectroscopy unit, and stimulated by a single instance of a monochromatic laser 18 at the position and size of the ROI. In an example, the lens assembly 14 can be used to control the size of the ROI to be stimulated by the laser light from the monochromatic laser 18 of the Raman spectroscopy unit, so that a single capture detects Raman spectroscopy information for the entire ROI. In some examples, a diaphragm, an iris, or a collimation device (not shown) can also be used to control the size of the ROI stimulated by the monochromatic laser 18.

[0049] In another example, each pixel of the ROI is scanned by stimulating each pixel with a monochromatic laser 18, and Raman spectral information is acquired by a Raman spectroscopy unit at each pixel of the ROI to create a Raman map of the ROI or calculate an integrated spectral result over the ROI. It will be appreciated that Raman scanning of the entire wide field of view of the retina is not required.

[0050] In various examples described in more detail herein, for a Raman spectroscopy unit, a filter 20 ( Figure 3 ) passes through a particular wavelength or frequency band of interest. Likewise, the computer 6 can perform digital filtering on a particular wavelength or frequency band of interest.

[0051] The operation of the hyperspectral camera 10 for performing hyperspectral imaging will now be described in more detail. The hyperspectral camera 10 comprises a two-dimensional array of photosensors, identified by pixels, that are sensitive to light in the visible and near-infrared ranges. A two-dimensional filter array is placed on top of this photosensor array. Each individual filter in the two-dimensional filter array selectively transmits light of a given wavelength, which is then detected by a dedicated pixel in the sensor array. The pattern of the filter array is repeated across the entire photosensor, so that light from every point in the field of view is filtered and detected by the sensor. In this way, all wavelength / frequency information from every area of ​​the field of view is simultaneously captured in a single acquisition. This differs from line hyperspectral cameras, which can only detect and distinguish different wavelengths of light along a one-dimensional line within the field of view. This also differs from typical multispectral approaches, which use multiple filters sequentially to capture wavelength information—that is, first capturing "red" information, then inserting a different filter to capture "green" information, and so on.

[0052] Figure 3An example embodiment of a Raman base station 1 is illustrated in more detail. A monochromatic laser 18 in the visible or near-infrared wavelength range is housed within the Raman base station 1. In an example, the laser 18 delivers coherent light at 532 nm, or in another example, at 785 nm. In other examples, the laser 18 can emit other specific wavelengths. The laser light output from the laser 18 is directed through a beam splitter 19 and coupled into an optical fiber 3 via a fiber adapter 22. The optical fiber 3 transmits the laser light to the interface unit 2. As described above, the interface unit 2 directs this laser light onto each ROI of the subject's retina (as identified by the computer 6 based on hyperspectral imaging). In an example, a lens assembly 14 and / or a diaphragm can be used to control the size (radius) of the laser light onto each ROI. In other examples, the laser light scans each ROI pixel by pixel. The laser light interacts with the tissue in these areas and, via the Raman phenomenon, the light is scattered from the tissue, where the change in wavelength is characteristic of the tissue being interrogated. This Raman scattered light is shaped and directed by collection optics, such as one or more additional lenses (not shown), so that it can be efficiently coupled into optical fiber 3 and brought back to Raman base station 1. Beam splitter 19 is used to redirect this returned light to spectrometer 21 for detection. Filter 20 may include a long-pass filter with a cutoff at 534 nm (greater than the wavelength of the laser light from laser 18) to remove any direct laser light that has experienced back reflection along the optical path and found its way back to Raman base station 1. In another example, filter 20 may include a notch filter with a narrow filter for a specific wavelength of the laser (e.g., 532 nm coherent light in one example, or 785 nm in another example). Filter 20 ensures that only light from the Raman phenomenon is detected by spectrometer 21, and light from the original laser 18 is removed by filter 20. Spectrometer 21 includes a refractive element to separate the various wavelength components and project them onto different pixels in the light sensor. The spectral information measured by the spectrometer 21 is then sent to the computer 6 via the cable 7 for further processing. As an alternative to or in combination with the physical filter 20, the computer 6 can perform further filtering (digital filtering) algorithmically. The Raman base station 1 may include one or more controllers or processors (not shown) for controlling the operation of the Raman base station 1 and for communicating with the computer 6 and the interface unit 2.

[0053] In some examples, the illumination and light collection systems may be implemented using adaptive optics (AO) systems and methods.

[0054] Then, a machine learning algorithm trained on the Raman spectra of one or more substances is executed by the computer 6 to identify one or more specific wavenumber shifts that are characteristic of the chemical composition of the source of the Raman signal, thereby specifically identifying the presence of protein aggregates or other pathologies associated with AD in the eye. This identification can include counting instances of one or more wavenumber shifts, and / or other mathematical formulas. Examples of fundus protein aggregates that can be detected by the device 100 include Tau neurofibrillary tangles (e.g., soluble or insoluble Tau oligomers or Tau fibrils), amyloid beta deposits (e.g., soluble amyloid beta aggregates or insoluble amyloid beta plaques, amyloid beta oligomers or amyloid beta precursors), and amyloid precursor protein (APP). Compared to hyperspectral imaging alone, the detection of this Raman signal allows for higher specificity in the detection of pathologies associated with AD. Pathologies associated with AD can also be tracked over time, where comparisons are made of Raman spectral information acquired from the same patient at different times to assess the classification of AD pathology or other AD conclusions. For example, the Raman count value (or ratio or other characteristic) of potential plaques at a specific ROI may increase over time in AD subjects. In some examples, machine learning algorithms use both Raman spectral information and hyperspectral reflectance information to better classify pathologies associated with AD or other conclusions associated with AD.

[0055] Computer 6 can interpret the Raman spectroscopy information and use a machine learning algorithm to classify the ROI as containing or not containing one or more pathologies associated with AD, such as protein aggregates. In an example embodiment, the classification of the subject can also be an AD conclusion regarding whether the subject has AD, or a precursor to AD, or whether the subject has been pre-screened for potential AD and requires further investigation. Computer 6 can be programmed to output the classification to a display screen, store it in local memory, or transmit it to another device (such as server 204, client station 206, or EMR server 214). Figure 13 )).

[0056] Figures 4 to 10 The diagram shows the imaging information used as training data for the machine learning algorithm. Figures 4 to 10 In FIG, the imaging information illustrates how the device 100 can be used in vivo to classify pathologies associated with AD in the eye of a particular subject (patient). Figures 4 to 10 Provide a description.

[0057] Figure 4A Raman map 400 of unstained, formalin-fixed, paraffin-embedded (FFPE) brain tissue from a postmortem AD patient is shown. Bright spots correspond to the locations of amyloid beta plaques, as independently verified by histology on adjacent ex vivo tissue sections from the same subject. This map 400 was generated by a computer 6 by plotting the signal intensity at a wavenumber of 1663 cm-1 (which corresponds to the Raman vibrational resonance of the beta-sheet protein structure) for each pixel and by subtracting the linear background of the Raman signal between 2000 cm-1 and 2500 cm-1. The axes correspond to the physical units of distance (in micrometers) of the tissue section. Bright spot 402 is located at Figure 4 In an example, for a single capture by the Raman spectroscopy unit, a specific Raman capture may cover more than one pixel of the ROI illuminated by the laser 18. In another example, each pixel of the ROI is scanned by stimulating each pixel with the monochromatic laser 18, and Raman spectral information is acquired at each pixel of the ROI by the Raman spectroscopy unit to create a Raman map of the ROI or calculate an integrated spectral result on the ROI.

[0058] Figure 5 A broadband Raman spectrum graph 500 of AD plaques is shown, which is similar to Figure 4 The bright spot 402 pixels (55um, 46um) corresponds to the graph 500. The graph 500 shows the relationship between the count of the Raman scattered light received at that pixel and the wave number shift. -1 and 1663cm -1 Peaks 502 and 504 at 1600 cm correspond to the Raman vibrational resonances of the α-helical and β-pleated sheet protein conformations, respectively; the so-called amide I band. Most of the remaining peaks correspond to the presence of paraffin wax. -1 and 1663cm -1 Peaks at these wavelengths indicate the presence of protein at this location; these peaks are relative to the low background signal present at these wavelengths in adjacent areas of the field of view ( Figure 6 , 602 and 604) are clearly visible. This confirms the local presence of the protein that is characteristic of amyloid beta plaques. Note that at 1800 cm -1 If there is no Raman signal from beta sheets at these locations, a plot showing the ratio of the Raman signal at 1663 cm-1 relative to 1800 cm-1 will show hot spots at locations corresponding to amyloid beta plaques. For a given level of background signal ("noise"), a criterion can be set based on the signal ratio at 1663 cm-1 relative to 1800 cm-1. For example, a signal-to-noise ratio of 3:1 can be used to identify the presence of amyloid beta plaques.

[0059] Figure 6 Shown Figure 4 Broadband Raman spectrum of background tissue pixels (10um, 15um) is shown in Figure 600. Note that at 1600cm -1 and 1663cm -1 The absence of peaks at α-helices and β-sheets indicates a sheet-like protein conformation lacking both α-helices and β-sheets. Independent verification was performed by histology on adjacent ex vivo tissue sections from the same subject. In some examples, the Raman spectrum graph 600 of background tissue or other Raman spectrum information can be used as control information (negative classification or as a value to be subtracted / removed) for training a machine learning algorithm.

[0060] In some examples, the Raman spectrum graph 600 obtained from the subject's ROI can be used by a machine learning algorithm to classify plaques or pathologies associated with AD. For example, the computer 6 compares the Raman spectrum graph 600 for the subject's background tissue with the Raman spectrum graph 500 for the subject's potential plaques. Figure 5 ) are compared between Raman spectral graph 600 and Raman spectral graph 500. Because Raman spectral graph 600 is from the same subject, the comparison provides useful results. The comparison may include a machine learning algorithm, comparison, formula, calculation, table, subtraction, ratio, or other comparison performed by computer 6 to classify as plaque or other pathology associated with AD.

[0061] In some examples, a Raman map of the verified training data is generated by integrating the Raman signal over a spectral region and plotting this integral for each pixel. Figure 7 An example of this is shown, where each pixel has a 1663 cm -1 and 1698cm -1 The integral counts between 2000 cm and 2000 cm are also subtracted. -1 and 2500cm -1 As independently confirmed by histology of adjacent tissue sections, Figure 7 The bright spots 702 in FIG. 5 are easily identifiable and correspond to amyloid beta plaques.

[0062] In other examples, chemometrics can be used to infer the spectral regions that best correspond to pathologies associated with AD. That is, algorithmic statistical analysis of broad Raman spectra can be performed to identify features specific to less obvious pathologies associated with AD.

[0063] In some examples, the acquired Raman signal may include a spectral region of interest (such as Figure 5Rather than a single broad spectral range, it may be one or more narrower spectral regions or bands centered around the spectrum (e.g., those identified in ).

[0064] In examples, Raman spectroscopy is performed at one or more identified ROIs rather than performing a raster scan over an extended area. In these cases, the result will be a single Raman spectrum graph (such as Figure 5 In some cases, the incoming laser beam is expanded to a larger diameter to cover a wider area, and the Raman scattered light is collected from the ROI covered by the widened laser spot size.

[0065] Figure 10 An exemplary hyperspectral image map 1000 of patient tissue is illustrated, illustrating hyperspectral imaging information that can be used to identify one or more ROIs for subsequent Raman spectroscopy. Hyperspectral image map 1000 may include multiple individual hyperspectral image maps 1000a, 1000b, ..., 1000e, each representing a count map for a specific detected wavelength from hyperspectral camera 10. Higher (or lower) counts at a pixel in a particular hyperspectral image map may indicate that the pixel warrants further investigation using Raman spectroscopy. Based on hyperspectral image maps 1000, computer 6 may use a machine learning algorithm to determine one or more ROIs 1002 (one shown), such as one or more pixels, that warrant further investigation using Raman spectroscopy. In other examples, each hyperspectral map 1000 may represent a range of wavelengths rather than a specific wavelength, with counts being specific to that particular wavelength range. In still other examples, hyperspectral map 1000 may be generated using a specific linear combination of wavelengths that best encapsulates the distinguishing characteristics of pathology associated with AD.

[0066] Another example representation of hyperspectral reflectance imaging information is a spectral graph (not shown) for each pixel or region of the subject. The spectral graph illustrates the relationship between the amount of light received at that pixel and the wavelength. Hyperspectral imaging spectral graphs can also be used for training machine learning algorithms and for classification performed by the machine learning algorithms.

[0067] In an example, as shown in the hyperspectral imaging graph 1000 or the hyperspectral imaging spectrum graph, a ROI may be determined from the hyperspectral imaging information.

[0068] Still refer to Figure 10Computer 6 uses a machine learning algorithm to determine which hyperspectral reflectance map 1000 and its corresponding wavelength to process, as some wavelengths of the hyperspectral imaging map 1000 provide better results than others. For example, rather than analyzing hyperspectral imaging maps 1000 corresponding to the entire visible and near-infrared spectra, computer 6 may select one or more specific wavelengths of the hyperspectral imaging map 1000 for further processing. In some other examples, hyperspectral imaging maps 1000 that are less relevant to the AD pathology of interest are given less weight, while more relevant hyperspectral imaging maps 1000 are given greater weight for computer 6 to determine the ROI for Raman spectroscopy.

[0069] In one example, the hyperspectral imaging graph 1000 or hyperspectral imaging spectrum graph of interest used by the computer 6 is in the visible-near infrared (VNIR) wavelength range (400 to 1400 nanometers), and can specifically be in the optical wavelength range of 460 nm to 600 nm or in the optical wavelength range of 650 nm to 950 nm, which can be more suitable for detecting protein aggregates, such as amyloid beta deposits. Different or more specific wavelength ranges are used in other example embodiments based on the pathology associated with the specific AD to be detected and the machine learning algorithm.

[0070] Reference again Figure 4 , a hyperspectral reflectance map 1000 (or a hyperspectral reflectance spectrum graph) is generated and used by computer 6 to determine the location and size of one or more specific ROIs of the subject for further analysis of the subject using Raman spectroscopy. The ROI may include one or more pixels. In this example, the ROI is Figure 4 Therefore, it is not necessary to perform Raman scanning on the entire field of view of the hyperspectral reflectance map, but rather on a local area (e.g., Figure 4 The pixel (55um, 46um) has the Raman spectrum information detected by the Raman spectroscopy unit in a single capture, and the local area is Figure 10 ROI 1002 in FIG is at the same location on the subject. In some examples, multiple pixels around a bright spot pixel or a defined radius of pixels around a bright spot pixel can also be analyzed by Raman spectroscopy in a single capture.

[0071] Figure 7 Another Raman map 700 of unstained FFPE brain tissue from a postmortem AD patient is shown. Again, the bright spots correspond to the location of amyloid beta plaques, as independently verified by histology on adjacent tissue sections from an ex vivo subject. Figure 7This graph 700 is obtained by plotting the 1663cm at each pixel. -1 and 1698cm -1 The integrated Raman signal between the two peaks was then subtracted at 2000 cm -1 and 2500cm -1 The axis corresponds to the physical unit of distance (in micrometers) between the tissue sections.

[0072] Figure 8 Shown with Figure 7 The broadband Raman spectrum curve corresponding to the bright spot 702 pixels (17um, 31um) is 800.1600cm -1 and 1663cm -1 Peaks 802 and 804 at 804 correspond to the Raman vibrational resonances of the amide I band (ie, α-helical and β-pleated sheet conformations, respectively). The remaining peaks correspond to the presence of paraffin wax.

[0073] Figure 9 Shown Figure 7 Broadband Raman spectrum of background tissue pixels (45um, 10um) in FIG900. Note that at 1600cm -1 and 1663cm -1 The absence of a peak at , indicates a sheet-like protein conformation lacking both alpha-helices and beta-sheets. This was independently verified by histology on adjacent tissue sections (see Figure 15 ). The Raman spectrum curve graph 900 or other Raman spectrum information of background tissue pixels can be used as control (negative classification) information for training machine learning algorithms. The Raman spectrum curve graph 900 or other Raman spectrum information of background tissue pixels can be used to compare with the spectrum graph 800 ( Figure 8 ) or other calculations to classify plaques.

[0074] Reference again Figure 7 , the computer 6 may use one or more hyperspectral reflectance maps 1000 ( Figure 10 ) to determine the location and size of a specific ROI of the subject's eye to be further investigated using Raman spectroscopy. In this example, the ROI is Figure 7 Therefore, when evaluating AD pathology, there is no need to Figure 7 The Raman scan is performed over the entire field of view of the Raman image 700 in FIG. Instead, Raman spectroscopy is used to compare the Raman image 700 with the Raman image 700 on the subject. Figure 10 Detection at the same position of ROI1002 such as Figure 7In one example, the Raman spectrum of the ROI is detected in a single capture by a Raman spectroscopy unit. In another example, the ROI is scanned pixel by pixel to generate a Raman map of the ROI or calculate the integrated counts of (one or more) specified wavelengths of interest.

[0075] Figures 4 to 10 The results in FIG. 4 illustrate validated training data that can be used to train classification and detection of amyloid beta plaques in subjects. In other example embodiments, other pathologies associated with AD are classified instead of or in addition to amyloid plaques. For example, when the pathology associated with AD is Tau neurofibrillary tangles, the Raman resonance wavelength of interest remains the same (1600 cm for phosphorylated Taus). -1 -1700cm -1 ), but now found inside cells. For other pathologies associated with AD, other Raman resonance wavelengths can also be used to classify and detect pathologies associated with AD.

[0076] Figure 13 A system 200 for detecting AD pathology in an eye of a subject is illustrated according to an example embodiment. In some examples, the system 200 implements a machine learning algorithm to operate the device 100 on the subject. The system 200 includes the device 100, a server 204, a client station 206, and an electronic medical record (EMR) server 214. In the system 200, there can be multiple devices of each type. The devices of the system 200 can communicate via a network 202. The client station 206 can be a computer, a laptop, a mobile phone, a tablet computer, etc. The network 202 can include a local area network (LAN), a wireless wide area network (WWAN), a private network, and the Internet. The computer 6 ( Figure 1 ) has a communication subsystem for communicating via network 202.

[0077] exist Figure 13In the embodiment of the present invention, server 204 is typically located remotely from device 100 and is configured to train the machine learning algorithm. Server 204 may include one or more dedicated servers, or one or more cloud servers. In some examples, server 204 may include or have access to a third-party machine learning platform, such as Amazon (TM) AWS, Microsoft (TM) Azure, Google (TM) Cloud, and IBM (TM) Watson. Server 204 may include a machine learning module 218 and a memory 216 for storing a database of verified training data and for storing trained neural networks. Server 204 may include one or more controllers or processors (not shown) configured to execute instructions stored in memory 216.

[0078] The EMR server 214 can be used to store, archive, and retrieve electronic medical records for patients. The EMR server 214 can include a memory that serves as a data repository for patient data. In some examples, the EMR server 214 can be a third-party server. The EMR server 214 can contain medical, demographic, and physical information about the patient. In some examples, the EMR server 214 can contain verified training data.

[0079] The memory 216 or EMR server 214 may contain previous hyperspectral imaging or Raman spectral information for a particular patient so that it can be compared with other Raman spectral information for the patient acquired at other times, allowing the computer 6 or server 204 to perform AD conclusions for the particular patient. For example, hyperspectral imaging or Raman spectral information for the same patient at different times and within the same ROI can be compared with the patient's personal medical history to examine disease progression (recovery). A patient's progression (recovery) can also be compared with other population cohorts and their historical progression (recovery).

[0080] Server 204 can implement a machine learning algorithm through one or more neural networks. The machine learning algorithm may include logistic regression, variational autoencoding, convolutional neural networks, or other statistical techniques for identifying and distinguishing pathologies associated with AD. The machine learning algorithm may also use a priori validated Raman scattering models, other scattering models, or optical physics models. The neural network may include multiple layers, some of which are defined and some of which are undefined (or hidden). The neural network is a supervised learning neural network.

[0081] In some examples, a neural network may include a neural network input layer, one or more neural network intermediate hidden layers, and a neural network output layer. Each neural network layer includes multiple nodes (or neurons). The nodes of a neural network layer are typically connected in series. The output of each node in a given neural network layer is connected to the input of one or more nodes in a subsequent neural network layer. Each node is a logical programming unit that performs an activation function (also known as a transfer function) to transform or manipulate data to generate an output based on its inputs, weights (if any), and bias factors (if any). The activation function of each node produces a specific output in response to specific inputs (one or more), weights (one or more), and bias factors (one or more). The inputs to each node may be scalars, vectors, matrices, objects, data structures, and / or other items or references thereto. Each node may store its corresponding activation function, weights (if any), and bias factors (if any) independently of other nodes. In some example embodiments, as understood in the art, a scoring function and / or a decision tree function may be used to calculate or determine the decision of one or more output nodes of the neural network output layer using previously determined weights and bias factors.

[0082] Server 204 can train the neural network using verified training data 208, which is input by a practitioner into client station 206. Additional training data sets can be obtained from EMR server 214 or from the operation of device 100 itself. For example, the operation of device 100 may result in the acquisition of hyperspectral reflectance information and Raman spectral information, which is stored in server 204 or EMR server 214. Additional subsequent Raman acquisitions can be performed at a later time to obtain more Raman spectral information. Historical trends in hyperspectral reflectance information and Raman spectral information can be confirmed at a later date to be indicative of AD or a precursor to AD. For example, years or decades later, a subject may be diagnosed with AD, and the earlier hyperspectral reflectance information and Raman spectral information can be used to classify the diagnosis as AD or pre-AD. Similarly, some subjects may have their EMR information updated in subsequent years and may be indicated as not having AD. In some examples, postmortem histology can be used to verify the patient's AD information. Histology can be performed using a microscope or other imaging modality.

[0083] In some examples, server 204 can implement two neural networks. As is understood in the art, each neural network can itself have one or more neural networks arranged in parallel, series, or otherwise. A first neural network is used to identify one or more ROIs as output of the first neural network based on hyperspectral reflectance information as input to the first neural network. A second neural network is used to classify Raman spectra returned from queries of these specific ROIs, with the Raman spectral information serving as input to the second neural network. The output of the second neural network is a classification of whether each ROI contains one or more pathologies of interest associated with AD (such as protein aggregates).

[0084] In some examples, the classification (output of the second neural network) can be one or more AD conclusions about whether the subject has AD or a precursor to AD, or whether he has been pre-screened for potential AD and needs further study. Such AD conclusions can be based on one or more AD pathologies classified by the second neural network and determined or calculated using, for example, a combined weighted score, a scorecard, or a probability determination. For example, the presence or probability classification of amyloid beta and Tau neurofibrillary tangles can lead to a higher probability conclusion of AD. In some examples, the AD conclusion can also be based on changes in the patient's pathology over time, such as by comparing with the patient's previous Raman spectroscopy information. In some examples, hyperspectral reflectance information is also used as input information to the second neural network, which further helps to classify the AD pathology.

[0085] Training of the neural network using server 204 will now be described in greater detail. Verified training data 208 is input to client station 206 and then transmitted by client station 206 to server 204. In an exemplary embodiment, verified training data 208 is obtained by comparing adjacent ex vivo tissue sections of a subject, wherein one section is analyzed to obtain hyperspectral reflectance information and Raman spectral information, and the adjacent section is verified histologically to obtain verified hyperspectral reflectance information and verified Raman spectral information. To train a first neural network, verified hyperspectral reflectance information 210 is input to client station 206. In an example, verified hyperspectral reflectance information 210 correlates counts of specific wavelengths of the hyperspectral reflectance map with one or more pathologies associated with AD. To train a second neural network, verified Raman spectral information 212 is input to client station 206. In an example, verified Raman spectral information 212 correlates counts of specific wavelengths of an ROI or Raman map with one or more pathologies associated with AD.

[0086] In some examples, the hyperspectral reflectance information can be used for more than just training the first neural network to determine the ROI. For example, the hyperspectral reflectance information can also be used to train a second neural network to help classify specific pathologies associated with AD. The hyperspectral reflectance information can be used in conjunction with the Raman spectral information and given weight or further assurance when classifying specific pathologies associated with AD. Similarly, a machine learning algorithm can determine correlations and relationships between the hyperspectral information and the Raman spectral information to classify specific pathologies associated with AD. When the computer 6 executes the trained neural network and uses both the hyperspectral reflectance information and the Raman spectral information for classification, the computer 6 can perform digital co-registration on the hyperspectral reflectance information and the Raman spectral information to align with the same ROI.

[0087] In some examples, the ROI can include a group of pixels covering the plaque. In one example, the size of the plaque (e.g., a circular area indicated by a radius or a rectangular area indicated by M×N pixels) is used to classify pathologies associated with AD. In some examples, the Raman spectral information 212 can have higher counts for a specific wavelength at the center of the ROI and fewer counts (but still higher than background tissue) at the periphery of the ROI. In some examples, the individual counts at different pixels within the ROI can be used to classify pathologies associated with AD. In other examples, for example, in a single Raman capture, the aggregate (integral) characteristics of the group of pixels in the ROI can be used to classify the plaque. Therefore, the size of the ROI of the plaque can also be used as part of training the second neural network as additional information to classify the plaque.

[0088] In some examples, Raman spectral information 212 of the subject's background tissue is also included in the verified training data 208. The Raman spectral information of the background tissue of a given patient can be used to compare with the Raman spectral information of the ROI of that patient. The comparison between the background tissue and the ROI can be part of the training of the second neural network to classify pathologies associated with AD. Other algorithms or calculations (including logistic regression, variational autoencoding, convolutional neural networks, and other statistical methods) can be used for supervised training of the second neural network.

[0089] Once server 204 has trained the neural network, server 204 can transmit the trained neural network to device 100 for execution by computer 6. Computer 6 is now informed of the criteria that should be used to assess pathology associated with AD. Training updates to the neural network can be performed by server 204 periodically, in real time, or whenever more training data is available, and those updated neural networks can be sent to device 100.

[0090] In other examples, at least some or all of the neural network is executed by server 204, and detected hyperspectral reflectance information, detected Raman spectral information, and control information are transmitted between server 204 and computer 6. In such examples, server 204 executes the neural network by receiving hyperspectral reflectance information from computer 6 and indicating to computer 6 what the ROI is for the Raman spectral cell. Server 204 receives the Raman spectral information from computer 6 and classifies a pathology associated with AD or a conclusion of AD.

[0091] Figure 11 100 for detecting pathology associated with AD in an eye of a subject, according to an example embodiment. In an example embodiment, the computer 6 of the device 100 uses a neural network for at least some of the method 1100. At step 1102, the device 100 controls the hyperspectral reflectance imaging unit ( Figure 1 ) and receives the hyperspectral reflectance information from the hyperspectral reflectance imaging unit to perform wide-field imaging of the fundus of the subject. At step 1104, using the hyperspectral reflectance information from the hyperspectral reflectance imaging unit and the first neural network, the computer 6 determines the location and size of one or more ROIs of the subject that require further examination. At step 1106, the device 100 controls the Raman spectroscopy unit ( Figure 1) to stimulate the ROI at the determined position and size in a single capture and receive Raman spectral information from the Raman spectroscopy unit to perform Raman spectroscopy on the ROI. At step 1108, the second neural network uses the Raman spectral information of the ROI obtained from the Raman spectroscopy unit and the hyperspectral reflectance information from the hyperspectral reflectance imaging unit to classify one or more pathologies associated with AD or AD conclusions. At step 1110, the device 100 outputs the classification(s) to an output device (e.g., a display screen), a memory, or another computer. In some other examples, steps 1108 and 1110 are performed by the server 204. In an example embodiment, when multiple pathologies associated with AD are of interest to be detected, the method 1100 can be performed (determined in parallel) for all pathologies associated with AD on one or more ROIs to detect all AD-associated pathologies of interest. When a practitioner, histologist, pathologist, etc. makes a positive diagnosis on the same sample, such practitioner can use the client station 206 to positively (independently) verify whether the subject suffers from one or more pathologies associated with AD or AD. The machine learning algorithm (first and second neural networks) can use this verification as further training data to improve the machine learning algorithm.

[0092] In some examples, after step 1106, device 100 can be configured to loop 1112 back to step 1102 to determine the Raman spectrum of another ROI identified by the hyperspectral reflectance imaging unit that may require further investigation by the Raman spectrometer. Loop 1112 can be executed within the same session, for example, while the user remains on chin rest 5. For example, at step 1104, computer 6 may have identified more than one ROI on the subject that may require further examination, and therefore loop 1112 is executed to investigate those additional ROIs. The classification at step 1108 can provide a conclusion based on multiple different individual captures of the same subject taken by the hyperspectral reflectance imaging unit and the Raman spectroscopy unit. In other examples, loop 1112 is not executed, and only a single Raman capture is performed on an ROI having a specific location and size determined based on the hyperspectral reflectance imaging information.

[0093] In some examples, using the hyperspectral reflectance information from the hyperspectral reflectance imaging unit, the computer 6 determines a baseline ROI (using a machine learning algorithm or a default location) associated with a portion of the eye that is not potentially pathologically associated with AD. The baseline ROI can be analyzed using Raman spectroscopy. At step 1108, the computer 6 can compare the baseline ROI with one or more of the ROIs analyzed using Raman spectroscopy to classify one or more pathologies associated with AD or a conclusion of AD.

[0094] In some examples, at step 1104, the computer 6 has pre-saved one or more potential AD-associated pathologies of interest (or specific ROIs) with respect to that particular patient (or verified from a known patient population). For example, a previous session using the device 100 has pre-saved one or more potential AD-associated pathologies. Specific landmarks can be used to locate one or more potential AD-associated pathologies in a particular patient, such as arterial vessels, optic nerves, etc. Using the hyperspectral reflectance information from the hyperspectral reflectance imaging unit and the first neural network, the computer 6 locates those pre-saved potential AD-associated pathologies (or specific ROIs) of the patient, determines the appropriate ROI, and then the computer 6 further investigates the appropriate ROI using the Raman spectrometer, all during the same session while the user is still on the chin rest 5.

[0095] Figure 12 A flow chart of a method 1200 for determining verified training data 208 for a neural network according to an example embodiment is illustrated. Generally speaking, verified training data 208 can be obtained by comparing adjacent ex vivo tissue slices from a subject, where one slice is analyzed by a hyperspectral reflectance imaging unit and a Raman spectroscopy unit, and an adjacent slice is verified by histology. An example result of method 1200 is Figures 4 to 9 The Raman spectrum information 212 shown in FIG. Figure 10 Hyperspectral reflectance information 210 shown in hyperspectral map 1000 shown in

[0047] Additionally, in some examples, in vivo imaging from operation of device 100 may also be used to obtain further training data.

[0096] In method 1200, ex vivo human brain tissue (cortex) from a deceased confirmed AD patient is obtained. Both fresh frozen and formalin-fixed, paraffin-embedded (FFPE) are used as samples. At step 1202, the sample is sliced ​​and placed on a slide. 12 μm thick sample slices are cut using a microtome or cryostat. A series of adjacent such slices are cut and placed on a microscope slide. At step 1204, each second slice in the series is stained with Congo red, which binds to amyloid beta, or a similar staining procedure, such as immunostaining. The remaining intermediate slides are unstained. At step 1206, amyloid beta plaques are identified on the stained slides using standard polarizing microscopy or other histological methods. Histology can be performed manually by a clinician, automatically by a computer, or both. The typical size of brain plaques is greater than 20 μm in diameter; therefore, there is a high probability that a given plaque spans multiple 12 μm slices.

[0097] At step 1208, the stained slides having one or more plaques are each co-registered with their adjacent unstained slides. Co-registration can be done automatically using a computer, performed manually, or both. Co-registration of adjacent slides allows identification of the location of the plaque on the unstained slide. Co-registration can be achieved by viewing multiple features at various scales. Folds in the cortex provide large-scale features for the general orientation of two adjacent slices. Blood vessels constitute smaller features that are used to co-register adjacent slices on a finer scale. Using multiple blood vessels within an image and co-locating them in adjacent slices facilitates locating a given plaque to within a few microns. In one example, alignment of the blood vessels allows co-registration of the images by overlaying images of two adjacent slices.

[0098] At step 1210, an imaging modality is performed on a co-registered position of an adjacent unstained slide at a corresponding position on the stained slide to determine imaging characteristics of pathology associated with AD. In other examples, the entire adjacent unstained slide is imaged using an imaging modality, for example, to obtain additional information about background tissue, other pathologies associated with AD, macrostructures, etc. In example embodiments, the imaging modality can be hyperspectral reflectance imaging or Raman spectroscopy, as described in detail herein. At step 1212, after imaging information of the plaque is acquired, the imaging information (or processed imaging information) is classified as a plaque. In this example, the verified hyperspectral reflectance information 210 is associated with amyloid beta plaques. An example of verified hyperspectral reflectance information 210 is Figure 16 In this example, the verified Raman spectral information 212 is also associated with amyloid beta plaques, see Figure 7Examples of the Raman spectrum information 212 are respectively Figure 4 and 8 Raman spectrum information 500, 800 is shown in FIG.

[0099] Verified training data 208 is input to client station 206 and may be transmitted to server 204 for use in training the neural network.

[0100] Method 1200 can be repeated for other pathologies (such as Tau disease, other protein aggregates and vascular characteristics) associated with AD, to obtain further verified training data 208. Any non-plaque area of ​​the background tissue detected using the image modality can be used as control information or negative classification information, to train neural network. Background tissue can also be used to perform subtraction or division calculation from the counting of ROI, so that ROI is classified as plaque. Other non-plaque areas can have the macrostructure for common registration, for example, for the relative position information of part of interest in the subject's body. The detection of background tissue can also be used for the training of neural network.

[0101] Method 1200 can be repeated for multiple tissue samples from a subject (e.g., brain tissue and eye tissue from a subject). Method 1200 can be repeated for tissue samples from different subjects. By using multiple tissue samples, a sufficient sample set is used to determine verified training data 208 for the machine learning algorithm. Baseline or control training data can also be obtained by obtaining hyperspectral reflectance information 210 and Raman spectral information 212 from healthy (non-AD) subjects.

[0102] Figure 14 A polarization microscope image 1400 is shown, which includes Figure 4 Polarization microscope image 1400 can be used to verify that Raman map 700 includes Raman spectrum information for plaque 1402 at the same ROI.

[0103] Figure 15 Two different magnifications (10x, left and 40x, right) of polarization microscope images of stained slides are shown, showing the Figure 7 The Raman map 700 corresponds to plaques 1502, 1504 (eg, plaques dyed red) next to the blood vessel (mirror image). The polarization microscope image can be used to verify that the Raman map 700 includes Raman spectral information of the plaques 1502, 1504 at the same ROI.

[0104] Figure 16A hyperspectral image 1600 (left) and a polarization microscope image (right) of an unstained slide are shown, the hyperspectral image being included in adjacent sections. Figure 15 The same blood vessel 1602 is seen, and the polarization microscope image shows the same blood vessel 1604 in the same adjacent slice.

[0105] The Raman spectroscopy unit 1 may also be configured to capture white light images. Figure 17A The diagram shows the Raman spectroscopy unit taking Figure 16 White light image 1700 of the same unstained slide (same adjacent section) is shown. Figure 16 The same blood vessel 1702 is shown Figure 17A This can be located and captured by the device 100 using a Raman spectroscopy unit.

[0106] Figure 17B Pictured Figure 17A White light image 1700 of FIG. 1 shows a region 1704 of a blood vessel 1702 that has been captured using Raman spectroscopy. Region 1704 can have a specific location and size determined by hyperspectral image 1600 and can be captured in a single capture by the Raman spectroscopy unit. In other examples, region 1704 can be scanned pixel by pixel using the Raman spectroscopy unit to generate a map or calculate integrated counts for a specific wavelength.

[0107] Reference again Figure 1 In some examples, device 100 can implement other types of imaging modalities to determine one or more specific wavelengths that are characteristic of the chemical composition of the underlying AD pathology in the ROI. In an example, the imaging modality is implemented using the inverse Raman effect, where one source is a broadband light source and the second source is a coherent single wavelength source at a specific wavelength.

[0108] In another example, the imaging modality is achieved using the stimulated Raman effect and two coherent lasers at a specific wavelength. In another example, the imaging modality is achieved using autofluorescence measurements at several different wavelengths using a pulsed light coherent illumination source.

[0109] In some examples, another imaging modality can be used, such as a white light non-hyperspectral fundus camera. This additional imaging modality can be used to guide the positioning of the wide field of view of the hyperspectral reflectance imaging unit. This positioning can be performed automatically by the computer 6 using image information from this additional imaging modality and / or can be performed manually by the operating clinician. The computer 6 can use machine learning and one or more neural networks to automatically perform positioning.

[0110] In an example embodiment, some of the components of system 200 are mounted, tightened, and enclosed to reduce relative movement, vibration, and reduce the amount of extraneous electromagnetic radiation entering system 200 .

[0111] In example embodiments, the computer 6, the server 204, and any of the devices of the system 200 may include one or more communication subsystems (wired or wireless) and one or more controllers. Depending on the specific application, component, or function, the controller may include hardware, software, or a combination of hardware and software. In some example embodiments, the one or more controllers may include analog or digital components and may include one or more processors, one or more non-transitory storage media (such as a memory storing instructions executable by one or more processors), and / or one or more analog circuit components.

[0112] An example embodiment is a non-invasive in vivo ophthalmic light-based detection device for detecting one or more pathologies associated with AD from an eye of a subject, comprising: a hyperspectral reflectance imaging unit including a broadband light source and a hyperspectral camera; a Raman spectroscopy unit including a laser and a spectrometer; a memory; and one or more processors configured to execute instructions stored in the memory to: control the hyperspectral reflectance imaging unit to illuminate a wide field of view of the fundus using the broadband light source, and to detect the resulting reflected and / or backscattered light from the eye using the hyperspectral camera to determine hyperspectral reflectance information, and to determine one or more ROIs based on the hyperspectral reflectance information. potential pathologies associated with AD, controlling the Raman spectroscopy unit to illuminate each of the one or more ROIs using a laser, and detecting Raman scattered light from the eye generated by the laser and detecting it using a spectrometer for determining Raman spectral information, and using the hyperspectral reflectance information and the Raman spectral information to classify the subject as having one or more pathologies associated with AD, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0113] In any of the above example embodiments, classifying the subject as having one or more pathologies associated with AD is further based on hyperspectral reflectance information.

[0114] In any of the above exemplary embodiments, classifying the subject as having one or more pathologies associated with AD is further based on previous hyperspectral reflectance information and / or Raman spectroscopy information of the subject stored in a memory or another device.

[0115] In any of the above exemplary embodiments, classifying the subject as having one or more pathologies associated with AD is further based on changes in the subject's hyperspectral reflectance information and / or Raman spectroscopy information over time.

[0116] In any of the above example embodiments, classifying the subject as having one or more pathologies associated with AD includes classifying the subject as having multiple pathologies associated with AD.

[0117] In any of the above example embodiments, the one or more processors are further configured to: determine the baseline ROI as background tissue that does not contain potential pathology associated with AD based on the hyperspectral imaging information, and control the Raman spectral imaging unit to illuminate the baseline ROI of the eye using a laser, and use a spectrometer for determining Raman spectral information of background tissue to detect light from the eye generated by the laser, wherein the classification is also based on comparing the Raman spectral information of potential pathology associated with AD with the Raman spectral information of background tissue.

[0118] In any of the above example embodiments, the one or more pathologies associated with AD include two or more of the pathologies associated with AD, including Tau neurofibrillary tangles.

[0119] In any of the above example embodiments, the one or more pathologies associated with AD include neural or glial pathology of the subject's eye, or vascular features of the blood vessels or choroid of the subject's eye.

[0120] In any of the above exemplary embodiments, when the one or more pathologies associated with AD include amyloid beta deposits, the classification is based on the presence of a 1600 cm -1 to 1700cm -1 The Raman spectral information is analyzed at one or more wavenumber shifts in the range of , which correspond to the Raman vibration resonances of α-helix and β-pleated sheet.

[0121] In any of the above example embodiments, when the one or more AD-related pathologies include Tau neurofibrillary tangles, wherein the classification is based on the -1 to 1700cm -1 The Raman spectral information is analyzed at one or more wavenumber shifts in the range of , which correspond to the Raman vibrational resonances of phosphorylated Taus.

[0122] In any of the above example embodiments, the one or more processors utilize a machine learning algorithm for one or both of: determining the one or more ROIs; or classifying the subject as having one or more pathologies associated with AD.

[0123] In any of the above example embodiments, the machine learning algorithm uses verified training data.

[0124] In any of the above example embodiments, the verified training data is obtained by the following operations: cutting an ex vivo tissue sample from a subject into tissue sections; placing the tissue sections on slides; staining a first slide of one of the tissue sections; providing a second slide having another tissue section adjacent to the first tissue section in the tissue sample and not stained; using histology to verify that the first slide has one or more pathologies associated with AD; performing at least one imaging modality on the second slide to obtain imaging information; and classifying the imaging information into one or more pathologies associated with AD.

[0125] In any of the above example embodiments, the at least one imaging modality is a Raman spectroscopy unit, a hyperspectral reflectance imaging unit, or both.

[0126] In any of the above example embodiments, the machine learning algorithm uses one or more neural networks.

[0127] In any of the above example embodiments, the one or more processors are further configured to: further train the machine learning algorithm using the following operations: i) classification of the one or more pathologies associated with AD, and ii) independent verification that the subject suffers from the one or more pathologies associated with AD.

[0128] In any of the above example embodiments, the one or more processors are further configured to classify the subject as having AD, or a precursor to AD, or a pre-screening classification for potential AD requiring further investigation, or responsiveness to treatment or intervention based on the Raman spectroscopy information.

[0129] In any of the above example embodiments, no exogenous fluorescer, dye, or tracer is required for classification of the one or more pathologies associated with AD.

[0130] In any of the above example embodiments, the apparatus further comprises one or more filters to filter out wavelengths of the laser before being detected by the spectrometer.

[0131] In any of the above example embodiments, the one or more processors are configured to determine a corresponding size of each of the one or more ROIs based on the hyperspectral reflectance information, and control the Raman spectroscopy unit to emit laser light onto each ROI of the eye having the corresponding size.

[0132] In any of the above example embodiments, the Raman spectroscopy unit is controlled by one or more processors to scan the corresponding ROI using a laser of the Raman spectroscopy unit for each of the one or more ROIs to determine Raman spectroscopy information.

[0133] In any of the above example embodiments, wherein the hyperspectral camera comprises: a two-dimensional array of light sensors, each light sensor being sensitive to a range of wavelengths of light; and a two-dimensional array of filters covering the light sensor array, each individual filter selectively transmitting light of a specific wavelength.

[0134] Another example embodiment is a method for non-invasively detecting one or more pathologies associated with AD in vivo from an eye of a subject, the method comprising: controlling a hyperspectral reflectance imaging unit to use a broadband light source to illuminate a wide field of view of the fundus; using a hyperspectral camera to detect light from the eye generated by the broadband light source to determine hyperspectral reflectance information; using one or more processors to determine the location of one or more ROIs as potential pathologies associated with AD based on the hyperspectral reflectance information; controlling a Raman spectroscopy unit to use a laser to illuminate each of the one or more ROIs; using a spectrometer to detect Raman scattered light from the eye generated by the laser to determine Raman spectral information; and using the one or more processors to classify the subject as having one or more pathologies associated with AD using the hyperspectral reflectance information and the Raman spectral information, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0135] Another example embodiment is a computer program product with a machine learning training process, the computer program product including instructions stored in a non-transitory computer-readable medium, the instructions, when executed by a computer, causing the computer to perform non-invasive in vivo detection of one or more pathologies associated with Alzheimer's disease (AD) from an eye of a subject, the machine learning training process comprising: using one or more processors, training the computer program using verified training data, the verified training data obtained by the following operations: cutting an ex vivo tissue sample from the subject into tissue sections, placing the tissue sections on slides, staining a first tissue section of the first slide, providing a second slide having a second tissue section adjacent to the first tissue section in the tissue sample and not stained, verifying using histology that the stained first tissue section has pathologies associated with AD, performing at least one imaging modality on the second slide to obtain imaging information, and classifying the imaging information into one or more pathologies associated with AD, the one or more pathologies associated with AD comprising protein aggregates, the protein aggregates comprising at least one of: tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

[0136] In any of the above example embodiments, verified training data is further obtained by co-registering the first tissue slice with the second tissue slice.

[0137] In any of the above example embodiments, the at least one imaging modality is a Raman spectroscopy unit, a hyperspectral reflectance imaging unit, or both a Raman spectroscopy unit and a hyperspectral reflectance imaging unit.

[0138] In any of the above example embodiments, performing at least one imaging modality includes performing at least two imaging modalities that are collectively used to classify the imaging information into one or more of the pathologies associated with AD.

[0139] Another example embodiment is a method for machine learning training of a computer program stored in a memory that, when executed by a computer, causes the computer to perform non-invasive in vivo detection of one or more pathologies associated with AD from an eye of a subject, the method comprising: using one or more processors, training the computer program using verified training data, the verified training data obtained by: cutting an ex vivo tissue sample from the subject into tissue sections, placing the tissue sections on slides, staining a first tissue section of the first slide, providing a second slide having an unstained second tissue section adjacent to the first tissue section in the tissue sample, verifying using histology that the stained first tissue section has one or more pathologies associated with AD, performing at least one imaging modality on the second slide to obtain detection information, and classifying the detection information as one or more pathologies associated with AD, the one or more pathologies associated with AD comprising protein aggregates comprising at least one of: tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein; and storing the trained computer program in the memory.

[0140] Although some embodiments of the present invention are described in terms of methods, those skilled in the art will appreciate that embodiments of the present invention are also directed to various apparatuses, such as controllers, processors, and circuit systems that include components for performing at least some aspects and features of the described methods, whether through hardware components, software, any combination of the two, or in any other manner as applicable.

[0141] In the accompanying drawings, where applicable, at least some or all of the subsystems or modules shown may include or be controlled by a processor that executes instructions stored in a memory or non-transitory computer-readable medium. Changes may be made to some of the example embodiments, which may include combinations and sub-combinations of any of the above examples. The various embodiments presented above are merely examples and are in no way meant to limit the scope of this disclosure. The innovative variations described herein will be clear to those of ordinary skill in the art who benefit from the example embodiments, and such variations are within the intended scope of this disclosure. In particular, features from one or more of the above-described embodiments may be selected to create alternative embodiments consisting of sub-combinations of features, which may not be explicitly described above. In addition, features from one or more of the above-described embodiments may be selected and combined to create alternative embodiments consisting of combinations of features not explicitly described above. After reviewing this disclosure as a whole, features suitable for such combinations and sub-combinations will be clear to those skilled in the art. The subject matter described herein is intended to cover and encompass all appropriate changes in technology.

[0142] Certain adaptations and modifications may be made to the described embodiments.Accordingly, the embodiments discussed above are considered to be illustrative rather than restrictive.

Claims

1. A non-invasive in vivo ophthalmic light-based detection system comprising: a spectral reflectance imaging unit comprising a broadband light source and a spectral camera, wherein the spectral camera comprises a hyperspectral camera or a multispectral camera; Raman spectroscopy unit, the Raman spectroscopy unit includes a light source and a spectrometer; as well as One or more processors configured to execute instructions stored in the memory to: controlling the spectral reflectance imaging unit to illuminate a wide field of view of the fundus using a broadband light source, and detecting reflected and / or backscattered light from the eye generated by the broadband light source using a spectral camera to determine spectral reflectance information, determining one or more regions of interest (ROIs) as potentially associated with Alzheimer's disease (AD) pathology based on the spectral reflectance information, and controlling the Raman spectroscopy unit to illuminate each of the one or more ROIs using the light source, detecting Raman scattered light from the eye generated by the light source, and determining Raman spectral information using a spectrometer, The spectral reflectance information and the Raman spectral information are used to classify the subject as having one or more pathologies associated with AD, wherein the one or more pathologies associated with AD include protein aggregates, the protein aggregates including at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

2. The detection system of claim 1, wherein classifying the subject as having one or more pathologies associated with AD is further based on previous spectral reflectance information and / or Raman spectral information of the subject stored in a memory or another device. 3 . The detection system of claim 1 , wherein classifying the subject as having one or more pathologies associated with AD is further based on changes in the subject's spectral reflectance information and / or Raman spectral information over time. 4 . The detection system of claim 1 , wherein classifying the subject as having one or more pathologies associated with AD comprises classifying the subject as having multiple pathologies associated with AD.

5. The detection system of claim 1 , wherein the one or more processors are further configured to: A baseline ROI is defined based on spectral reflectance information as background tissue that does not contain potential AD-associated pathology, and Control the Raman spectroscopy imaging unit to use the light source to illuminate the baseline ROI of the eye, and Light from the eye generated by a light source is detected using a spectrometer for determining Raman spectral information of background tissue, wherein the classification is further based on comparing the Raman spectral information of potential AD-associated pathology with the Raman spectral information of the background tissue. 6 . The detection system of claim 1 , wherein the one or more pathologies associated with AD include two or more pathologies associated with AD, the two or more pathologies associated with AD including Tau neurofibrillary tangles.

7. The detection system of claim 1, wherein the one or more pathologies associated with AD include neural or glial cell pathology of the subject's eye, or vascular characteristics of blood vessels of the subject's eye.

8. The detection system according to claim 1, wherein: When the one or more pathologies associated with AD include amyloid beta deposits, classification is based on the presence of -1 to 1700cm -1 The Raman spectral information is analyzed at one or more wavenumber shifts in a range of , which correspond to the Raman vibration resonances of α-helices and β-pleated sheets.

9. The detection system according to claim 1, wherein: When the one or more AD-related pathologies include Tau neurofibrillary tangles, wherein the classification is based on the -1 to 1700cm -1 The Raman spectral information is analyzed at one or more wavenumber shifts in the range of , which correspond to the Raman vibration resonance of phosphorylated Tau.

10. The detection system of claim 1, wherein the one or more processors utilize a machine learning algorithm for one or both of: determining the one or more ROIs; or classifying a subject as having one or more pathologies associated with AD.

11. The detection system of claim 1 , wherein the one or more processors are further configured to classify the subject as having AD, or a precursor to AD, or a pre-screening classification for potential AD requiring further investigation, or responsiveness to treatment or intervention based on the Raman spectroscopy information.

12. The detection system of claim 1, wherein no tracer is required for classification of the one or more pathologies associated with AD.

13. The detection system of claim 1, further comprising one or more filters to filter out wavelengths of the light source before being detected by the spectrometer.

14. The detection system of claim 1 , wherein the one or more processors are configured to determine a corresponding size of each of the one or more ROIs based on the spectral reflectance information, and control the Raman spectroscopy unit to emit a light source onto each ROI of the eye having the corresponding size. 15 . The detection system of claim 1 , wherein the Raman spectroscopy unit is controlled by the one or more processors to scan each of the one or more ROIs using a light source of the Raman spectroscopy unit to determine Raman spectral information.

16. The detection system of claim 1, wherein the spectral camera comprises: A two-dimensional array of light sensors, each sensitive to a range of wavelengths of light; as well as A two-dimensional array of filters overlays the photosensor array, with each individual filter selectively transmitting light of a specific wavelength.

17. The detection system of claim 1, wherein the pathology associated with AD comprises additional protein aggregates, the additional protein aggregates comprising one or more amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

18. The detection system of claim 1, wherein the one or more pathologies associated with AD include vascular characteristics of the choroid of the subject's eye.

19. The detection system of claim 1, wherein no exogenous fluorescent agent is required for classification of the one or more pathologies associated with AD.

20. The detection system of claim 1, wherein no dye is required for classification of the one or more pathologies associated with AD.

21. A non-invasive in vivo ophthalmic light-based detection system comprising: a first imaging modality; a second imaging modality; and One or more processors configured to execute instructions stored in memory to: controlling the first imaging modality to illuminate a wide field of view of the fundus and detecting light from the eye to determine first imaging modality information, determining one or more regions of interest (ROIs) as potentially associated with pathology associated with Alzheimer's disease (AD) from the first imaging modality information, controlling a second imaging modality to illuminate each of the one or more ROIs and detecting light from the eye to determine second imaging modality information, and Using the first imaging modality information and the second imaging modality information, the subject is classified as having one or more pathologies associated with AD, wherein the one or more pathologies associated with AD include protein aggregates, which protein aggregates include at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

22. The detection system of claim 21, wherein: The first imaging modality is a reflectivity system comprising a hyperspectral or multispectral camera, and the second imaging modality is a Raman spectroscopy device.

23. A method for non-invasively detecting one or more pathologies associated with Alzheimer's disease (AD) in vivo from an eye of a subject, comprising: Using one or more processors, controlling a spectral reflectance imaging unit to use a broadband light source to illuminate a wide field of view of the fundus; determining, using one or more processors, spectral reflectance information based on detecting light from the eye generated by a broadband light source using a spectral camera, the spectral camera comprising a hyperspectral camera or a multispectral camera, wherein the spectral reflectance imaging unit comprises the broadband light source and the spectral camera; determining, using one or more processors, locations of one or more regions of interest (ROIs) as potential pathologies associated with AD based on the spectral reflectance information; Using one or more processors, controlling a Raman spectroscopy unit including a light source and a spectrometer to illuminate each of the one or more ROIs using the light source, and detecting Raman scattered light from the eye generated by the light source, and determining Raman spectral information using the spectrometer, and Using one or more processors, the spectral reflectance information and the Raman spectral information are used to classify the subject as having one or more pathologies associated with AD, wherein the one or more pathologies associated with AD include protein aggregates, the protein aggregates including at least one of the following: Tau neurofibrillary tangles, amyloid beta deposits, soluble amyloid beta aggregates, or amyloid precursor protein.

Images