Spatial-spectral imaging method and retinal imaging device
Through the combination of beam splitter and imaging spectrometer, synchronous recording of spatial images and spectral images is achieved, solving the problem that spectral imaging systems cannot synchronously record and have low resolution in the prior art, and improving the accuracy and application breadth of spectral imaging .
Patent Information
- Application Number
- CN202110892209.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2016-03-10
- Filing Date
- 2017-03-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2037-03-10
AI Technical Summary
Existing spectral imaging systems cannot synchronize spatial images and spectral images, and cannot obtain continuous spectra with regular interval wavelengths, resulting in spectral reconstruction requiring a large amount of calculations and low spectral resolution.
The beam is divided into the first and second beams through the beam splitter, and is directed to the spatial camera and the spectral camera respectively. The incident slit of the imaging spectrometer defines the correspondence between the image area divided into the spectrum and the spatial image area to realize the spectrum and the spectrum and the spectrum and the spectrum and the incident slit are used to define the corresponding relationship between the image area divided into the spectrum and the spatial image area. Synchronous recording and one-to-one mapping of spatial images.
The accurate correspondence between spectral and spatial images is achieved, and the resolution and accuracy of spectral imaging is improved. It is suitable for applications in multiple medical and materials science fields, including ophthalmic disease screening and wound healing evaluation.
Smart Images

Figure CN113520301B_ABST
Abstract
Description
[0001] This application is a divisional application of the Chinese national phase application of the PCT application with an application date of March 10, 2017, an international application number of PCT / US2017 / 021787, and an invention name of “Spectral-Spatial Imaging Device”. The entry date of the Chinese national phase application into the Chinese national phase is November 7, 2018, and the application number is 201780028377.X. All the contents are incorporated herein by reference.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS
[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 306,520, filed on March 10, 2016, which is incorporated herein by reference in its entirety. Technical Field
[0004] The present disclosure relates to spectral imaging. Background Art
[0005] The optical spectrum of the light reflected from the object can be obtained by replacing the conventional imaging system of the front optical device of a lens camera or microscope with an imaging spectrometer. Utilizing this construction, the optical spectrum generated from a single line on the object is obtained by exposing the camera to the light reflected from the object. Only when a separate image of the recorded surface is taken thereafter, the position where the spectrum starts on the object is roughly known. If the object is attached to a mobile platform with controlled motion, or to a motorized microscope stage, a group of lines containing the spectrum can be obtained, so that individual spectra can be referenced to the physical characteristics of the object using a hyperspectral imaging (HSI) method. By utilizing this technology of a motion-controlled mobile platform of HSI, a conventional image of the spectrum of each point on the grid can be obtained, so this method is often used to map the spectrum to object features.
[0006] A "snapshot" HSI system can obtain spectra from all points on an image at a single instant. However, the system does not obtain a continuous spectrum with regularly spaced wavelengths, but rather a spectrum with a limited number (less than thirty) of unequally spaced wavelengths, and may require a lot of computation to reconstruct. Summary of the invention
[0007] An imaging system for synchronously recording an aerial image and a spectral image of a portion of the aerial image is generally described. In some examples, a beam splitter of the imaging system splits a beam obtained from an observation device into a first sub-beam and a second sub-beam, the first sub-beam being directed by the imaging system to a spatial camera, and the second sub-beam being directed by the imaging system to an entrance slit of an imaging spectrometer coupled to the spectral camera. Electronic equipment synchronously triggers the spatial camera and the spectral camera to synchronously record the aerial image and the spectral image, respectively.
[0008] The entrance slit of the imaging spectrometer defines the image area to be separated into spectra by the imaging spectrometer. Because the first sub-beam and the second sub-beam are separated from the common beam, the entrance slit of the imaging spectrometer defines the image area separated into spectra and is associated with the corresponding area of the spatial image. Therefore, the imaging system can synchronize the spectral recording of the optical spectrum of the points in the defined area of the image with the spatial image recording of the image including the defined area. Because the optical spectrum of the points in the defined area of the image is mapped to the corresponding point in the spatial image, the imaging system can directly assign the optical spectrum to the physical features contained in the conventional image by means of a one-to-one mapping between the spectrum and the position in the image. In other words, the imaging system and technology described herein can allow the user to produce a substantially accurate and well-documented assignment between the recording of the optical spectrum and the specific physical or structural features of the object being measured.
[0009] Image systems can provide advantages in a variety of fields such as materials science, biomedical research, and medical diagnostics where analysis is performed based on chemometric spectral data. In ophthalmology, applications can include retinal metabolic imaging, the role of pigmentation in retinal diseases (macular degeneration, retinitis pigmentosa), blood-borne diseases, and Alzheimer's disease screening, as well as screening for Parkinson's disease, Huntington's disease, and other amyloid-related neurological diseases. In wound healing, applications can include assessment of the healing of chronic wounds and pathogen detection. As another example, if the spectral recording is used for colon cancer detection, the imaging system can allow the spectral correlation of suspected cancerous tissue to be assigned to specific structural features of the tissue.
[0010] In one example, a spatial-spectral imaging device includes: a beam splitter configured to receive a light beam carrying an image of an object and split the light beam into a first sub-beam and a second sub-beam; an imaging spectrometer configured to receive the first sub-beam and disperse the wavelength range of the first sub-beam to form a spectral image including a plurality of spectra; a spatial image camera configured to receive the second sub-beam; and a spectral image camera configured to receive a spectral image from the imaging spectrometer, wherein the spectral image camera and the spatial image camera are configured to synchronously record the spectral image and the spatial image carried by the second sub-beam, respectively.
[0011] In another example, an imaging system includes: a retinal observation device configured to output a light beam carrying an image of an object; and a spatial-spectral imaging device, comprising: a beam splitter configured to receive a light beam carrying an image of an object and split the light beam into a first sub-beam and a second sub-beam; an imaging spectrometer configured to receive the first sub-beam and disperse the wavelength range of the first sub-beam to form a spectral image including a plurality of spectra; a spatial image camera configured to receive the second sub-beam; and a spectral image camera configured to receive a spectral image from the imaging spectrometer, wherein the spectral image camera and the spatial image camera are configured to synchronously record the spectral image and the spatial image carried by the second sub-beam, respectively.
[0012] In another example, a method includes triggering a spatial-spectral imaging device having a spectral image camera and a spatial image camera to trigger the spectral image camera and the spatial image camera to synchronously record a spectral image of an object and a spatial image of the object, respectively.
[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 is a block diagram illustrating an imaging system in accordance with techniques described in this disclosure.
[0015] Figure 2 is a recorded spatial image of the human fundus according to the techniques described herein.
[0016] Figure 3 is a recorded spectral image according to the disclosed technique, which depicts the Figure 2 The spatial image is the optical spectrum of each point where the lines intersect.
[0017] Figure 4 is a graph of the calculated absorption spectrum of capillary blood from the optic disc in the visible light spectrum in accordance with the techniques of the present disclosure.
[0018] Figure 5 is a diagram showing a conceptual diagram of a selected portion of the human fundus.
[0019] Figure 6 Included are aerial images recorded by operation of an imaging system according to the techniques described herein.
[0020] 7A to 7D Each depicts a pair of correlated recorded spatial and spectral images of a portion of a human fundus recorded by operation of an imaging system according to the techniques described in this disclosure.
[0021] Figures 8 to 9 Included are aerial images recorded by operation of an imaging system according to the techniques described herein.
[0022] FIG. 10A to FIG. 10D Each depicts a graph of the average absorption spectrum of capillary blood from a portion of a human fundus in the visible light spectrum calculated according to the techniques described herein.
[0023] FIG. 11A to FIG. 11D Each depicts a graph of the average absorption spectrum of capillary blood from a portion of a human fundus in the visible light spectrum calculated according to the techniques described herein.
[0024] Like reference symbols refer to like elements throughout the various drawings and text. DETAILED DESCRIPTION
[0025] Figure 1 1 is a block diagram illustrating an imaging system according to the techniques described in the present disclosure. In this example, the imaging system 10 includes a spatial-spectral imaging device 20 attached to a retinal viewing device 12. The retinal viewing device 12 may represent an ophthalmic fundus camera or other device for obtaining an image 50. In this example, the image 50 is an image of the fundus 49, but the image 50 may be an image of any object. The light source ( Figure 1 (not shown) illuminates the fundus 49 to produce an image 50 which is input to the ophthalmic lens 14 of the retinal viewing device 12. Figure 1 1, but the retinal viewing device 12 may include a mesh arm with attached light that, for ophthalmic applications, allows the operator to control the gaze direction of the eye being viewed. The fundus 49 may be that of a human or other living being and has independent movement according to the subject's muscle activity. In other words, the subject may perform normal eye movements, thereby forming movement of the fundus and corresponding different images 50 over time.
[0026] The ophthalmic lens 14 includes one or more lenses configured to modify the image 50 to produce the image 51. The retinal viewing device 12 includes an eyepiece (or "eyepiece") 16 to focus the image 51 for viewing by a user (not shown) of the retinal viewing device 12. The retinal viewing device 12 includes optical hardware configured to direct and output the image 51 along an imaging path via an output port 17. The output port 17 may represent an auxiliary camera port of the retinal viewing device 12.
[0027] According to the techniques described in the present disclosure, the spatial-spectral imaging device 20 is attached to the retinal viewing device 12 to receive the image 51 output via the output port 17. The spatial-spectral imaging device 20 includes an adapter 18 configured to attach the spatial-spectral imaging device 20 to the retinal viewing device 12. The optical lens 30 focuses the image 51 from the retinal viewing device 12 to the spatial image camera 30 and the entrance slit of the imaging spectrometer 36.
[0028] The spatial-spectral imaging device 20 includes an optical beam splitter 35 located above the optical lens 30, and the optical beam splitter is used to split the light beam conveying the image 51 into a transmitted light beam and a reflected light beam. The beam splitter 35 can represent a partial reflector, a beam splitter cube, a film, a diaphragm, or other devices for splitting the light beam conveying the image 51. The transmitted sub-beam transmitted by the optical beam splitter 35 transmits the image 51 as an image 54 to the spectral imaging path 22 received by the spectral imaging camera 40. The reflected sub-beam reflected by the optical beam splitter 35 reflects the image 51 as an image 52 to the spatial imaging path 24 received by the spatial image camera 30. As described in further detail below, the spatial imaging path 24 is generated by optical devices and other devices. As described in further detail below, the spectral imaging path 22 is generated by optical devices and other devices. In some example configurations of the spatial-spectral imaging apparatus 20 , the respective devices for the spatial imaging path 22 and the spectral imaging path 24 may be interchanged such that the spatial imaging path 24 receives the transmission image 54 from the beam splitter 35 and the spectral imaging path 24 receives the reflection image 52 from the beam splitter 35 .
[0029] The spatial imaging path 24 includes a relay optical lens 26, which is coupled to the beam splitter 35 via an optional spacer to operate on the image 52. The relay optical lens 26 adjusts the size of the spatial image so that the size of the spatial image is compatible with the size of the film or digital image sensor of the spatial image camera 30. The optional filter compartment 28 can be equipped with filters to filter one or more wavelengths of the image 52. The choice of filter may be application-dependent. For imaging the fundus 49, for example, the optional filter compartment 28 can be equipped with a red filter to enhance the contrast of the image 52 of the fundus 49 to be recorded by the spatial image camera 30. The spatial image camera 30 can represent a digital or film camera. The spatial image camera 30 can represent a color or monochrome camera. The spatial image camera 30 records the image 52 as one or more recorded spatial images. Figure 2 An example of a recorded aerial image is shown in .
[0030] The spectral imaging path 22 includes a relay optical lens 34 coupled to a beam splitter 35 via an optional spacer to operate on an image 54. The relay optical lens 34 adjusts the size of the spectral image so that the size of the spectral image is compatible with the size of the film or digital image sensor of the spectral image camera 40. The relay optical lens 34 can also set the image focus at the location of the entrance slit 36 of the spectrometer 36. The spectrometer bracket 33 is configured to attach the imaging spectrometer 36 to the relay optical lens 34. The camera bracket 38 is configured to attach the spectral image camera 40 to the imaging spectrometer 36. The spectral image camera 40 can represent a digital or film camera. The spectral image camera 40 can represent a monochrome camera.
[0031] The spectral imaging path 22 may or may not include a filter compartment, i.e., the image may pass unfiltered from the ophthalmic lens 14 to the spectrometer 36. Thus, in some configurations, the aerial image camera 30 may receive a filtered image of the image 50, while the spectral image camera 40 may receive an unfiltered image of the image 50.
[0032] An entrance slit of the imaging spectrometer 36 receives the image 54 from the beam splitter 35. The entrance slit may be located in the body of the imaging spectrometer 36 and defines the region of the image 54 that is delivered to and dispersed by the imaging spectrometer 36. In some examples, the entrance slit may have a width between 1 μm and 100 μm. The region of the image 54 delivered to the imaging spectrometer may substantially conform to the line shape of the entrance slit.
[0033] The imaging spectrometer 36 separates (or "disperses") the wavelengths included in the region of the image 54 into a continuous two-dimensional spectrum formed in the longitudinal dimension of the entrance slit, which has a wavelength axis substantially transverse to the longitudinal dimension of the entrance slit of the imaging spectrometer 36. In other words, the entrance slit disperses the wavelengths to form a spectrum whose wavelength axis is parallel to the transverse direction of the entrance slit. For a digital camera, this same spectral direction can be oriented with the columns (or rows) of pixels on the two-dimensional image sensor of the digital camera. The entrance slit continuously forms a spectrum at each point along its length, which can be oriented parallel to the rows (or columns) of pixels on the two-dimensional image sensor of the digital camera. The spatial-spectral image formed by this process can be recorded as a single image frame.
[0034] The continuous spectrum of the area of the image 54 forms a spectral image 57 that is output to the spectral image camera 40 , which records the spectral image 57 as one or more recorded spectral images and stores the recorded spectral images to a storage medium, such as a hard drive. Figure 3An example of a recorded spectral image is shown in . Because the digital camera image sensor has discrete detection elements, the recorded spectral image 57 represents multiple spectra with wavelength axes parallel to the lateral direction of the entrance slit. Post-processing of the recorded spectral image 57 can be used to combine two or more spectra of the recorded spectral image 57 for display or analysis.
[0035] In some examples, beam splitter 35 is configured to reflect about 30% of the light beam from output port 17 along imaging path 24 and transmit about 70% of the light beam along imaging path 22. In this way, a higher proportion of the light beam received at spectral image camera 40 relative to that received at aerial image camera 30 can compensate for the dilution of the wavelength spectrum along the pixels of spectral image 57 recorded by the spectral image camera relative to the pixels of aerial image 52 recorded by aerial image camera 30.
[0036] The imaging system 10 includes a trigger device 60 ("trigger 60") that is communicatively coupled to the spatial image camera 30 and the spectral image camera 40 via respective signal links 62 and 64. The signal links 62 and 64 may represent wired or wireless links for transmitting signals that, when received by the cameras 30 and 40, cause the cameras to take a picture. The trigger 60 may represent any electronic device that is configured to provide a message, such as a packet, an electrical signal, an optical signal, or other type of signal, to the trigger to cause the cameras 30 and 40 to take a picture. The common trigger 60 may be activated manually or automatically. For example, a user (e.g., a clinician or researcher) may manually press a button of the trigger 60, which initiates a respective signal to the cameras 30 and 40. As another example, a periodic timer of the trigger 60 may initiate a signal to the cameras 30 and 40. For fundus or other ophthalmic applications, a user of the imaging system 10 may direct the subject's gaze using an articulated optical system or by verbal instructions, and activate the trigger 60 when the subject's gaze is in the desired direction.
[0037] In some examples, including Figure 1 In the illustrated example, trigger 60 is also communicatively coupled to retinal viewing device 12 via signal link 65. Common trigger 60 can additionally signal retinal viewing device 12 by generating a flash of light to, for example, illuminate a subject (e.g., fundus 49). The signal sent via signal link 65 can be synchronized with the signal sent via signal links 62, 64, so that cameras 30 and 40 take a picture of the subject illuminated by retinal viewing device 12.
[0038] The spectral image camera 40 and the spatial image camera 30 are configured to record images (e.g., take pictures) in response to receiving a signal from the trigger 60. When triggered by the common trigger 60, the spectral image camera 40 and the spatial image camera 30 synchronously record a spectral image from the spectral image 57 and a spatial image from the image 52, respectively. The spectral image camera 40 and the spatial image camera 30 may record association data associated with the respective recorded images to enable subsequent association of the recorded images when representing the spatial image and the spectral image recorded simultaneously (i.e., synchronously). The association data may represent an image number for the image stored by each camera 30, 40 (e.g., an integer indicating the first, second, etc. picture taken for the session and in some cases indicating the file name of the recorded image), a timestamp, or other data indicating that a given pair of recorded spectral and spatial images are temporally related.
[0039] In this way, the imaging system 10 synchronously records the aerial image and the spectral image of a portion of the aerial image. As described above, the entrance slit of the imaging spectrometer defines the region of the image 54 transmitted to the imaging spectrometer 36, through which the light from the region is dispersed into a continuous set of spectra and recorded by the spectral image camera 40. Because the first sub-beam carrying the image 54 and the second sub-beam carrying the image 52 are separated from the common beam carrying the image 51, the entrance slit of the imaging spectrometer 36 defines the region of the given image divided into spectra and is associated with the corresponding region of the aerial image. Therefore, the imaging system 10 can achieve the synchronous recording of the spectral image of the optical spectrum of the point in the defined area of the image and the recording of the aerial image including the spectral image of the defined area. In some examples, the imaging system 10 generates a spectrum with uniform spatial wavelength, that is, up to hundreds of wavelengths in the visible and near infrared range. Because the optical spectrum of the point in the defined area of the image is mapped to the corresponding point in the aerial image, the imaging system 10 can achieve the direct assignment of the optical spectrum to the physical features contained in the conventional image by means of a one-to-one mapping between the spectrum and the position in the image. In other words, the imaging system 10 and techniques described herein may allow a user to produce a substantially accurate and well-documented assignment between a record of an optical spectrum and a specific physical or structural feature of an object being measured.
[0040] The imaging system 10 may in this way provide advantages over imaging systems in which an optical spectrum generated from a single line on an object is obtained by exposing a camera to light reflected from the object, and in which the location on the object where the spectrum begins is approximately determined based on different subsequent (i.e., non-synchronous) pictures of the recorded surface. For example, the imaging system 10 may be particularly suitable for applications in which the object serving as the image source is not precisely controlled by a user, for example, has independent motion as in the case of a human or other living being, or in which the object is sensitive to light. For example, the imaging system 10 may be particularly suitable for situations in which hyperspectral imaging of a motion-controlled mobile platform is not feasible. The imaging system 10 may achieve a co-assignment of high-resolution spectra and object features when it is not necessary to obtain an optical spectrum for every point on a grid, but it is still necessary to fully record the object features around the measurement line.
[0041] Figure 2 1 is a recorded aerial image 100 of the optic disc and retina surrounding the eye of a human subject. Line 102 depicts a region of the aerial image 100 that is input to an imaging spectrometer to produce a spectral image that is synchronously recorded by the imaging system 10, as described in the present disclosure. Line 102 may correspond in position to an entrance slit of an imaging spectrometer 36 of the imaging system 102. In some instances, line 102 may span the entire width of the aerial image 100. In some instances, line 102 may have a width of one or more pixels.
[0042] Figure 3 is a recorded spectral image 150 according to the techniques described herein, which depicts the Figure 2 The spectral image 150 depicts the optical spectrum at each point where the line 102 intersects the aerial image 100. The spectral image 150 depicts the wavelengths of light in the spectrum along the vertical image dimension, and a one-dimensional image of the object along the horizontal dimension, which corresponds to the central horizontal line of the aerial image 100. The position of the central horizontal line in the aerial image 100 is fixed and Figure 2 102 in the spectral image 150. As a further illustration of the relationship between the images, the brightly reflected optic disc in the aerial image 100 is associated with a bright band in the spectral image 150, i.e., the spectrum of the optic disc. Moreover, the blood vessels in the aerial image 100 correspond to the location of the dark vertical band in the spectral image 150. Each column of pixels in the spectral image 150 may represent a spectrum for the width of a single pixel on line 102 of the aerial image 100. An image analysis program may be used to obtain a reflected light spectrogram to measure light along a vertical line in the spectral image, which light is then the subject of various specific spectral analyses.
[0043] Aerial image 100 and spectral image 150 may be recorded using imaging system 10 according to the techniques described in this disclosure.
[0044] In order to identify the location of the aerial image 100 corresponding to the spectrum of the spectral image 150 so as to map the spectrum directly to the coordinates and physical features of the aerial image 100, the location of, for example, line 102 of the aerial image 100 recorded by the imaging system 10 can be determined. In some examples, the user can create a target with a discrete contrast agent. By precisely controlling the movement of the target and moving the target until the recorded spectral image indicates that the recorded spectrum is for the contrast agent, the user of the imaging system 10 can determine the location of the line 102 based on the simultaneously recorded aerial image.
[0045] Figure 4 is a graph of a calculated absorption spectrum 200 of capillary blood from the optic disc in the visible light spectrum according to the techniques of the present disclosure. The imaging system 10 described in the present disclosure can be used to synchronously record an aerial image and a spectral image of a subject. The aerial image and the spectral image can be used to detect ophthalmic conditions, retinal diseases, or Alzheimer's disease. An example description of using images to detect Alzheimer's disease can be found in U.S. Patent Publication 2014 / 0348750, published on November 27, 2014, which is incorporated herein by reference in its entirety. The calculated absorption spectrum is the spectrum produced by the capillary blood in the disc tissue, representing oxygenated hemoglobin. The absorption spectrum is obtained using vertical line profiles from the retinal image and a blank image of a neutral reflecting surface, which are used together to calculate light absorption. The same or similar image analysis procedures can be used to detect optical signals representing early Alzheimer's disease. There is also the effect of light scattered from the blood and other small molecules in the retina in this spectrum.
[0046] Figure 5 is a diagram illustrating a conceptual view of a selected portion of a human fundus. Diagram 300 illustrates an optic disc 302, a nerve fiber layer and a plurality of blood vessels 304, an upper retinal region (“upper retina”) 306, an lower retinal region (“lower retina”) 308, and a juxta-fovea. A user of imaging system 10 may focus ophthalmic lens 14 to configure imaging system 10 to synchronously capture one or more spatial and spectral images 50 of portions of a subject's fundus 49, including one or more spatial and spectral images 50 of any portion shown in diagram 300.
[0047] Figure 6 Aerial images recorded by operation of imaging system 10 according to the techniques described herein are included. Aerial image 400 is an image of an optic disc of an example of a subject's fundus 49. Aerial image 410 is an image of a nerve fiber layer of an example of a subject's fundus 49. Aerial image 420 is an image of a macula of an example of a subject's fundus 49. Aerial image 430 is an image of a retina of an example of a subject's fundus 49.
[0048] 7A to 7DEach depicts a pair of correlated recorded aerial and spectral images of a portion of a human fundus recorded by operation of imaging system 10 in accordance with the techniques described in this disclosure. For each pair of aerial-spectral images, the spectral image depicts the wavelengths of light of the spectrum along the vertical image dimension, and a one-dimensional image of the object along the horizontal dimension, the horizontal dimension corresponding to the central horizontal line of the aerial image. Fig. 7A Correlated spatial image 480A and spectral image 480B are depicted of the optic disc. Figure 7B Correlated spatial image 482A and spectral image 482B depicting the nerve fiber layer. Figure 7C Depicting the correlated spatial image 484A and spectral image 484B of the retina. Fig.7D Correlated spatial image 486A and spectral image 486B of the macula are depicted.
[0049] Figures 8 to 9 Aerial images recorded by operation of imaging system 10 according to the techniques described herein are included. Aerial image 500 is an image of a human retina that has not been diagnosed with Alzheimer's disease. Aerial image 510 is an image of a human retina that has been diagnosed with Alzheimer's disease.
[0050] FIG. 10A to FIG. 10D Each depicts a graph of an average absorption spectrum of capillary blood from a portion of a human fundus in the visible light spectrum calculated according to the techniques described herein. The imaging system 10 described in the present disclosure can be used to synchronously record an aerial image and a spectral image of a subject. The calculated absorption spectrum is the spectrum produced by the capillary blood in the disc tissue, representing oxygenated hemoglobin. The absorption spectrum is obtained using vertical line profiles from the retinal image and a blank image of a neutral reflecting surface, which together are used to calculate light absorption.
[0051] Each of the graphs 600A to 600D includes an average absorption spectrum for a group of control subjects (11 subjects in the example data set used to calculate the spectra), and an average absorption spectrum for a different group of Alzheimer's disease subjects, each subject being diagnosed with Alzheimer's disease (5 subjects in the example data set used to calculate the spectra). Each subject exhibits a different absorption spectrum, where the average absorption spectrum for a group is calculated as the average of the values of the corresponding absorption spectrum for each wavelength data point.
[0052] Diagram 600A depicts an average absorption spectrum 602A for the group of Alzheimer's disease subjects and an average absorption spectrum 604A for the group of control subjects, the spectra 602A, 604A being calculated using spectra from a spectral image of the optic disc, captured using an example of the imaging system 10 .
[0053] Diagram 600B depicts an average absorption spectrum 602B for the group of Alzheimer's disease subjects and an average absorption spectrum 604B for the group of control subjects, the spectra 602B, 604B being calculated using spectra from a spectral image of the macula, captured using an example of imaging system 10 .
[0054] Diagram 600C depicts an average absorption spectrum 602C for the group of Alzheimer's disease subjects and an average absorption spectrum 604C for the group of control subjects, the spectra 602C, 604C being calculated using spectra from a spectral image of the retinal nerve fiber layer (RNFL), captured using an example of imaging system 10 .
[0055] Diagram 600D depicts an average absorption spectrum 602D for the group of Alzheimer's disease subjects and an average absorption spectrum 604D for the group of control subjects, the spectra 602D, 604D being calculated using spectra from a spectral image of the retina, captured using an example of imaging system 10 .
[0056] FIG. 11A to FIG. 11D Each depicts a graph of an average absorption spectrum of capillary blood from a portion of a human fundus in the visible light spectrum calculated according to the techniques described herein. The imaging system 10 described in the present disclosure can be used to synchronously record an aerial image and a spectral image of a subject. The calculated absorption spectrum is the spectrum produced by the capillary blood in the disc tissue, representing oxygenated hemoglobin. The absorption spectrum is obtained using vertical line profiles from the retinal image and a blank image of a neutral reflecting surface, which together are used to calculate light absorption.
[0057] Each of the graphs 700A to 700D includes the average absorption spectrum of the group of control subjects and the average absorption spectrum of the group of Alzheimer's disease subjects from the data set of graphs 600A to 600D. Graphs 700A to 700D also include corresponding average absorption spectra 706A to 706D for additional data sets representing subjects with early Alzheimer's disease. Each subject presents a different absorption spectrum, where the average absorption spectrum of a group is calculated as the average of the values of the corresponding absorption spectra for each wavelength data point.
[0058] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. The various features described as modules, units, or components may be implemented together in an integrated logic circuit, or separately as discrete but interoperable logic devices or other hardware devices. In some cases, the various features of the electronic circuit may be implemented as one or more integrated circuit devices, such as an integrated circuit chip or chipset.
[0059] If implemented in hardware, the present disclosure may be directed to an apparatus such as a processor or an integrated circuit device, such as an integrated circuit chip or chipset. Alternatively or additionally, if implemented in software or firmware, the techniques may be implemented at least in part by a computer-readable data storage medium that includes instructions that, when executed, cause a processor to perform one or more of the above methods. For example, a computer-readable data storage medium may store the instructions for execution by a processor.
[0060] The computer-readable medium may form part of a computer program product, which may include the packet material. The computer-readable medium may include computer data storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, etc. In some examples, the article of manufacture may include one or more computer-readable storage media.
[0061] In some instances, computer-readable storage media may include non-transitory media. The term "non-transitory" may indicate that the storage medium is not implemented in a carrier wave or propagating signal. In some instances, non-transitory storage media may store data that may change over time (e.g., stored in RAM or a cache).
[0062] The code or instructions may be software and / or hardware executed by a processing circuit, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described in the present disclosure may be provided within a software module or a hardware module.
[0063] In addition to or as an alternative to the above, the following examples are described. Features described in any of the following examples may be used in any of the other examples described herein.
[0064] Example 1. A spatial-spectral imaging device, comprising: a beam splitter, which is configured to receive a light beam carrying an image of an object and split the light beam into a first sub-beam and a second sub-beam; an imaging spectrometer, which is configured to receive the first sub-beam and separate the wavelength range of the first sub-beam to form a spectral image including multiple spectra; a spatial image camera, which is configured to receive the second sub-beam; and a spectral image camera, which is configured to receive the spectral image from the imaging spectrometer, wherein the spectral image camera and the spatial image camera are configured to synchronously record the spectral image and the spatial image carried by the second sub-beam, respectively.
[0065] Example 2. The spatial-spectral imaging device of claim 1, wherein the beam splitter is configured to split approximately 70% of the light beam into the first partial beam and approximately 30% of the light beam into the second partial beam.
[0066] Example 3. The spatial-spectral imaging device as described in claim 1, further comprising: a trigger device, which is configured to send corresponding signals to the spatial image camera and the spectral image camera in response to a common trigger, wherein the spatial image camera and the spectral image camera are configured to synchronously record the spatial image and the spectral image carried by the second sub-beam, respectively, in response to receiving the corresponding signals.
[0067] Example 4. A spatial-spectral imaging device as described in claim 1, wherein the first partial beam carries a first image corresponding to the image, and the second partial beam carries a second image corresponding to the image, wherein the imaging spectrometer includes an entrance slit, which limits the area of the first image that is input to the imaging spectrometer and separated into the wavelength range to form the spectral image.
[0068] Example 5. The spatial-spectral imaging device of claim 4, wherein the region of the first image is correlated with a corresponding region of the second image.
[0069] Example 6. The spatial-spectral imaging device of claim 4, wherein each spectrum of the plurality of spectra maps to a point in the second image.
[0070] Example 7. A spatial-spectral imaging device as described in claim 4, wherein the first image includes: the wavelength range for each of the multiple spectra in the first dimension; a one-dimensional image of the object in the second dimension; wherein the horizontal lines of the second image correspond to the one-dimensional image of the object in the second dimension.
[0071] Example 8. The spatial-spectral imaging device of claim 1, wherein the wavelength ranges are evenly spaced.
[0072] Example 9. The spatial-spectral imaging device of claim 1, wherein the wavelength range includes more than 30 wavelengths.
[0073] Example 10. The spatial-spectral imaging device of claim 1, wherein the wavelength range includes more than 100 wavelengths.
[0074] Example 11. The spatial-spectral imaging device of claim 1, wherein the spatial image camera is configured to generate a recorded spatial image of the spatial image, and wherein the spectral image camera is configured to generate a recorded spectral image of the spectral image.
[0075] Example 12. The spatial-spectral imaging device of claim 1, further comprising: an adapter for attaching the spatial-spectral imaging device to a retinal observation device, the retinal observation device being configured to output the light beam carrying the image of the object.
[0076] Example 13. The spatial-spectral imaging device of claim 1, further comprising: a filter compartment for a filter, the filter compartment being located in an imaging path of the second partial light beam.
[0077] Example 14. A spatial-spectral imaging device as described in claim 1, wherein the spectral image camera is configured to store first associated data of the spectral image, and wherein the spatial image camera is configured to store second associated data of the spatial image, and the first associated data and the second associated data can be used to determine that the spectral image and the spatial image are recorded synchronously.
[0078] Example 15. An imaging system, comprising: a retinal observation device configured to output a light beam carrying an image of an object; and a spatial-spectral imaging device as described in any one of claims 1-14.
[0079] Example 16. The spatial-spectral imaging apparatus of claim 15, wherein the retinal viewing device comprises a fundus camera.
[0080] Example 17. A method, the method comprising: triggering the spatial-spectral imaging device of any one of claims 1-14 to trigger the spectral image camera and the spatial image camera to synchronously record images.
[0081] Example 18. The method of claim 17, wherein the image is of an object having inherent motion that is not directly under the control of a user of the spatio-spectral imaging device.
[0082] Example 19. The method of claim 17, further comprising associating a recorded aerial image recorded by the aerial image camera with a recorded spectral image synchronously recorded by the spectral image camera.
[0083] Example 20. The method of claim 19, further comprising mapping a spectrum of a plurality of spectra of the recorded spectral image to a point in the recorded aerial image based on the correlation between the recorded aerial image and the recorded spectral image.
[0084] Example 21. A method comprising using the spatial-spectral imaging device of any one of claims 1-14 to detect retinal disease or other diseases that present symptoms through the retina.
[0085] Example 22. A method comprising detecting one of wound healing, Alzheimer's disease, and skin aging effects using the spatial-spectral imaging device of any one of claims 1-14.
[0086] Example 23. A method comprising using the spatial-spectral imaging device of any one of claims 1-14 to perform any of the applications of the present disclosure.
[0087] Furthermore, any specific features set forth in any of the above examples may be combined into beneficial examples of the described technology. That is, any specific features are generally applicable to all examples of the invention. Various examples of the invention have been described.
Claims
1. A spatial-spectral imaging method, the method comprising: Splitting the light beam into a first sub-beam carrying a spectral image and a second sub-beam carrying a spatial image by a beam splitter of an imaging device; A recorded spectral image of the spectral image and a recorded spatial image of the spatial image are synchronously generated by a spectral image camera of the imaging device and by a spatial image camera of the imaging device, wherein the recorded spectral image includes a plurality of optical spectra in a defined area, and the optical spectra are mapped to corresponding points in the recorded spatial image so that each optical spectrum of the recorded spectral image can be assigned to a physical feature in the recorded spatial image.
2. The method of claim 1, wherein the wavelengths of each of the plurality of optical spectra are evenly spaced. The method of claim 1 , wherein the spectral image is a hyperspectral image.
4. The method according to any one of claims 1 to 3, wherein the spectral image and the spatial image are images of an object.
5. The method of claim 4, wherein the object is a human fundus of a living human subject.
6. The method of claim 4, further comprising: The recorded spectral image is processed to calculate the absorption spectrum.
7. The method of claim 6, further comprising: identifying locations of features of the object based on the recorded aerial image; as well as mapping said positions to one or more spectra of said optical spectrum of said recorded spectral image, Wherein processing the recorded spectral image to calculate the absorption spectrum comprises: calculating the absorption spectrum of one or more spectra of the optical spectrum of the recorded spectral image.
8. The method of claim 7, wherein the feature comprises one of an optic disc, a retinal area, and a juxtafovea.
9. The method according to any one of claims 1 to 3, wherein the spectral image and the aerial image are formed by a common light beam including the light beam.
10. The method according to claim 9, wherein the common light beam carries an image of the object, and The limited area is a limited area of the image of the object.
11. The method of any one of claims 1 to 3, wherein the aerial image and the spectral image are images of objects with inherent motion.
12. The method according to any one of claims 1 to 3, further comprising: associating the recorded aerial image with the recorded spectral image, and Based on the association between the recorded aerial image and the recorded spectral image, a spectrum among a plurality of spectra of the recorded spectral image is mapped to a point in the recorded aerial image.
13. A spatial-spectral imaging method, the method comprising: Splitting the light beam into a first sub-beam carrying a spectral image and a second sub-beam carrying a spatial image by a beam splitter of an imaging device; A recorded spectral image of the spectral image and a recorded spatial image of the spatial image are synchronously generated by a spectral image camera of the imaging device and by a spatial image camera of the imaging device, wherein the recorded spectral image includes a plurality of optical spectra along a first image dimension and a one-dimensional image along a second image dimension, and the second image dimension corresponds to a central horizontal line of the recorded spatial image, so that each optical spectrum of the recorded spectral image can be assigned to a physical feature of the recorded spatial image.
14. The method of claim 13, wherein the wavelengths of each of the plurality of optical spectra are evenly spaced. The method of claim 13 , wherein the spectral image is a hyperspectral image.
16. The method of claim 13, wherein the spectral image and the aerial image are formed by a common light beam including the light beam.
17. The method of claim 13, wherein the aerial image and the spectral image are images of objects having inherent motion.
18. The method of any one of claims 13 to 17, mapping a spectrum of a plurality of spectra of the recorded spectral image to a point in the recorded aerial image based on the association of the recorded aerial image with the recorded spectral image.
19. A retinal imaging device comprising: Spectral imaging camera; as well as Space imaging camera, The spectral image camera and the spatial image camera are configured to receive light carrying an image of an object and synchronously record the image of the object to respectively generate a recorded spectral image of the image of the object and a recorded spatial image of the image of the object, the recorded spectral image comprising a plurality of optical spectra in a defined area of the image of the object, the optical spectra being mapped to corresponding points in the recorded spatial image so that each optical spectrum of the recorded spectral image can be assigned to a physical feature in the recorded spatial image.
20. The retinal imaging device of claim 19, wherein the spectral imaging camera comprises a spectral sensor.
21. The retinal imaging device of claim 19, wherein the aerial image camera comprises a aerial sensor.
22. The retinal imaging device of claim 19, wherein the retinal imaging device is configured to receive light bearing the image of the object from a fundus camera.
23. The retinal imaging device of claim 22, wherein an adapter attaches the retinal imaging device to the fundus camera.
24. A retinal imaging device as claimed in any one of claims 19 to 23, wherein the recorded spectral image is a hyperspectral image.
25. The retinal imaging device of any one of claims 19 to 23, further comprising: an imaging spectrometer configured to receive the light beam and disperse the wavelength range of the light beam to form a spectral image including the plurality of optical spectra, The spectral image camera is configured to receive the spectral image and generate the recorded spectral image from the spectral image.
