Stereoscopic volumetric intraoperative OCT and visualization system with stereoscopic camera

Abstract

The visualization system (10) includes a housing assembly (20) having a head unit (18) configured to be at least partially directed toward a target region (16, 116). An optical coherence tomography (OCT) module (14, 114, 214) and a stereoscopic camera (12, 112, 212) are disposed within the housing assembly. A controller (C) communicates with the OCT module (14, 114, 214) and the stereoscopic camera (12, 112, 212). The controller is adapted to acquire left and right OCT data of the target region via the OCT module and to synchronously acquire left and right camera data of the target region via the stereoscopic camera. The controller is adapted to generate a volume rendering image including first and second OCT images based on the OCT data and first and second camera images based on the camera data. The first and second OCT images and the first and second camera images have matching parallax.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 400,276, filed August 23, 2022, which is incorporated herein by reference in its entirety.

[0002] introduction The present disclosure relates to a visualization system comprising a stereoscopic volumetric intraoperative OCT (optical coherence tomography) module and a stereoscopic camera. [Background technology]

[0003] Various imaging modalities are commonly used worldwide to image various parts of the human body. For example, in the field of ophthalmology, OCT-based techniques have proven advantageous in many different situations, such as visualizing the presence of retinal detachment, retinal schisis, macular disorders, intraocular foreign bodies, and vitreous traction. One challenge in viewing the 2D images produced by these modalities is that surgeons must develop a mental 3D model to visualize the anatomical structures. Another challenge is that many 2D representations require segmentation of the 3D data. Summary of the Invention [Means for solving the problem]

[0004] Disclosed herein is a visualization system including a housing assembly having a head unit configured to be at least partially directed toward a target region. An optical coherence tomography (OCT) module and a stereoscopic camera are disposed within the housing assembly. A controller is in communication with the OCT module and the stereoscopic camera, the controller having a processor and a tangible, non-transitory memory having instructions recorded thereon. The controller is adapted to acquire left and right OCT data of the target region via the OCT module and to synchronously acquire left and right camera data of the target region via the stereoscopic camera. The controller is adapted to generate a volume rendering image including first and second OCT images based on the left and right OCT data, respectively, and first and second camera images based on the left and right camera data, respectively. The first and second OCT images and the first and second camera images have matching parallax.

[0005] The volume-rendered images are aligned in viewpoint, orientation, and scaling, respectively. The target region may be an eye. The visualization system includes a display unit in communication with a controller. The controller may be adapted to display the volume-rendered images adjacent to one another on the display unit. The controller may be adapted to use a transparency-based channel to display the volume-rendered images in grayscale. The transparency-based channel has a maximum scale indicating full transparency and a minimum scale indicating full opacity. The controller may be adapted to use a variable gamma factor as a function of depth to display the volume-rendered images.

[0006] In some embodiments, the OCT module includes a single light source. In other embodiments, the OCT module includes at least two light sources. The OCT module may include a light source that outputs a spectrum with a series of frequency sweeps, each incorporating multiple pulses. The series of frequency sweeps are equally spaced apart. The OCT module may include a Fourier domain mode-locked laser.

[0007] In one embodiment, the controller is adapted to obtain the volume-rendered image via direct ray casting. In another embodiment, the controller is adapted to obtain each of the volume-rendered images by first projecting each voxel from the 3D dataset onto a 2D viewing plane to generate a respective projection. Second, a composite of the respective projections is created by convolving each of the projections with one another, and each projection is estimated based in part on Gaussian splats. The controller may incorporate a graphics processing unit using multiple instruction multiple data (MIMD) techniques.

[0008] Disclosed herein is a method for visualizing a target region, the method comprising: a housing assembly having a head unit configured to be at least partially directed toward the target region; an optical coherence tomography (OCT) module and a stereoscopic camera mounted within the housing assembly; a controller in communication with the OCT module and the stereoscopic camera, the controller having a processor and a tangible, non-transitory memory having instructions recorded thereon; the method includes acquiring left and right OCT data of the target region via the OCT module; and synchronously acquiring left and right camera data of the target region via the stereoscopic camera; the method includes generating a volume rendering image including first and second OCT images based on the left and right OCT data, respectively, and first and second camera images based on the left and right camera data, respectively; the first and second OCT images and the first and second camera images have matching parallax;

[0009] The above and other features and advantages of the present disclosure will become more readily apparent from the following detailed description of the best mode for carrying out the disclosure, when read in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic fragmentary perspective view of a visualization system having an optical coherence tomography (OCT) module, a stereoscopic camera, a display unit, and a controller. [Figure 2] FIG. 2 is a schematic diagram of a portion of the system of FIG. 1 according to one embodiment. [Figure 3] 2 is a schematic diagram of a portion of the system of FIG. 1 according to another embodiment. [Figure 4] 2 is a flowchart of an exemplary method executable by the controller of FIG. 1. [Figure 5] 2 is a schematic, exemplary graph of the intensity output of an exemplary OCT light source usable in the system of FIG. 1; [Figure 6] 2 is a schematic example of an exemplary graph of a gamma trace that can be used to display an image produced by the system of FIG. 1, with screen luminance shown on the vertical axis and pixel values ​​shown on the horizontal axis. [Figure 7] FIG. 2 is a schematic diagram of an exemplary display unit usable in the system of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0011] Exemplary embodiments of the present disclosure are shown by way of non-limiting example in the drawings and are described in more detail below. However, it should be understood that the novel aspects of the present disclosure are not limited to the particular forms shown in the above-listed drawings. Rather, the present disclosure encompasses modifications, equivalents, combinations, subcombinations, permutations, groupings, and alternatives that are included within the scope of the present disclosure, such as, for example, within the scope of the appended claims.

[0012] Referring to the drawings, wherein like reference numbers refer to like components, FIG. 1 schematically illustrates a visualization system 10 having a stereoscopic camera 12 and an optical coherence tomography module 14 (hereinafter referred to as "OCT module 14"). Visualization system 10 (hereinafter referred to as "system 10") is configured to image a target site 16. Stereoscopic camera 12 is configured to record first and second images of target site 16, which may be used to generate a live two-dimensional stereoscopic view of target site 16. System 10 may include an integrated digital surgical microscope system. It will be understood that system 10 may take many different forms and may include multiple and / or alternative components and equipment.

[0013] Referring to FIG. 1 , stereoscopic camera 12 may be at least partially disposed within head unit 18 of housing assembly 20, which is configured to be at least partially directed toward target site 16. Housing assembly 20 may be self-contained and movable between various locations. Target site 16 may be an anatomical location on a patient, a biological sample in a laboratory, a calibration slide / template, etc. In the example shown in FIGS. 2-3 , target site 16 is eye E.

[0014] Referring to FIG. 1 , at least one input device 22 (hereafter "at least one" omitted) is operably connected to the stereoscopic camera 12 (e.g., at the head unit 18) to allow a user to manually position the stereoscopic camera 12. The input device 22 may include respective controls for enabling or selecting particular features, such as adjusting focus, magnification, the amount / type of light projected onto the target region 16, and other features. It is understood that the number and form of the input devices 22 may vary; for example, the input device 22 may include a joystick, a wheel, a mouse, or a touchscreen device. In some embodiments, the input device 22 may be controlled via a remote control unit 23 (see FIG. 1 ).

[0015] In some embodiments, the system 10 may include a robotic arm 24 operably connected to the head unit 18 and configured to selectively move the head unit 18. For example, with reference to FIG. 2, the robotic arm 24 may be selectively operable to extend the field of view of the OCT module 14 along the X, Y, and Z directions. With reference to FIG. 1, the head unit 18 may be mechanically coupled to the robotic arm 24 via a coupling plate 26. The robotic arm 24 may include one or more joints, such as a first joint 30 and a second joint 32, configured to provide additional degrees of positioning and / or orientation of the head unit 18. With reference to FIG. 1, a respective joint motor (e.g., joint motor 31) and a respective joint sensor (e.g., joint sensor 33) may be coupled to each joint. The joint motor 31 is configured to rotate the first joint 30 about an axis, while the joint sensor 33 is configured to transmit the position of the first joint 30 (in 3D space).

[0016] 1, the system 10 includes a controller C having at least one processor P and at least one memory M (or non-transitory tangible computer-readable storage medium) having instructions recorded thereon for performing a method 300 of operating a stereoscopic volumetric mode 50 to generate a volume-rendered image, described below with respect to FIG. 4. The memory M may store a set of controller-executable instructions, and the processor P may execute the set of controller-executable instructions stored in the memory M. The OCT module 14 and the stereoscopic camera 12 may include an integrated processor in communication with the controller C.

[0017] In many two-dimensional representations of three-dimensional data, a challenge is integrating different image processing steps so that the resulting visualization accurately reflects various aspects of the anatomy, including depth information. Additionally, when a surgical instrument (e.g., tool 132, 232 in FIGS. 2 and 3) is represented as a surface along with the anatomy, portions of the surface may be occluded.

[0018] The stereoscopic volumetric mode 50 eliminates segmentation and minimizes errors associated with the complex anatomical structures found in ophthalmic surgery. While visualization methods are generally based on normal anatomy, surgery is often performed on abnormal anatomy. As described below, the controller C is adapted to acquire left and right OCT data of the target region 16 via the OCT module 14, and to synchronously acquire left and right camera data of the target region 16 via the stereoscopic camera 12. The system 10 generates volume-rendered stereoscopic intraoperative images that are parallax-matched, as opposed to en face images.

[0019] Referring to FIG. 1 , the controller C may be configured to process signals for broadcast on a display unit 35, which may include first and second displays 36 and 38. As described below with respect to FIG. 7 , the controller C may be adapted to display volume-rendered images adjacent to one another on the display unit. The volume-rendered OCT display is displayed side-by-side with, and not superimposed on, the digital stereoscopic surgical video display (from the camera 12). The display unit 35 may include any type of stereoscopic display available to those skilled in the art, such as a high-definition television, an ultra-high-definition television, a projector, one or more computer screens, a laptop computer, or a digital eyepiece (a close-view display similar to AR / VR goggles), and may include a touchscreen. Referring to FIG. 1 , the first display 36 may be connected to the cart 34 via a flexible mechanical arm 40 with one or more joints to enable flexible positioning. The flexible mechanical arm 40 may be configured to have a length sufficient to extend over the patient and provide the surgeon with a relatively close view during surgery.

[0020] Images of the target region 16 may be presented in a variety of forms, including, but not limited to, captured still images, real-time images, and / or digital video signals. As used herein, "real-time" generally refers to updating information at the same rate as the data is received. More specifically, "real-time" means that image data is acquired, processed, and transmitted at a sufficiently high data rate and sufficiently low latency so that when the data is displayed, the object moves smoothly without noticeable jitter or latency. Typically, this occurs when new images are acquired, processed, and transmitted at a rate of at least about 30 frames per second (fps) and displayed at about 60 fps, and when the composite processing of the video signal has a latency of about 1 / 30th of a second or less.

[0021] Referring to FIG. 2, a portion of the system 10 according to the first embodiment is shown. The apparatus 100 of FIG. 2 includes an exemplary stereoscopic camera 112 integrated with an OCT module 114 via a shared objective lens 102. Referring to FIG. 2, the stereoscopic camera 112 includes two beam splitters 108A, 108B, two sets of magnifying or focusing optics 110A, 110B, and two high-resolution two-dimensional cameras 118A, 118B for imaging a target region 116. The cameras 118A, 118B may be high-dynamic-range cameras with an optimized view achieved by compressing their dynamic range to the visible range. The apparatus 100 may include any suitable additional optical or mechanical components for manipulating the light beams and automating adjustments.

[0022] 2, the target site 116 is illuminated by a first beam from the OCT module 14 that at least partially overlaps with a second beam (originating from a light source within the stereoscopic camera 112) at the target site 116 (which is eye E). Various surgical tools 132 may be used by the surgeon, along with an illuminator 134 that provides illumination from within the fundus of eye E. The objective lens 102 may receive light through an ophthalmic lens 136 that rests on the cornea of ​​eye E.

[0023] In the embodiment shown in FIG. 2, the OCT module 114 includes an OCT scanner 120 (with a single light source L) for generating a sample beam 122. The sample beam 122 is directed to a round-trip optical system 126 and then to a set of partial mirrors 128A, 128B. The OCT module 114 receives a measurement beam 124 that is reflected back in response to photons of the sample beam 122 interacting with the target region 116. FIG. 2 shows left and right optical paths 104A, 104B configured so that the view generated by the OCT module 114 provides the same parallax angle as the camera view. The left and right optical paths 104A, 104B have equal total distances. To reduce speckle noise (due to angular diversity), a single light source L may be used to generate both left and right volumetric data.

[0024] The OCT scanner 120 may include an XY galvo scanner set, a resonant scanner set, a microelectromechanical system (MEMS) scanner, or other types of scanners. Galvo scanners, also known as galvanometer optical scanners, include motorized mirror mounts for laser beam steering or scanning applications. The OCT module 114 includes an OCT engine 125, which may be spectral-domain OCT, swept-source OCT, or time-domain OCT utilizing optical point scanning or point detection techniques. Referring to FIG. 2, the system 10 may further include a binocular surgical microscope with two eyepieces 130A and 130B connected to tube lens sets 132A and 132B, respectively. Acquired images may be viewable on a display unit 135.

[0025] Referring now to FIG. 3 , a portion of the system 10 is shown in accordance with a second embodiment. The apparatus 200 of FIG. 3 includes an exemplary stereoscopic camera 212 integrated with an OCT module 214 via a shared objective lens 202. Referring to FIG. 3 , the stereoscopic camera 212 includes two beam splitters 208A, 208B and two high-resolution two-dimensional cameras 218A, 218B for imaging a target region 216. In some embodiments, the high-resolution two-dimensional cameras 218A, 218B may replace the eyepieces and be mountable to a surgical microscope. For simplicity, beam conditioning components are not shown. It is understood that the apparatus 200 may include any suitable additional optical or mechanical components.

[0026] In the embodiment shown in FIG. 3, the OCT module 214 includes an OCT scanner 220 with two light sources L1 and L2 that generate sample beams 222A and 222B, respectively. The OCT scanner 220 may include an XY galvo scanner set, a resonant scanner set, a microelectromechanical system (MEMS) scanner, or other types of scanners. The OCT module 214 receives measurement beams 224A and 224B that are reflected back in response to photons of the sample beams 222A and 222B interacting with the target region 216. FIG. 3 shows left and right optical paths 204A and 204B configured so that the view generated by the OCT module 214 provides the same parallax angle as the camera view. The left and right optical paths 204A and 204B have equal total distances.

[0027] 3 , sample beams 222A, 222B are directed onto sets of partial mirrors 228A, 228B, respectively. OCT module 214 includes an OCT engine 225, which may be spectral-domain OCT, swept-source OCT, or time-domain OCT utilizing light point scanning or point detection techniques. Various surgical tools 232 may be used by the surgeon, along with an illuminator 234 that provides illumination from within the fundus of eye E. Objective lens 202 may receive light through an ophthalmic lens 236 that rests on the cornea of ​​eye E. Acquired images may be viewable on a display unit 235.

[0028] Referring now to Figure 4, a flowchart of an exemplary method 300 for operating the stereo volumetric mode 50 of Figure 1 is shown. The method 300 may be embodied as computer-readable code or instructions stored in and partially executable by the controller C of Figure 1. The method 300 need not be applied in the particular order described herein, but may be performed dynamically. Furthermore, it should be understood that some steps may be omitted. The method 300 may be performed periodically or at predetermined time intervals.

[0029] According to block 302 of FIG. 4, the controller C is programmed to send a command to initiate the combined stereo volumetric mode 50. Proceeding to block 304 of FIG. 4, the controller C is programmed to acquire raw OCT data of the scanned region, including receiving left and right OCT scan data, for example, via the left optical path 104A, 204A and the right optical path 104B, 204B, respectively. The scanned OCT data set may be a 3D volume or a 2D B-frame, including sequential A-scans (depth scans) scanned according to various scan patterns. The raw OCT data is post-processed into a depth-resolved 3D volume through an OCT reconstruction pipeline, including background subtraction, spectral windowing, dispersion compensation, fast Fourier transform, and logarithmic compression. While several processing techniques have been described above, it will be understood that other techniques may be used.

[0030] 5 is a schematic, illustrative graph of the spectrum of an exemplary OCT light source (e.g., the single light source L of FIG. 2 and the light sources L1 and L2 of FIG. 3) that can be used in system 10, with frequency shown on the horizontal axis 402 and intensity or amplitude shown on the vertical axis 404. The spectrum has a series of equally spaced frequency sweeps 406. Each sweep incorporates multiple pulses 408 that provide an ultrafast sampling rate.

[0031] An OCT light source (e.g., the single light source L in FIG. 2 and light sources L1 and L2 in FIG. 3) can include a Fourier domain mode-locked laser. Mode-locking allows the laser to generate pulses of light of extremely short duration by creating a fixed phase relationship between the longitudinal modes of the resonant cavity within the laser. Constructive interference between these modes causes the laser light to be generated as a pulse train. The pulse duration can be on the order of picoseconds (10 -12 seconds) or femtoseconds (10 -15The coherence length may be on the order of 1000 s (seconds). In one example, the coherence length of an exemplary OCT light source is 30-50 mm. In another example, the coherence length is at least 50 mm. To manage artifacts caused by increased coherence length, a circular scope may be used.

[0032] Proceeding to block 306 of FIG. 5, in synchronization with the scan, controller C is programmed to acquire left and right camera data, for example, via left optical paths 104A, 204A and right optical paths 104B, 204B, respectively.

[0033] Proceeding to block 308 of FIG. 4, the controller C is adapted to generate a volume-rendered image including first and second OCT images 602 (see FIG. 7) based on the left OCT data and right OCT data, respectively, and first and second camera images 604 (see FIG. 7) based on the left camera data and right camera data, respectively. The first and second OCT images 602 (see FIG. 7) and the first and second camera images 604 (see FIG. 7) have matching parallax. Parallax is the displacement or difference in the apparent position of an object viewed along two different lines of sight, measured by the angle or half-angle of tilt between these two lines. The volume-rendered images are aligned in perspective, orientation, and scaling, respectively.

[0034] Volume rendering is understood to be a set of techniques used to display a two-dimensional projection of a three-dimensional discretely sampled data set. Direct volume rendering involves mapping volume data directly to optical properties, while indirect volume rendering maps portions of volume data to points, lines, and surfaces. The mapping is defined by a transfer function.

[0035] In one embodiment, the controller C is adapted to obtain a volume-rendered image via ray casting, where for each pixel in the two-dimensional projection, a ray is traced back into each voxel. A voxel is an individual volume element that corresponds to a location in three-dimensional data space and has one or more associated data values. The controller C is adapted to determine a grayscale value by considering the transparency-weighted voxels involved. The controller C may apply a ray tracing technique that uses Snell's law to trace the propagation of light through the eye E via reflection and refraction.

[0036] In another embodiment, the controller C is adapted to obtain a volume-rendered image by first projecting voxels from the 3D data set onto a 2D viewing plane to generate respective projections. Second, a composite is created by superimposing the respective projections on one another. The projections are estimated based in part on Gaussian splats. Alternatively, the system 10 may use texture-based volume rendering, in which voxels are loaded into texture mapping hardware.

[0037] According to block 310 of FIG. 4, the controller is adapted to display the stereoscopic OCT and camera images adjacently. FIG. 7 is a schematic diagram of an exemplary display unit 635 usable in the system. Referring now to FIG. 7, the volume-rendered first and second OCT images 602 and the volume-rendered first and second camera images 604 may be displayed adjacent to one another on the display unit 635. The volume-rendered first and second OCT images 602 and the volume-rendered first and second camera images 604 are presented to the surgeon's visual cortex as a stereoscopic pair of volumes. The display unit 635 incorporates a stereoscopic display system with a two-dimensional display having separate images for the left eye (left view 606) and the right eye (right view 608).

[0038] Referring to FIG. 7 , the user wears special glasses (e.g., optical viewer 610) that work in conjunction with the display unit 635 to present a left view 606 to the user's left eye and a right view 608 to the user's right eye. The optical viewer 610 may utilize lenses, prisms, or mirrors to simultaneously but independently deliver separate stereoscopic images to each eye, allowing the brain to fuse the pairs and recreate a three-dimensional image. Additionally, the optical viewer 610 may include a field mask to prevent cross-viewing between the eyes. The surgeon may intuitively slightly rotate the nasal / temporal (left / right) and superior / inferior (upper / lower) eyes to resolve ambiguity issues and projection artifacts and improve spatial understanding. In other words, eye rotation using left and right pairs of surgical instruments is used to resolve ambiguity, projection artifacts, and build a 3D mental model that correlates with the digital video 3D appearance without manipulating the user interface orientation. System 10 involves cognitive or voluntary selection of a depth subset of the volume (visual horopter or Panum's fusional region) by the user. If the inconsistent object is within Panum's fusional region, the images are fused so that the user sees a single image of the object. If the object is outside Panum's fusional region, the user sees two images of the object. In one example, display unit 635 is a 35-55 inch 3D 4K organic light-emitting diode (OLED) surgical display viewed using passive circularly polarized 3D glasses.

[0039] The controller C is adapted to use a transparency-based channel to display the volume-rendered image in grayscale. The transparency-based channel has a maximum scale (e.g., a scale having a value of 1.0) that indicates nearly complete transparency and a minimum scale (e.g., a scale having a value of 0.0) that indicates nearly complete opacity. In one embodiment, the system 10 uses only the transparency-based channel in grayscale, which does not include any red, green, or blue components.

[0040] The controller C may be adapted to use a variable gamma coefficient as a function of depth to display the volume-rendered image. In other words, objects at different depths are depicted in the 2D representation using different gammas. Figure 6 is a schematic example graph of a gamma trace usable by the system 10, showing relative brightness (e.g., running from 0 to 1) on the vertical axis 504 and pixel value (e.g., running from 0 to 256) on the horizontal axis 502. Gamma traces 506, 508, 510, 512, and 514 represent gamma values ​​of 1, 1.5, 1.8, 2.2, and 2.5, respectively. Each gamma trace is used for a different depth. In one example, lower gamma values ​​are used for anatomical regions where higher visibility or higher brightness is desired. This allows the semi-transparent or nearly transparent cornea and crystalline lens (or intraocular lens) to be displayed in front of the vitreous and retina.

[0041] In some embodiments, the controller C may incorporate a graphics processing unit using multiple instruction multiple data (MIMD) technology, which may be defined as an execution architecture that simultaneously processes two or more independent sets of instructions on two or more data sets. Here, the controller C includes multiple processors that function asynchronously and independently. The controller C may use a graphics processing unit (GPU) that supports multi-view rendering, such as rendering up to four views in a single pass. Rendering may be performed using a graphics processing unit (GPU) with volumetric rendering firmware, such as, for example, NVIDA's Turing Engine or a field programmable gate array (FPGA).

[0042] The controller C of Figure 1 may contain or otherwise access information downloaded from remote light sources and / or executable programs. Referring to Figure 1, the controller C may be configured to communicate with a remote server 60 and / or a cloud unit 62 via a network 64. The remote server 60 may be a private or public information source maintained by an organization such as, for example, a research institute, a company, a university, and / or a hospital. The cloud unit 62 may include one or more servers hosted on the Internet for data storage, management, and processing.

[0043] Network 64 may be a serial communications bus in the form of a local area network. Local area networks may include, but are not limited to, Controller Area Network (CAN), Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, Bluetooth, Wi-Fi, and other data types. Network 64 may be a wireless local area network (LAN) that links multiple devices in a wirelessly distributed manner, a wireless metropolitan area network (MAN) that connects multiple wireless LANs, or a wireless wide area network (WAN) that covers a large geographic area such as a nearby city or town. Other types of connections may also be used.

[0044] In summary, system 10 enables the observation of the human visual cortex in volume-rendered stereo pairs, thereby providing accurate 3D visualization during surgery. System 10 improves signal-to-noise ratio. The views of stereoscopic camera 12 are aligned to the views of OCT module 14 in rotation, translation, scaling, and perspective, respectively. System 10 addresses complex anatomical segmentation errors, flattening, and projection artifacts. Volume-rendered images incorporate non-flattened and non-segmented stereo views. In other words, system 10 is free of segmentation-related errors and free of flattening-related errors inherent in en face viewing modes.

[0045] The controller C in FIG. 1 may be an integral part of other controllers integrated into the OCT module 14 and the stereoscopic camera 12, or may be a separate module operatively connected thereto. The controller C comprises a computer-readable medium (also referred to as a processor-readable medium), including non-transitory (e.g., tangible) media involved in providing data (e.g., instructions) that can be read by a computer (e.g., by a computer processor). Such media may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media may include, for example, optical or magnetic disks and other persistent memory. Volatile media may include, for example, dynamic random access memory (DRAM), which may constitute main memory. Such instructions may be transmitted over one or more transmission media, including coaxial cables, copper wire, and fiber optics, including the wires that comprise a system bus coupled to the computer's processor. Some forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, other magnetic media, CD-ROM, DVD, other optical media, RAM, PROM, EPROM, Flash EEPROM, other memory chips or cartridges, or other medium from which a computer can read.

[0046] The lookup tables, databases, data repositories, or other data stores described herein may include various types of mechanisms for storing, accessing, and retrieving various types of data, including hierarchical databases, sets of files in a file-rechargeable energy storage system, proprietary application databases, relational database management systems (RDBMS), etc. Each such data store may be contained within a computing device that uses a computer operating system, such as one of the computer operating systems described above, and may be accessed over a network in one or more of a variety of ways. A file system may be accessible from a computer operating system and may include files stored in various formats. An RDBMS may use a structured query language (SQL) in addition to a language for creating, saving, editing, and executing stored procedures, such as the PL / SQL language described above.

[0047] The illustrated flowcharts illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, and includes one or more executable instructions for implementing the specified logical function(s). It should also be noted that each block of the block diagrams and / or flowchart examples, and combinations of blocks in the block diagrams and / or flowchart examples, may be implemented by a special-purpose hardware-based system or a combination of special-purpose hardware and computer instructions that performs the specified functions or acts. These computer program instructions may also be stored on a computer-readable medium. The computer program instructions, including instructions for implementing the functions / acts specified in the flowchart and / or block diagram blocks, may direct a controller or other programmable data processing apparatus to function in a particular manner to produce a product.

[0048] Numerical values ​​of parameters (e.g., amounts or conditions) in this specification, including the appended claims, are understood to be modified by "about" in each and every instance, regardless of whether the word "about" actually precedes the numerical value. "About" indicates that the stated numerical value allows for some slight imprecision (close to, roughly or reasonably close to, approximately the exact value). Where the imprecision provided by "about" is not understood in this ordinary sense in the art, "about," as used herein, will at least account for the variation that can result from ordinary methods of measuring and using such parameters. In addition, the disclosure of ranges includes the disclosure of each value and sub-ranges within the entire range. Each value within a range and the endpoints of the range are disclosed herein as separate embodiments.

[0049] While the detailed description and drawings or figures support and explain the present disclosure, the scope of the present disclosure is defined solely by the claims. While the best mode and some other embodiments for carrying out the claimed disclosure have been described in detail, various alternative designs and embodiments exist for carrying out the disclosure defined in the appended claims. Furthermore, the features of the embodiments shown in the drawings or the various embodiments described herein should not necessarily be understood as independent embodiments. Rather, each of the characteristics described in one example embodiment can be combined with one or more other desirable characteristics from other embodiments, resulting in other embodiments not described in language or by reference to the drawings. Accordingly, such other embodiments are encompassed within the scope of the appended claims.

Claims

1. 1. A visualization system comprising: a housing assembly having a head unit configured to be at least partially directed toward a target site; an optical coherence tomography (OCT) module and a stereoscopic camera disposed within the housing assembly; a controller in communication with the OCT module and the stereoscopic camera, the controller having a processor and a tangible, non-transitory memory having instructions recorded thereon; the controller is adapted to acquire left OCT data and right OCT data of the target area via the OCT module, and to synchronously acquire left camera data and right camera data of the target area via the stereoscopic camera; the controller is adapted to generate a volume rendering image including first and second OCT images based on the left OCT data and the right OCT data, respectively, and first and second camera images based on the left camera data and the right camera data, respectively; A visualization system wherein the first and second OCT images and the first and second camera images have matching parallax.

2. The visualization system of claim 1 , wherein the target site is an eye.

3. The visualization system of claim 1 , wherein the volume-rendered images are aligned in perspective and scaling.

4. a display unit in communication with the controller; The visualization system of claim 1 , wherein the controller is adapted to display the volume-rendered images adjacent to one another on the display unit.

5. the controller uses a transparency-based channel to display the volume-rendered image from the OCT module in grayscale; The visualization system of claim 1 , wherein the transparency-based channel has a maximum scale indicating nearly complete transparency and a minimum scale indicating nearly complete opacity.

6. The visualization system of claim 5 , wherein the controller is adapted to use a variable gamma factor as a function of depth to display the volume-rendered image.

7. 7. The visualization system of claim 6, wherein the target site is an eye, and the variable gamma factor is selected such that in the volume-rendered image from the OCT module, the cornea and / or lens of the eye is substantially transparent and the retina of the eye is substantially opaque.

8. The visualization system of claim 1 , wherein the OCT module includes a single light source.

9. The visualization system of claim 1 , wherein the OCT module includes at least two light sources.

10. The visualization system of claim 1 , wherein the OCT module includes a light source that outputs a spectrum with a series of frequency sweeps, each incorporating multiple pulses.

11. The visualization system of claim 10 , wherein the series of frequency sweeps are equally spaced apart.

12. The visualization system of claim 10 , wherein the OCT module includes a mode-locked laser.

13. The visualization system of claim 10 , wherein the OCT module includes a laser having a coherence length of approximately 30 mm to 50 mm.

14. The visualization system of claim 1 , wherein the controller incorporates a graphics processing unit that uses multiple instruction multiple data (MIMD) techniques.

15. The visualization system of claim 1 , wherein the controller is adapted to obtain the volume-rendered image based at least in part on ray casting.

16. The controller: projecting each voxel from the 3D data set onto a 2D viewing plane to generate a respective projection; 2. The visualization system of claim 1, adapted to obtain each of the volume rendering images by: creating a composite of the respective projections superimposed on one another, the respective projections being estimated in part based on Gaussian splats.

17. 1. A visualization system comprising: a housing assembly having a head unit configured to be at least partially directed toward a target site; an optical coherence tomography (OCT) module and a stereoscopic camera disposed within the housing assembly; a controller in communication with the OCT module and the stereoscopic camera, the controller having a processor and a tangible, non-transitory memory having instructions stored thereon; a display unit in communication with the controller; the controller is adapted to acquire left OCT data and right OCT data of the target area via the OCT module, and to synchronously acquire left camera data and right camera data of the target area via the stereoscopic camera; the controller is adapted to generate a volume rendering image including first and second OCT images based on the left OCT data and the right OCT data, respectively, and first and second camera images based on the left camera data and the right camera data, respectively; the first and second OCT images and the first and second camera images have matching parallax, and the volume rendering images are aligned in perspective and scaling, respectively; The visualization system, wherein the controller is adapted to display the volume-rendered images adjacent to one another on the display unit.

18. the controller uses a transparency-based channel to display the volume-rendered image in grayscale; 20. The visualization system of claim 17, wherein the transparency-based channel has a maximum scale indicating nearly complete transparency and a minimum scale indicating nearly complete opacity.

19. 20. The visualization system of claim 17, wherein the controller is adapted to use a variable gamma factor as a function of depth to display the volume-rendered image.

Images