Surgical 3D Imaging

Abstract

In some embodiments, the imager is coupled to a sensor that measures one or more of the position or orientation of the imager's measurement beam or imaging beam while the imager is acquiring image data. A processor is configured to receive the image data and, in response to the image data and sensor data, construct an image such as a 3D image. In some embodiments, the imager is configured to emit a beam of imaging energy, and the motion of the imager is used to construct an image such as a 3D image. In some embodiments, motion occurring during image data acquisition allows for the construction of a 3D image without a scanner, thereby reducing the complexity of the imaging device. Alternatively, or in combination, the imager may include a scanner or a beamformer, and motion data can be used to improve image quality.

Description

Technical Field

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 597,240, filed Nov. 8, 2023, and U.S. Provisional Patent Application No. 63 / 485,808, filed Feb. 17, 2023, under 35 U.S.C. § 119(e), the disclosures of which are hereby incorporated by reference in their entireties.

Background Art

[0002] Previous approaches to imaging tissue using imaging such as optical coherence tomography (OCT) may not be ideal in at least some respects. Research related to the present disclosure suggests that at least some of the previous approaches to imaging objects such as tissue may be somewhat more complex than ideal. Imaging techniques such as OCT, ultrasound, and photoacoustic imaging are applied in many fields, such as imaging tissue for diagnosis and surgery, but at least some of the previous approaches are not as suitable as would be ideal for integration with, for example, surgical systems. Some conventional imaging systems can generate images such as B-scans and 3D images such as tomographic images using a series of A-scans. However, due to unintentional movement of the imaging beam, in at least some cases, it can be more difficult than ideal to construct an image from data from imaging components where the unintentional movement is inherent. Also, at least some of the conventional imaging systems rely on scanning devices to move the imaging beam, which in at least some cases can be more complex than ideal. In imaging systems where data is acquired using a handheld probe, for example, when a surgeon holds or operates a surgical endoscope, a potential source of movement is body tremors.

[0003] Another example is motion, such as resonant motion, associated with mechanical devices, including surgical robots. Furthermore, research related to this disclosure suggests that human tremors from a robotic arm controlled by a human operator may cause at least some motion of the robotic arm related to the operator's tremor. Research related to this disclosure suggests that at least some of the previous methods do not adequately address motion, including motion associated with human tremors and mechanical resonance.

[0004] Examples of surgeries that can be treated with improved imaging include ophthalmic surgeries, such as glaucoma surgery. While some procedures are successful, previous methods for treating glaucoma were not ideal in at least some respects. One method for treating glaucoma is minimally invasive glaucoma surgery ("MIGS"). In lumen-based MIGS, small openings are created through the trabecular meshwork to allow fluid to drain into Schlemm's canal. These openings can be created in many ways, for example, using implants or lasers such as femtosecond lasers. One method used was excimer laser trabeculostomy ("ELT"). This procedure uses an ultraviolet laser, such as an excimer laser, to excise one or more openings to Schlemm's canal through the trabecular meshwork. Another method involved placing an implant that extended into Schlemm's canal through the trabecular meshwork.

[0005] One potentially challenging aspect of lumen-based MIGS procedures is the alignment of surgical instruments with Schlemm's canal, which is approximately 200 micrometers ("μm") to 400 μm in height. In some cases, Schlemm's canal may not be easily observable, in which case the surgeon must attempt to estimate its location, which can be difficult and, in some cases, may fall short of ideal accuracy. In some implant procedures, an inaccurate assessment of Schlemm's canal location can lead to several undesirable situations, such as the implant not being properly positioned within the lumen, trabecular meshwork rupture, and in some cases, the implant mispositioning may lead to, for example, it becoming dislodged later.

[0006] While OCT, ultrasound, and photoacoustic imaging have been proposed as techniques for imaging Schlemm's canal, research related to this disclosure suggests that previous techniques for imaging Schlemm's canal may not be ideal in at least some cases. For example, imaging systems may be somewhat more complex than ideal, and conventional systems may not adequately address motion, such as human tremor or in vivo movement of the tissue itself. In at least some cases, tissue movement may be related to the inherent pulsation of the organ's vascular and capillary network due to the periodic filling and emptying of the heart, which can result in tissue movement, and previous techniques may not be ideal for addressing this pulsating tissue movement.

[0007] In light of the above, improvements to OCT methods and apparatus, such as facilitating the creation of openings in Schlemm's canal and the placement of implants, would be beneficial. [Overview of the Initiative] [Means for solving the problem]

[0008] In some embodiments, the imager is configured to generate a beam of imaging energy, and the imager is coupled to a sensor that measures one or more positions or orientations of the imaging beam while the imager acquires image data. The sensor may include one or more of position sensors, orientation sensors, accelerometers, or image sensors. A processor is configured to receive the image data and construct an image, such as a 3D image, in response to the acquired image data and acquired sensor data. In some embodiments, the imager is configured to emit a beam of imaging energy, and motion data of the imaging beam is used to construct an image, such as a 3D image. In some embodiments, this motion occurring during image data acquisition allows for the construction of a 3D image without the use of a scanner, thereby reducing the complexity of the imaging device. Alternatively, or in combination, the imager may include a scanner or a beamformer, and the motion data can be used to improve the quality of the 3D image.

[0009] In some embodiments, simultaneous position and orientation data of an imaging device are used to determine the position and orientation of an imaging beam while image data is being acquired, and the position and orientation data of the imaging beam are used to construct a 3D image of the tissue. In some embodiments, the imaging beam is configured to acquire A-scan data, and the position and orientation of the imaging beam are acquired for each of a plurality of A-scans, and the position and orientation data are combined with the plurality of A-scans to generate a 3D image.

[0010] In some embodiments, the distal end of the imaging channel is positioned proximal to the distal end of the treatment channel so that the distal end of the treatment channel can be observed in an imaging channel such as a 3D imaging channel. In some embodiments, the tissue and one or more treatment fibers or implants can be observed in a 3D image generated by the imaging channel to observe the relationship between the tip of the treatment channel or implant and the tissue, thereby facilitating the positioning of the procedure. In some embodiments, the imaging channel includes an optical fiber with a distal end located proximal to the distal end of the treatment channel, such as a treatment optical fiber or implant, to observe the relationship between the tissue and the distal end of the optical fiber or implant.

[0011] All patents, applications, and publications referenced and identified herein are incorporated by reference in their entirety, and are deemed to be fully incorporated by reference if they are referenced in any other part of this application.

[0012] The features, advantages, and principles of this disclosure can be better understood by referring to the following detailed description and accompanying drawings illustrating illustrative embodiments. [Brief explanation of the drawing]

[0013] [Figure 1] These are schematic cross-sectional views of the eye showing its anatomical structure, based on several examples. [Figure 2] These are fluoroscopic partial views of anatomical structures adjacent to the anterior chamber of the eye, based on several examples. [Figure 3] This is a schematic cross-sectional view of an eye showing, in several embodiments, an optical fiber probe and an imaging probe traversing the anterior chamber from the limbal puncture site toward the trabecular meshwork within the anterior chamber of the eye. [Figure 4A] This is a partial schematic diagram of the anatomical structure of the anterior chamber angle of the eye, showing Schlemm's canal, scleral promontory, and Schwalbe line, based on several examples. [Figure 4B]These are partial diagrams of the anatomical structure of the eye, as shown in several examples, and are representations obtained using an endoscope or other imaging system from an internal eye perspective. [Figure 5A] This figure shows components of an OCT imaging system and treatment probe for imaging and treating ocular tissue, according to several embodiments. [Figure 5B] This figure shows an ultrasound imaging system and treatment probe for imaging and treating eye tissue, according to several embodiments. [Figure 5C] This figure shows a photoacoustic imaging system and treatment probe for imaging and treating eye tissue, according to several embodiments. [Figure 6A] This figure shows a scanning OCT imaging system and treatment probe for imaging and treating ocular tissue, according to several embodiments. [Figure 6B] This figure shows an ultrasound imaging system and treatment probe for imaging and treating eye tissue, according to several embodiments. [Figure 6C] This figure shows a scanning photoacoustic imaging system and treatment probe for imaging and treating eye tissue, according to several embodiments. [Figure 7] This is a diagram of an eye surgery apparatus, showing several embodiments. [Figure 8] The diagram shows, in several embodiments, an optical surgical microscope view and an extended image including one or more 2D images, 3D images, or models with treatment site markers superimposed. [Figure 9] This figure shows the movement of the probe and the corresponding position of the measurement beam or imaging beam in several embodiments. [Figure 10] This figure shows a probe including a direction sensor that rotates around an opening, according to several embodiments. [Figure 11] This figure shows a probe including an endoscope configured to determine the location of a 3D imager's measurement beam or imaging beam, according to several embodiments. [Figure 12] A diagram showing a probe including a treatment channel, a 3D imager, and an endoscope for determining the location of a measurement beam or an imaging beam of a 3D imager according to some embodiments. [Figure 13A] A diagram showing a probe including a treatment channel and 2D and 3D imaging components including overlapping optical paths according to some embodiments. [Figure 13B] A diagram showing a probe including a treatment channel and a 3D imaging optical fiber and a plurality of 2D imaging optical fibers according to some embodiments. [Figure 14A] A diagram showing an endoscope image including one or more tissue structures and the location of a measurement beam or an imaging beam at a first time according to some embodiments. [Figure 14B] A diagram showing an endoscope image including one or more tissue structures and the location of a measurement beam or an imaging beam at a second time according to some embodiments. [Figure 14C] A diagram showing a displacement vector of an image and a corresponding measurement location according to some embodiments. [Figure 15] A diagram showing the movement of a probe and the position of a corresponding measurement beam or an imaging beam according to some embodiments. [Figure 16] A diagram of a method for imaging tissue according to some embodiments. [Figure 17] A diagram of a femtosecond laser and an OCT system including a sensor according to some embodiments.

Best Mode for Carrying Out the Invention

[0014] The following detailed description provides a better understanding of the features and advantages of the invention described in this disclosure in accordance with the embodiments described in this disclosure. The detailed description includes many specific embodiments, but these are provided only as examples and should not be construed as limiting the scope of the invention disclosed herein.

[0015] The methods and systems disclosed herein can generate images of the eye, such as 3D OCT images of the eye and images of the anatomical structure of the eye, thereby enabling more ophthalmic surgeons to better image the eye and perform MIGS procedures, such as implant placement and creation of openings in the trabecular meshwork, which involve one or more of the following: tissue manipulation, incision, or excision. The methods and systems disclosed herein are well suited for use with surgical instruments such as handheld instruments and robotic manipulators. For example, the disclosed methods and apparatus enable surgery to create openings more uniformly and consistently, for example, to improve the outflow of fluid from the anterior chamber of the eye to Schlemm's canal. In addition, the disclosed systems and methods may lead to improved surgical outcomes by enabling surgeons to generate images of the eye, such as 3D images, and to identify target locations for openings to Schlemm's canal intended to increase outflow. In some cases, the target location may include a 3D position in the tissue of a tissue volume, surface, or layer, or, for example, the trabecular meshwork, juxtacanalicular trabecular meshwork (JCTM), the inner wall of Schlemm's canal, the outer wall of Schlemm's canal, the sclera, or a desired combination thereof.

[0016] The methods and apparatus of this disclosure may include a combination of imaging devices, imaging sensors, and position and orientation sensors, thereby enabling real-time display of 2D and 3D images and models for simultaneous viewing by a surgeon. Position and orientation sensor data can be combined with image data to provide improved images. Real-time display of 2D and 3D images and models may include 2D and 3D images and models that are updated during the procedure with reduced latency. In some embodiments, real-time augmented display shows images including moving images of 2D and 3D images and models as events occur. These augmented images and models allow surgeons to observe, target, and treat intraocular locations that are not easily visualized with a surgical microscope or camera alone, because the visualization of these locations within the eye is hindered by total internal reflection in unassisted microscopic images. Such structures include the trabecular meshwork and Schlemm's canal.

[0017] In some embodiments, the imager, such as a 3D imager, includes one or more of an OCT system, an ultrasonic system, or a photoacoustic system, and includes one or more emitters and associated sensors positioned on the probe's handpiece.

[0018] Images and models generated by imaging systems, as well as position and orientation sensor data, can be presented to surgeons in many ways. For example, images and models can be superimposed on images observed through a monitor or similar viewing device, such as augmented reality glasses or goggles, or virtual reality glasses or goggles. In some embodiments, real-time images from an imager are presented on a monocular or binocular head-up display along with optical images from a microscope, such as a surgical microscope, allowing the surgeon to view both the optical image and the generated 2D and 3D images and models while looking through the microscope or viewing the images generated by the microscope. The surgeon may also be provided with additional information such as virtual images and models of structures that are not normally observable, and one or more symbols indicating both the distance and movement from the probe tip to the trabecular meshwork and Schlemm's canal.

[0019] In some embodiments, imaging systems can be used to identify the collecting canals of the eye, allowing surgeons to locate these target locations displayed to the user (e.g., by using graphical visual elements such as treatment reference markers to identify target locations) to help create openings in the appropriate locations in the trabecular meshwork to increase flow. In some embodiments, images such as collecting canal images can be acquired preoperatively using OCT imaging and overlaid on images of the eye to allow surgeons to locate the collecting canals. In some embodiments, image analysis algorithms can be applied to images to recognize intraocular anatomical features during surgery, and a head-up display can augment real-time 2D and 3D images and models using the recognized features, guides, locations, markers, etc., to assist surgeons in performing surgery.

[0020] Such displays can be coupled to a surgical microscope to present monocular or binocular virtual and / or augmented 2D and 3D images and models from a display visually combined with the binocular optical real image of the eye. The methods and apparatus disclosed herein are well suited for use in conjunction with ELT surgery and implant devices such as stent surgery, which provides an opening for draining bodily fluids from the eye. However, the systems and methods provided can also be applied to a variety of other surgical procedures in which fiber-optic-based imaging may be used, such as any surgery using an endoscope.

[0021] While there is a specific reference to the treatment of glaucoma using excimer laser trabeculotomy ("ELT"), the methods and systems disclosed herein can also be used in many other types of surgery. For example, the embodiments disclosed herein can be used in conjunction with other surgical procedures, including, to name a few, endoscopic procedures related to orthopedics, neurosurgery, neurology, otolaryngology (ENT), abdominal, thoracic, cardiovascular, epicardial, endocardial, and other applications. The methods and apparatus disclosed herein can utilize in-situ imaging to improve targeting accuracy and provide virtual visualizations that enable surgeons to perform procedures in areas that are not readily visualized microscopically or endoscopically, such as images obtained with surgical microscopes, gonioscopy lenses, and slit lamps. Such applications include any endoscopic procedure in which virtual visualizations are extended to images of actual objects to support surgical precision in three-dimensional space, one example being endovascular procedures involving curved or bent blood vessels. In some cases, an in-situ imaging system is carried by or with the treatment probe to capture images from or along the route to the treatment site, allowing the surgeon to see the actual anatomical features.

[0022] Some embodiments can also be used to treat and modify organs such as the brain, heart, lungs, intestines, skin, kidneys, liver, pancreas, stomach, uterus, ovaries, testicles, bladder, ears, nose, and mouth; bone marrow, fatty tissue, muscle, glandular and mucous membrane tissue, spinal cord and nerve tissue, soft tissues such as cartilage; hard biological tissues such as teeth and bone; and other organs such as body cavities and passages such as the sinuses, ureters, colon, esophagus, pulmonary airways, blood vessels, and throat. For example, the devices disclosed herein may be inserted through existing body cavities or through openings created within body tissue.

[0023] In some embodiments, tissue motion is measured and used to construct one or more images as described herein. In some embodiments, the tissue being imaged includes vascular tissue or tissue of a pulsatile fluid system such as the trabecular meshwork and collecting ducts, as described herein. In some embodiments, the collecting ducts and Schlemm's canal are coupled to the vascular system by pulsatile flow through the ocular tissue being imaged. In some embodiments, a pulsatile pump, such as the heart, generates periodic motion of tissue associated with the pulsatile flow. In some embodiments, the pulsatile flow may cause biological tissue to move in a "jellyfish"-like manner. In some embodiments, this pulsating motion causes image displacement in images acquired from imaging sensors as described herein. In some embodiments, the displacement can be monitored and corrected by, for example, detecting the pulsations of the cardiac cycle and correcting the displacement in response to a sensor configured to detect the cardiac cycle, for example.

[0024] Referring to Figure 1, a brief overview of the anatomical structure of the eye E is given to help understand the embodiment described. As schematically shown in Figure 1, the outer layer of the eye includes the sclera 17. The cornea 15 is the transparent tissue that allows light to enter the eye. The anterior chamber 7 is located between the cornea 15 and the iris 19. The anterior chamber 7 contains a clear fluid called aqueous humor 1 that flows constantly. The lens 4 is supported and moved within the eye by fibrous zonules attached to the ciliary body 20. The iris 19 is attached circumferentially to the scleral promontory and contains the central pupil 5. The diameter of the pupil 5 controls the amount of light that passes through the lens 4 to the retina 8. The posterior chamber 2 is located between the iris 19 and the ciliary body 20.

[0025] As shown in Figure 2, the anatomical structure of the eye also includes the trabecular meshwork (TM) 9, a triangular band of intraocular spongy tissue located anterior to the attachment of the iris 19 to the scleral promontory. The mobile trabecular meshwork is constantly changing in shape, and its size is microscopic. Its cross-section is generally triangular, and its thickness varies between approximately 100 and 200 μm. It is composed of various fibrous layers with micron-sized pores that form fluid pathways for draining aqueous humor from the anterior chamber. The trabecular meshwork 9 has been measured to be approximately 100 μm thick at its anterior edge, the Schwalbe line 18, roughly at the junction of the cornea 15 and sclera 17.

[0026] The trabecular meshwork extends to approximately 200 μm at its base, where it and the iris 19 attach to the scleral promontory. The height of the trabecular meshwork can be approximately 400 μm. The passages through the pores of the trabecular meshwork 9 pass through a very thin, porous tissue called the perivascular trabecular meshwork 13, which is in contact with the inner wall of Schlemm's canal 11, a vascular structure. The height of Schlemm's canal can be approximately 200 μm, or about half the height of the trabecular meshwork. Schlemm's canal (SC) 11 is filled with a mixture of aqueous humor and blood components and connects to a series of collecting channels (CC) 12 that drain the aqueous humor into the venous system. Aqueous humor is constantly produced by the ciliary body and enters the anterior chamber through the pupil. From there, it flows through pores in the TM and JCTM into the SC, collecting duct, and aqueous vein. Therefore, obstruction in the trabecular meshwork, peritubular trabecular meshwork, or Schlemm's canal prevents the aqueous humor from easily draining from the anterior chamber. Because the eye is essentially a sphere surrounded by thick walls, inflow under conditions of obstructed outflow can increase intraocular pressure. Increased intraocular pressure can cause damage to the retina and optic nerve, and may ultimately lead to blindness.

[0027] In most open-angle glaucomas (i.e., glaucomas characterized by the easy observation of the trabecular meshwork on gonioscopy), obstruction of aqueous humor outflow is usually localized between the trabecular meshwork 9 and Schlemm's canal 11, more specifically in the region of the perivascular trabecular meshwork (JCTM) 13 located on the inner wall of Schlemm's canal.

[0028] For example, if occlusion develops in the perivascular trabecular meshwork 13, intraocular pressure gradually increases over time. Therefore, the goal of current glaucoma treatment is to prevent optic nerve damage by reducing or slowing the progressive and chronic increase in intraocular pressure.

[0029] Referring to Figure 3, a lateral cross-sectional view of the internal anatomical structure of a human eye E is shown, and according to several embodiments, a treatment probe including an optical fiber probe 23 and an imaging probe such as a 2D or 3D imaging device coupled to or incorporated into the probe is inserted into the eye. A small self-closing puncture incision 14 is made in the cornea 15. The anterior chamber can be stabilized using an anterior chamber maintainer utilizing fluid flow or a viscoelastic agent. The optical fiber treatment probe 23 and the imaging probe 25, such as a 2D or 3D imaging device, are then positioned within the anterior chamber 7 through the incision 14 and advanced until the distal end of the optical fiber treatment probe 23 contacts and slightly compresses the desired target TM tissue. The imaging probe 25 may include any suitable imaging probe such as an optical imaging probe, endoscope, OCT imaging probe, ultrasound imaging probe, or photoacoustic imaging probe, as described herein.

[0030] In some embodiments, the distal tip of a treatment probe, such as an optical fiber probe 23, extends beyond the distal tip of an imaging probe 23, and the distal tip of the treatment probe is imaged with the imaging probe. Although separate probes are mentioned, in some embodiments, the probes are combined into a single probe to perform imaging and treatment using a housing that covers both the treatment channel and the imaging channel, for example, as described herein. In some embodiments, the imaging channel and the treatment channel are configured to image the target tissue region and the tip of the treatment probe simultaneously, so that the surgeon can see where the treatment is being performed, with the tip of the treatment probe extending beyond the tip of the imaging probe, for example, to observe the treatment probe with the imaging probe. This configuration, in which the treatment tip extends beyond the imaging tip, can be used, for example, for the tip of a laser treatment probe, the tip of a stent treatment probe, the tip of a tissue manipulator, and the tip of an incision instrument.

[0031] In some embodiments, the distal end of the treatment channel, for example the treatment fiber in the case of a laser, is offset a few millimeters forward of the observation fiber so that the treatment channel can be visualized using an imager. In some embodiments, the treatment channel includes an implant that can be visualized using an imager, such as a 3D imager, for implant placement.

[0032] The photoablation laser energy produced by the laser unit 31 (shown in Figure 7) is guided from the distal end of the optical fiber probe 23 in contact with the tissue to be excised. The tissue to be excised may include the trabecular meshwork 9, the circumferential trabecular meshwork 13, and the inner wall of Schlemm's canal 11. An opening is made in the proximal inner wall of Schlemm's canal 11 in a manner that does not penetrate the distal outer wall of Schlemm's canal. In some embodiments, additional openings are made in the target tissue. Thus, one or more resulting openings are effective in restoring a relatively normal drainage rate of aqueous humor. The photoablation laser energy may include one or more types of laser energy, such as visible, ultraviolet, near-infrared, or infrared laser energy, and combinations thereof. In some embodiments, the laser energy includes 308 nm laser energy from a xenon chloride excimer laser. The laser may include, for example, pulsed energy or substantially continuous energy. In the examples, the laser energy guided from the probe includes, for example, femtosecond or picosecond laser energy.

[0033] The optical fiber probe 23 may include one or more optical fibers encapsulated by an encapsulation sheath. In some embodiments, the diameter of a single treatment optical fiber should be large enough to transmit sufficient light energy to effectively cause excision, such as photoablation, of the target tissue. In some embodiments, the diameter of the imaging optical fiber is in the range of about 4 to 6 μm. A single optical fiber or multiple optical fibers can be used as a bundle, for example, in the range of diameters from about 100 μm to about 1000 μm. The optical fiber core and cladding can be enclosed within an outer metal sleeve or shield. In some embodiments, the sleeve is formed from stainless steel. In some embodiments, the outer diameter of the sleeve is less than about 100 μm. In some embodiments, the diameter can be reduced to 100 μm, and thinner optical fibers are implemented in the laser light guide system. In some cases, the optical fiber has a diameter of about 200 μm, and the optical fiber probe 23 may have a larger diameter, such as 500 μm, to encapsulate one or more optical fibers. In some embodiments, the sleeve can be flexible so that it can be bent or angled.

[0034] Figures 4A and 4B show internal structures of the eye that can be observed with imagers such as OCT, ultrasound, and photoacoustic devices and systems described herein. These structures can also be observed using one or more of the following: for example, gonioscopy lenses, slit lamps, or microscopes. Structures that can be observed with a 3D imager using ab interno techniques as described herein include the ciliary zone 302 and the scleral promontory 304. In some embodiments, the Schwalbe lines 306 can be observed with the 3D imager or a part of the 3D imager, such as the emitter, inserted into the eye. In some embodiments, Schlemm's canal 308 may be visible in the image during surgery, depending on the intraocular pressure of the eye. The methods and apparatus disclosed herein may be well suited for identifying or estimating ocular location structures that are not readily observable with simple camera images, such as images from a camera optically coupled to an endoscope inserted into the eye.

[0035] Referring to Figure 5A, an example probe apparatus 501 and imaging device 502 according to several embodiments are shown. The imaging device 502 may include an OCT imaging device. Part of the imaging device 502 may be housed in a housing 504, such as a handheld housing, including the interferometer portion of an OCT imager, including an OCT light source, a beam splitter, and reference and sample detectors such as an array of detector elements. Alternatively, or in combination, part of the imaging device may be located outside the handheld housing, such as the interferometer portion of an OCT imager, including an OCT light source, a beam splitter, and reference and sample detectors such as an array of detector elements. In some embodiments, one or more channels 550 may extend along the length of the probe 560.

[0036] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an optical fiber for transmitting and receiving an OCT measurement or imaging beam. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye. Although separate imaging and treatment channels are mentioned, in some embodiments, for example, two channels are coupled into a single channel when a single optical fiber is used in a configuration multiplexed for the OCT measurement or imaging beam and the laser treatment beam.

[0037] As those skilled in the art will understand, the OCT imaging device may include any suitable OCT imaging device, such as a wide-spectrum imaging device with a movable mirror, a Fourier-region imaging device, a spectral-region OCT imaging device, or a sweep light source OCT imaging device.

[0038] The probe device 501 may include a motion sensor 520 that measures the movement of the probe device 501 in three translational degrees of freedom and three rotational degrees of freedom. For example, the sensor 520, which may be called a motion sensor, can measure the movement of the probe device 501 in three translational degrees of freedom and three rotational degrees of freedom. The three translational degrees of freedom may be three orthogonal axes such as the x-axis 522, the y-axis 524, and the z-axis 526. While the x-axis and y-axis are in a plane perpendicular to the z-axis, the z-axis may be aligned with or parallel to the imaging axis and / or treatment axis 564 of the probe 560. The three rotational degrees of freedom may be around three orthogonal axes, for example, a first rotational degree of freedom 523 around the x-axis 522, a second rotational degree of freedom 525 around the y-axis 525, and a third rotational degree of freedom 527 around the z-axis.

[0039] The sensor 520 may include any suitable sensor, such as one or more of the following: an accelerometer, a micro-electro-mechanical system ("MEMS") accelerometer, an inertial sensor, a magnetic field sensor, a gyroscope, a gyrocompass, or an inertial measurement unit (IMU). The sensor may be configured to measure orientation and acceleration along three or more axes, such as a three-axis accelerometer configured to measure the position and orientation of a probe.

[0040] The treatment probe 560 can include any suitable length between its distal tip 562 and the handpiece housing 504, for example, 2 mm to 50 mm in length, such as 2.5 mm to 40 mm. One or more components of the imaging device 502, such as an imaging sensor, can be positioned between 2 mm and 50 mm from the distal tip 562 of the treatment probe, such as 2.5 mm to 30 mm from the tip of the treatment probe. In some embodiments, one or more components of the imaging device 502 are located within the housing 504 of the handpiece of the treatment apparatus 501. Alternatively, or in combination, one or more components of the 3D imaging device 502 can be located away from the handpiece containing the treatment probe 560 and housing 504, for example, within the console of a surgical imaging system as described herein. One or more channels 550 may include optical fibers for transmitting light from the tip 562 of the probe.

[0041] In some embodiments, the imaging channel 552 extends to a distal tip 563, which is offset by a certain axial distance from the distal tip 562 of the probe 560, including the treatment channel 554, in order to simultaneously image the distal tip 562 and tissue using the imaging channel 552. In some embodiments, the distal tip 562 of the probe 560, including the treatment channel 554, extends distally to a longer distance than the distal tip 563 of the imaging channel 552, such as a 3D imaging channel, in order to simultaneously image the tissue and the distal tip 562 with the energy emitted from the tip 563 of the imaging channel 552. In some embodiments, the distance between the distal tips 562 of the probe 560, including the treatment channel 554, is in the range of 0 mm to 20 mm, and can be, for example, in the range of 1 mm to 10 mm or 2 mm to 8 mm.

[0042] In some embodiments, the distal end of the probe 560 and one or more channels 550 may be formed as inclined surfaces having an angle with respect to the longitudinal axis of the probe 560 in order to compress the trabecular meshwork, which may be inclined with respect to the elongated axis of the probe.

[0043] The imaging channel 552 and the treatment channel 554 can be positioned in several ways. For example, the imaging channel 552 can be positioned below the treatment channel 554, as shown. Alternatively, the imaging channel 552 can be positioned above the treatment channel 554, for example.

[0044] In some embodiments, for example, a single optical fiber is used as the treatment channel or imaging channel. In some embodiments, a bundle of optical fibers, such as two or more fibers, can be used with the disclosed systems and methods. In some examples, the treatment channel 554 of the probe 560 includes a bundle of optical fibers, each having a distal end, at or near the distal end of the probe device 501.

[0045] In some embodiments, the 3D image is constructed in response to image displacement, which may be related to the altered orientation of one or more fibers, such as a single fiber or one or more fibers in a multi-fiber array. The image displacement can be used to generate depth data. The image displacement may be related to the movement of the probe or to the movement of tissue, such as pulsating motion as described herein.

[0046] In some embodiments, the OCT system 502 images the eye using OCT A scans. During procedures such as imaging the patient's eye, the probe device 501 moves in translation and rotation, allowing the OCT system 502 to generate A scans from different positions and orientations. This movement can result from various causes, such as human tremor or resonance modes from a robotic arm. Position and orientation data can be combined with OCT data to construct an image in response to the probe's position and orientation.

[0047] The probe device 501 may be coupled electronically to a 3D imager 401 and a control unit 410, both of which are described in further detail herein, including with respect to Figure 7. The 3D imager 401 may include one or more of any suitable imagers for imaging tissue, such as an optical imager, an optical coherence tomography (OCT) imager, an ultrasound (US) imager, or a photoacoustic imager. In some embodiments, a sensor is coupled to the imager to acquire sensor data associated with one or more of the imager's position or orientation. In some embodiments, a processor is coupled to the imager and sensor to acquire image data and sensor data, and the processor consists of instructions to construct a 3D image of the tissue in response to the imager data and sensor data.

[0048] In some embodiments, the system can be configured in many ways, but the imager includes a sensor configured to generate motion data and a processor configured to reconstruct a 3D image in response to the motion data.

[0049] The imager may include any imager described herein, but in some embodiments, the 3D imager 401 includes an optical imager, which includes one or more of an endoscope, microscope, stereomicroscope, and stereoendoscopy. In some embodiments, the optical imager is configured to capture motion-related images, and the processor is configured to generate motion data and construct a 3D image in response to the motion data, as described herein. The optical imager may be configured to generate 3D image data, such as stereo photogrammetry using images captured from a sensor array, as described herein.

[0050] During operation, the imaging device 502, such as an OCT imaging device, can generate imaging data, such as A-scans, while the sensor 520 measures both the translational and rotational movement of the imaging device. The imaging data and motion data are time-stamped or otherwise associated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the OCT scanner. The imaging data and motion data may be processed by a 3D imager 401 and / or a control unit 410 to construct a 3D model. The 3D model can be constructed based on the acquired data while the OCT imaging device moves to capture data from different parts of the patient's eye. The 3D imager 401 and / or control unit 410 can then assemble the 3D model by positioning each A-scan in a spatial orientation relative to the other A-scans, based on the sensor's motion data.

[0051] In some embodiments, the imaging device and sensor may be coupled to a processor configured to acquire image data from the image sensor and sensor data such as motion data from the sensor, and to construct one or more of 2D images, 3D images, or 3D models based on or in response to the image data and sensor data. The imaging device may have an axis such as the z-axis, and along that axis, emits a beam of imaging energy such as coherent light energy. The axis of the beam may be aligned with or parallel to at least one of the sensor axes 522, 524, 526 for measuring one or more of the position or orientation beams of imaging energy. The sensor may be configured to measure the motion of the imaging device and / or imaging beam in a direction along the axis of the beam, corresponding to the motion of the measurement beam or imaging beam along the axis. The processor can then construct a 2D image, 3D image, or 3D model such as a tissue model in response to the motion of the measurement beam or imaging beam along the axis.

[0052] In some embodiments, the sensor is configured to measure the motion of the imager orthogonal to the axis of the measurement beam or imaging beam axis 564. Such motion includes motion in a plane substantially perpendicular to the axis of the measurement beam or imaging beam (e.g., within about 10 degrees from the perpendicular), and in some embodiments, is also perpendicular to the z axis, such as the plane defined by the x axis 522 and the y axis 524. The processor can then construct a 2D image, a 3D image, or a 3D model such as a tissue model in response to the motion of the measurement beam or imaging beam orthogonal to the axis. In some embodiments, the motion data includes rotational motion data, such as rotation around one or more of the x axis 522, the y axis 524, or the z axis 526. In some embodiments, rotation of the probe around its elongated axis, for example rotation around the z axis 526, causes the imaging channel 552 and the treatment channel 554 to rotate relative to each other, and the processor can be configured with instructions to construct an image such as a 3D image in response to the rotation of the probe around its elongated axis.

[0053] In some embodiments, the motion data may include rotational data of the probe and the attached imaging device. In some embodiments, the motion data may include translational data of the probe and the attached imaging device.

[0054] In some embodiments, one or more processors located in one or more of the 3D imager 401 and control units 410 can store position and orientation data of the imager while the beam of imaging energy is directed toward the tissue, and can construct 2D images, 3D images, or 3D models from multiple positions and orientations of the sensor.

[0055] As discussed herein, an imager can generate A-scans using a measurement beam or imaging beam, and a processor can use sensor data to determine the position of the measurement beam or imaging beam for each of a plurality of A-scans. The processor can then construct an image in response to a plurality of positions of the measurement beam or imaging beam. In some embodiments, the position of the measurement beam or imaging beam is related to one or more of the positions or orientations of the measurement beam or imaging beam.

[0056] In some embodiments, the probe device 501 may include a treatment channel 554 for treating tissue. The motion sensor and imager may be coupled to the probe 560, imaging channel 552, and treatment channel 554 in a fixed relationship, such as fixed position and orientation relative to each other.

[0057] One or more channels 550 are shaped or otherwise configured to deliver an implant, one or more of the treatment energy, and a combination thereof to the tissue for treatment. In some embodiments, one or more channels 550 include optical fibers for guiding laser energy to the tissue. In some embodiments, the channels include a working channel for delivering the implant to the tissue.

[0058] As discussed herein, the probe device 501 may include a housing 504. The housing may be a handheld housing, such as a handpiece, which may be coupled to or include an imager and a sensor. In some embodiments, the handheld device is moved during use. Some movements may be purposeful or intentional, and some movements may be unintentional. Motion sensors can measure both types of movements. One type of involuntary movement is tremor in a user, such as a surgeon, when holding the probe during a patient procedure. This tremor can be relatively small, ranging from one to a few millimeters or less. Because the structure of the eye is small, even such small hand movements, with amplitudes of less than one millimeter or less than three millimeters, can be used to construct one or more 2D images, 3D images, or 3D models, or the treatment area of ​​the eye. In some embodiments, a tremor model can be constructed to characterize the tremor of a user, such as a surgeon. This tremor model, which may be based on periodic motion including the frequency and amplitude of tremors measured over time by the user, is used to generate sensor data, which is then further used together with imager data to generate 2D images, 3D images, or 3D models of the patient's tissue. In some embodiments, the processor can determine the position of the measurement beam or imaging beam within the tissue, for example, based on the sensor data and the tremor model.

[0059] While the movement of the probe and the structure of sensors are mentioned, in some embodiments, the tissue moves with respect to the probe in a repetitive motion, such as described herein. In some embodiments, the repetitive motion of the tissue is related to a pulsating motion, such as described herein. In some embodiments, the repetitive motion, such as a periodic motion, is measured using a detector, such as described herein. The detector can be configured in many ways, but in some embodiments, the detector includes an array detector. The processor can be configured to fit the motion data to a periodic model, such as a tremor or resonance model, as described herein, and the motion data can be used, for example, to construct a 3D image.

[0060] In some embodiments, an artificial intelligence model trained using eye imaging data can generate 2D images, 3D images, or 3D models of patient tissue from, for example, A-scan data and one or more tremor or resonance models. Positional and orientation data can be used together with one or more tremor or resonance models to develop one or more tremor or resonance models.

[0061] In some embodiments, motion data includes periodic motion associated with pulsatile flow, such as the movement of one or more tissues connected to the cardiovascular system, such as one or more Schlemm's canals, collecting canals, or trabecular meshwork. Periodic data can be acquired and used to construct 3D images as described herein. Periodic data may include any suitable periodic data associated with one or more of, for example, tremors, resonances, or heartbeats. Motion data may include periodic motion data such as periodic motion associated with one or more of, for example, tremors, resonances, or pulsating tissue movements. In some embodiments, the periodic motion corresponds to harmonics that can be determined in response to the motion data. In some embodiments, the periodic motion corresponds to pulsations in tissues such as the choroid beneath the retina.

[0062] Referring to Figure 5B, an example probe apparatus 501 and imaging device 502 according to several embodiments are shown. The imaging device 502 may include, for example, an ultrasonic imaging device. Part of the imaging device 502 may be housed in a housing 504, such as a handheld housing, including, for example, a control circuit for an ultrasonic transducer 565 configured to emit a measurement beam and receive the beam with respect to axis 564. The ultrasonic transducer 565 may be located, for example, near the tip 562 of the probe 560 and coupled to the rest of the imaging device 502. In some embodiments, one or more channels 550 may extend along the length of the probe 560. The treatment probe can be between 2 mm and 10 mm in length, such as between 2.5 mm and 5 mm.

[0063] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an ultrasound transducer 565 and associated wiring for coupling the transducer to an imaging device 502. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye.

[0064] The transducer 565 of the ultrasound imaging device 502 can be configured in many ways, for example, including one or more single transducers or arrays of transducers. The ultrasound imaging device 502 can image the eye, for example, using an ultrasound A-scan. During a procedure, such as imaging a patient's eye, the probe device 501 can move in translation and rotation, causing the ultrasound system 502 to generate an A-scan from different positions and orientations of the measurement beam or imaging beam. Position and orientation data acquired using sensors can be combined with the A-scan to generate an image of the tissue.

[0065] The probe device 501 may include a motion sensor 520 that measures the movement of the probe device 501 in three translational degrees of freedom and three rotational degrees of freedom. For example, the sensor 520 can measure the movement of the probe device 501 in three translational degrees of freedom and three rotational degrees of freedom. The three translational degrees of freedom may be three orthogonal axes such as the x-axis 522, the y-axis 524, and the z-axis 526. The z-axis may be aligned with or parallel to the imaging axis and / or treatment axis 564 of the probe 560, while the x-axis and y-axis are in a plane substantially perpendicular to the z-axis, for example, within about 10 degrees from the perpendicular. The three rotational degrees of freedom may be around three orthogonal axes, for example, a first rotational degree of freedom 523 around the x-axis 522, a second rotational degree of freedom 525 around the y-axis 525, and a third rotational degree of freedom 527 around the z-axis.

[0066] The probe device 501 may be coupled electronically to the 3D imager 401 and the control unit 410, both of which are described in further detail herein, including with respect to Figure 7. During operation, the imaging device 502, such as an ultrasound imaging device, can generate imaging data, such as A-scans, while the sensor 520 measures both translational and rotational motion of the imaging device. The imaging data and motion data are time-stamped or otherwise associated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the ultrasound imaging device. The imaging data and motion data may be processed by the 3D imager 401 and / or the control unit 410 to construct a 3D model. The 3D model can be constructed while the ultrasound imaging device moves to capture data from different parts of the patient's eye. The 3D imager 401 and / or the control unit 410 can then assemble the 3D model by positioning each A-scan or other ultrasound image data in spatial orientation relative to other A-scans, based on the sensor's motion data.

[0067] In some embodiments, the imaging device and sensor may be coupled to a processor configured to acquire image data from the image sensor and sensor data such as motion data from the sensor, and to construct one or more of 2D images, 3D images, or 3D models based on or in response to the image data and sensor data. The imaging device may have an axis such as the z-axis, which emits a beam of imaging energy, such as ultrasound, along that axis. The axis of the beam may be aligned with or parallel to at least one of the sensor axes 522, 524, 526 for measuring one or more of the position or orientation beams of imaging energy. The sensor may be configured to measure the motion of the imaging device and / or the imaging beam in a direction along the axis of the beam, corresponding to the motion of the measurement beam or imaging beam along the axis. The processor can then construct one or more of 2D images, 3D images, or 3D models in response to the motion of the measurement beam or imaging beam along the axis.

[0068] In some embodiments, the sensor is configured to measure the motion of the imager orthogonal to the axis 564 of the measurement beam or imaging beam. Such motion can be in a plane substantially perpendicular to the axis of the measurement beam or imaging beam (e.g., within about 10 degrees from the vertical), and in some embodiments, also substantially perpendicular to the z axis (e.g., within about 10 degrees from the vertical), such as the plane defined by the x axis 522 and the y axis 524. The processor can then construct one or more of a 2D image, a 3D image, or a 3D model in response to the motion of the measurement beam or imaging beam perpendicular to the axis.

[0069] In some embodiments, the motion data may include rotational data of the probe and the attached imaging device. In some embodiments, the motion data may include translational data of the probe and the attached imaging device.

[0070] In some embodiments, one or more processors located in one or more of the 3D imager 401 and control units 410 can store position and orientation data of the imager while the ultrasonic beam of imaging energy is directed toward the tissue, and can construct one or more of 2D images, 3D images, or 3D models from multiple positions and orientations of the sensor.

[0071] As described herein, an imager generates A-scans using an ultrasonic measurement beam or imaging beam, and a processor can use sensor data to determine one or more positions or orientations of the ultrasonic measurement beam or imaging beam for each of a plurality of A-scans. The processor can then construct an image in response to multiple positions of the ultrasonic measurement beam or imaging beam.

[0072] In some embodiments, the probe device 501 may include a probe 560 and a treatment channel 554 for treating tissue. The motion sensor and imager may be coupled to the probe 560, imaging channel 552, and treatment channel 554 in a fixed relationship, such as fixed position and orientation relative to each other.

[0073] One or more channels 550 are shaped or otherwise configured to deliver one or more of the implant, treatment energy, or other components to the tissue for treatment. In some embodiments, one or more channels 550 include optical fibers for guiding laser energy to the tissue. In some embodiments, the channels include working channels for delivering the implant to the tissue.

[0074] As discussed herein, the probe device 501 may include a housing 504. The housing may be a handheld housing, such as a handpiece, which may be coupled to or include an imager and a sensor. Handheld devices are moved during use. Some movements are purposeful or intentional, and some movements are involuntary. Motion sensors can measure both types of movements. One type of involuntary movement is tremor data of a user, such as a surgeon, when holding the probe during a patient procedure. This tremor movement may be relatively small, on the order of 1 to a few millimeters. Because the structure of the eye is small, even such small hand movements, with amplitudes of less than 1 millimeter or less than 3 millimeters, can be used to construct 2D images, 3D images or 3D models, or the treatment area of ​​the eye. In some embodiments, a tremor model can be constructed to characterize the surgeon's tremor. This tremor model, which may be based on periodic motion including the frequency and amplitude of tremors measured over time by the surgeon, is used to generate sensor data, which is then further used together with imager data to generate 2D images, 3D images, or 3D models of the patient's tissue. In some embodiments, the processor can determine the position of the measurement beam or imaging beam within the tissue based on the sensor data and the tremor model.

[0075] In some embodiments, an artificial intelligence model trained using eye imaging data can generate 2D images, 3D images, or 3D models of patient tissue from A-scan data and tremor models, for example, without using positional and orientation data.

[0076] Referring to Figure 5C, an example probe device 501 and imaging device 502 according to several embodiments are shown. The imaging device 502 may include a photoacoustic imaging device. Components of the imaging device 502 can be located in any suitable location, such as near the probe tip, inside the housing of the handpiece, or inside the console of a treatment station as described herein. Part of the imaging device 502, such as the acoustic transducer circuit portion 568 of the photoacoustic imaging device, may be housed in a housing 504, such as a handheld housing. The sensor may be located on the outer surface of the housing 504 or may be acoustically coupled to the external environment outside the housing. In some embodiments, the sensor includes an ultrasonic detector for detecting sound waves produced when light from the imaging device comes into contact with tissue. In some embodiments, the acoustic transducer 565 may be located at the distal end of the probe, such as the tip. In some embodiments, one or more channels 550 may extend along the length of the probe 560. The treatment channel of the probe 560 can be between 2 mm and 10 mm in length, such as between 2.5 mm and 5 mm. The imaging device 502, including the imaging sensor, can be between 2 mm and 10 mm from the tissue, such as between 2.5 mm and 5 mm, when in use. One or more channels 550 can be optical guide channels that direct light from the photoacoustic transducer from the end of the treatment probe to the tissue. One or more channels 550 can include optical fibers for passing light from the tip 562 of the probe.

[0077] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an optical fiber for transmitting a photoacoustic excitation beam to induce measurable vibrations in tissue by a transducer 567. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye. Although separate imaging and treatment channels are mentioned, in some embodiments, for example, two channels are coupled into a single channel when a single optical fiber is used in a configuration multiplexed for the photoacoustic excitation beam and the laser treatment beam.

[0078] The distal end of the probe 560 can be formed as an inclined surface at an angle to the longitudinal axis of the probe 560, for example, so that the angled end of the optical fiber extending along the treatment channel 554 contacts the trabecular meshwork tissue.

[0079] In some embodiments, a single optical fiber is used for the treatment channel 554 and the imaging channel 554. In some embodiments, a bundle of optical fibers, such as two or more fibers, can be used with the disclosed systems and methods. In some examples, the probe 560 includes a bundle of optical fibers, each having a distal end, at or near the corner of the distal end of the treatment probe 500.

[0080] The photoacoustic imaging device 502 can image the eye using photoacoustic A-scans. During procedures such as imaging the patient's eye, the probe device 501 can move in translation and rotation, causing the photoacoustic system 502 to generate A-scans from different positions and orientations.

[0081] The probe device 501 includes a motion sensor 520 as described herein, and motion data can be measured as described herein and combined with image data.

[0082] The probe device 501 may be coupled electronically to the 3D imager 401 and the control unit 410, both of which are described in further detail herein, including with respect to Figure 7. During operation, the imaging device 502, such as a photoacoustic imaging device, can generate imaging data, such as A-scans, while the sensor 520 measures both the translational and rotational motion of the imaging device. The imaging data and motion data are time-stamped or otherwise associated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the photoacoustic imager. The imaging data and motion data may be processed by the 3D imager 401 and / or the control unit 410 to construct a 3D model. The 3D model can be constructed while the photoacoustic imaging device moves to capture data from different parts of the patient's eye. The 3D imager 401 and / or the control unit 410 can then assemble the 3D model by positioning each A-scan in a spatial orientation relative to the other A-scans, based on the motion data from the sensor.

[0083] In some embodiments, the imaging device and sensor may be coupled to a processor configured to acquire image data from the image sensor and sensor data such as motion data from the sensor, and to construct one or more of 2D images, 3D images, or 3D models based on or in response to the image data and sensor data as described herein.

[0084] In some embodiments, the sensor is configured to measure the motion of the imager orthogonal to the axis 564 of the measurement beam or imaging beam. Such motion can be in a plane substantially perpendicular to the axis of the measurement beam or imaging beam (e.g., within 10 degrees from the vertical), and in some embodiments, also substantially perpendicular to the z axis (e.g., within 10 degrees from the vertical), such as the plane defined by the x axis 522 and the y axis 524. The processor can then construct one or more of 2D images, 3D images, or 3D models in response to the motion of the measurement beam or imaging beam perpendicular to the axis.

[0085] In some embodiments, the motion data may include rotational data of the probe and the attached imaging device. In some embodiments, the motion data may include translational data of the probe and the attached imaging device.

[0086] In some embodiments, one or more processors located in one or more of the 3D imager 401 and control units 410 can store position and orientation data of the imager while the beam of imaging energy is directed toward the tissue, and can construct 2D images, 3D images, or 3D models from multiple positions and orientations of the sensor.

[0087] As described herein, an imager can generate an A-scan using a measurement beam or an imaging beam, and a processor can use sensor data to determine the position of the measurement beam or imaging beam for each of a plurality of A-scans. The processor can then construct an image in response to multiple positions of the measurement beam or imaging beam.

[0088] In some embodiments, the probe device 501 may include a treatment channel 554 for treating tissue. The motion sensor and imager may be coupled to the probe 560, imaging channel 552, and treatment channel 554 in a fixed relationship, such as fixed position and orientation relative to each other.

[0089] One or more treatment channels 554 of channel 550 are shaped or otherwise configured to deliver one or more of an implant or treatment energy to the tissue in order to treat the tissue. In some embodiments, one or more channels 550 include optical fibers for guiding laser energy to the tissue. In some embodiments, the channels include working channels for delivering the implant to the tissue.

[0090] As discussed herein, the probe device 501 may include a housing 504. The housing may be a handheld housing, such as a handpiece, which may be coupled to or include an imager and a sensor. The handheld device is moved during use. Some movements may be purposeful or intentional, and some movements may be unintentional. Motion sensors can measure both types of movements. One type of involuntary movement is tremor data of a user, such as a surgeon, when holding the probe during a patient procedure. This tremor movement may be relatively small, on the order of 1 to a few millimeters. Because the structure of the eye is small, even such small hand movements, with amplitudes of less than 1 millimeter or less than 3 millimeters, can be used to construct 2D images, 3D images or 3D models, or the treatment area of ​​the eye. In some embodiments, a tremor model can be constructed to characterize the surgeon's tremor. This tremor model, which may be based on periodic motion including the frequency and amplitude of tremors measured over time by the surgeon, is used to generate sensor data, which is then further used together with imager data to generate 2D images, 3D images, or 3D models of the patient's tissue. In some embodiments, the processor can determine the position of the measurement beam or imaging beam within the tissue based on the sensor data and the tremor model.

[0091] In some embodiments, an artificial intelligence model trained using ocular imaging data can generate 2D images, 3D images, or 3D models of patient tissue from A-scan data and tremor models as described herein.

[0092] Referring to Figure 6A, several exemplary probe devices 501 and imaging devices 502 are shown according to some embodiments. These may be similar to the probe devices 501 in Figures 5A to 5C and have similar features. The imaging device 502 may include a scanning OCT imaging device. Part of the imaging device 502, including the interferometer portion of the scanning OCT imaging device, such as the OCT light source, beam splitter, and reference and sample detector arrays, may be housed in a housing 504, such as a handheld housing. In some embodiments, one or more channels 550 may extend along the length of the probe 560. The treatment channel 554 of the probe can be between 2 mm and 10 mm in length, for example, between 2.5 mm and 5 mm. One or more channels 550 may be optical guide channels that guide light from the OCT system from the end of the treatment probe. One or more channels 550 may include optical fibers for passing light from the tip 562 of the probe.

[0093] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an optical fiber for transmitting and receiving an OCT measurement or imaging beam. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye. In some embodiments, the distal end of the imaging channel 552 is located proximal to the distal end of the treatment channel in order to image the tissue and the distal end of the treatment channel as described herein. Although separate imaging and treatment channels are mentioned, in some embodiments, for example, two channels are coupled into a single channel when a single optical fiber is used in a configuration multiplexed for the OCT measurement or imaging beam and the laser treatment beam.

[0094] As those skilled in the art will understand, the OCT imaging device may include any suitable OCT imaging device, such as a wide-spectrum imaging device with a movable mirror, a Fourier-region imaging device, a spectral-region OCT imaging device, or a sweep light source OCT imaging device.

[0095] The distal end of the probe 560 may include a scanner 567 that scans the OCT measurement beam or imaging beam in a pattern with respect to a measurement axis such as axis 564 by deflecting the distal end of the OCT imaging fiber or measurement beam or imaging beam, for example, using piezoelectric deflection, piezoelectric deflection of optical fiber, galvanic deflection, or a moving reflective surface such as a mirror. In some embodiments, the scanner includes a single OCT imaging fiber configured to deflect to scan the imaging beam. A scanning optical fiber suitable for incorporation in accordance with this disclosure is described in PCT / US2001 / 016844, titled “Medical imaging, diagnosis, and therapy using a scanning single optical fiber system,” filed on 23 May 2001 and published on 27 December 2001 as WO2001097902.

[0096] While scanners have been mentioned, in some embodiments, the processor is configured to combine tremor data and data from a scanning imaging beam to construct a 3D image as described herein.

[0097] The scanning OCT imaging device 502 can image the eye using an OCT A scan obtained by scanning the patient's tissue with the scanner 562. During procedures such as imaging the patient's eye, the probe device 501 moves in translation and rotation, causing the OCT system 502 to generate multiple scanning A scans from the scanner 567 at different positions and orientations of the probe, as measured using the sensor 520.

[0098] The probe device 501 may include motion sensors 520 that measure the translational and rotational motion of the probe device 501, as described herein.

[0099] The probe device 501 may be coupled electronically to the 3D imager 401 and / or control unit 410, both of which are described in further detail herein, with reference to Figure 7, etc. During operation, the imaging device 502, such as an OCT imaging device, can generate imaging data, such as A-scans, in a scan pattern while the sensor 520 measures both the translational and rotational motion of the imaging device. The imaging data and motion data, including the position of the measurement beam or imaging beam within the scan pattern, are time-stamped or otherwise correlated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the OCT scanner. The imaging data and motion data may be processed by the 3D imager 401 and / or control unit 410 to construct a 3D model. The 3D model can be constructed based on the acquired data while the OCT imaging device moves to capture data from different parts of the patient's eye. The 3D imager 401 and / or control unit 410 can then assemble a 3D model by positioning each A-scan in a spatial orientation relative to other A-scans, based on sensor motion data, including the use of tremor with respect to probe movement, as discussed herein, such as in Figure 5A.

[0100] Referring to Figure 6B, several examples of probe devices 501 and imaging devices 502 are shown according to several embodiments. These may be similar to the probe device 501 in Figure 5B and have similar features. The imaging device 502 may include an ultrasonic imaging device. Part of the imaging device 502, such as a control circuit for an ultrasonic transducer 565, may be housed in a housing 504, such as a handheld housing. The ultrasonic transducer 565 may be positioned near the tip 562 of the probe 560. The ultrasonic transducer 565 may include an array of sensors, such as a 1D linear array or a 2D area or planar array. The treatment probe can be between 2 mm and 10 mm in length, such as between 2.5 mm and 5 mm.

[0101] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an ultrasound transducer 565 and associated wiring for coupling the transducer to an imaging device 502. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye.

[0102] The ultrasound system 502 can image the eye, for example, using an ultrasound A scan or a beamformer that generates 2D or 3D images. During procedures such as imaging the patient's eye, the probe device 501 can move in translation and rotation, causing the ultrasound system 502 to generate imaging scans from different positions and orientations.

[0103] The probe device 501 may include a motion sensor 520 that measures the translational and rotational motion of the probe device 501. For example, the sensor 520, which may be called a motion sensor, can measure the translational and rotational motion of the three degrees of freedom, as will be discussed herein with respect to Figure 5B and other figures.

[0104] The probe device 501 may be coupled electronically to a 3D imager 401 and a control unit 410, both of which are described in further detail herein, including with respect to Figure 7. During operation, the imaging device 502, such as an ultrasound imaging device, can generate imaging data, such as A-scans formed by a 1D or 2D sensor array, while the sensor 520 measures both translational and rotational motion of the imaging device. The imaging data and motion data are time-stamped or otherwise correlated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the ultrasound imaging device. The imaging data and motion data may be processed by the 3D imager 401 and / or control unit 410 to construct a 3D model. A 3D model can be constructed because the ultrasound imaging device captures data from different parts of the patient's eye as it moves. The 3D imager 401 and / or control unit 410 can then assemble a 3D model by positioning each A-scan in a spatial orientation relative to other A-scans, based on sensor motion data, including using the tremor or resonance mode of a robotic arm with respect to probe movement, as discussed herein, such as in Figure 5B.

[0105] In some embodiments, the imaging device and sensor may be coupled to a processor configured to acquire image data from the image sensor and sensor data such as motion data from the sensor, and to construct one or more of 2D images, 3D images, or 3D models based on or in response to the image data and sensor data. The imaging device may have an axis such as the z-axis, and emit a beam of imaging energy, such as ultrasound, along that axis. The axis of the beam may be aligned with or parallel to at least one of the sensor axes 522, 524, 526 for measuring one or more of the position or orientation beams of imaging energy. In some embodiments, the sensor array may be oriented substantially perpendicular to axis 564, for example, within 10 degrees from the vertical. For example, a 1D array may be substantially perpendicular, and the plane of a 2D array may be substantially perpendicular to axis 564. The sensor may be configured to measure the movement of the imaging device and / or the imaging beam in a direction along the axis of the beam, corresponding to the movement of the measurement beam or imaging beam along the axis. The processor can then construct one or more of a 2D image, a 3D image, or a 3D model in response to the movement of the measurement beam or imaging beam along the axis.

[0106] In some embodiments, the sensor is configured to measure the motion of the imager orthogonal to the axis 564 of the measurement beam or imaging beam. Such motion may be in a plane substantially perpendicular to the axis of the measurement beam or imaging beam (e.g., within 10 degrees from the vertical), and in some embodiments, it may also be substantially perpendicular to the z axis (e.g., within 10 degrees from the vertical), such as the plane defined by the x axis 522 and the y axis 524. The processor can then construct one or more of a 2D image, a 3D image, or a 3D model in response to the motion of the measurement beam or imaging beam perpendicular to the axis.

[0107] Referring to Figure 6C, an example probe device 501 and imaging device 502 according to several embodiments are shown. This may be similar to the probe device 501 in Figure 5C and have similar features. The imaging device 502 may include a photoacoustic imaging device. Part of the imaging device 502, such as the transducer portion 568 of the photoacoustic imaging device, may be housed in a housing 504, such as a handheld housing. In some embodiments, the sensor 565 includes an ultrasonic detector for detecting sound waves produced when light from the imaging device comes into contact with tissue and induces vibration. In some embodiments, the acoustic sensor may be located at the distal end of the probe, such as the tip. In some embodiments, one or more channels 550 may extend along the length of the probe 560. The treatment probe may be between 2 mm and 10 mm in length, such as between 2.5 mm and 5 mm. The imaging device 502, including the imaging sensor, may be between 2 mm and 10 mm from the tissue, such as between 2.5 mm and 5 mm, when in use. One or more channels 550 may include an optical guide channel that directs light from the photoacoustic transducer out of the end of the treatment probe. One or more channels 550 may include an optical fiber for passing light from the tip 562 of the probe.

[0108] In some embodiments, one or more channels 550 include an imaging channel 552 and a treatment channel 554. In some embodiments, the imaging channel 552 includes an optical fiber for transmitting a photoacoustic excitation beam to induce measurable vibrations in tissue by a transducer 567. In some embodiments, the treatment channel 554 includes one or more optical fibers for guiding treatment light energy to the eye, or a channel for carrying an implant to the eye. Although separate imaging and treatment channels are mentioned, in some embodiments, for example, two channels are coupled into a single channel when a single optical fiber is used in a configuration multiplexed for the photoacoustic excitation beam and the laser treatment beam.

[0109] The distal end of the probe 560 may include a scanner 567 that scans the measurement beam or imaging beam in a pattern with respect to a measurement axis such as axis 564 by deflecting the distal end of the photoacoustic beam using, for example, piezoelectric deflection, optical fiber deflection, piezoelectric deflection of an optical fiber, galvanic deflection, or one or more moving reflective surfaces such as mirrors.

[0110] The photoacoustic device 502 can image the eye using a photoacoustic scan obtained by scanning the patient's tissue with the scanner 567. During procedures such as imaging the patient's eye, the probe device 501 can move in translation and rotation, causing the photoacoustic system 502 to generate A-scans from different positions and orientations.

[0111] The probe device 501 may include a motion sensor 520 that measures the translational and rotational motion of the probe device 501. For example, the sensor 520, which may be called a motion sensor, can measure the translational and rotational motion of the three degrees of freedom, as discussed herein.

[0112] The probe device 501 may be coupled electronically to the 3D imager 401 and the control unit 410, both of which are described in further detail herein, including with respect to Figure 7. During operation, the imaging device 502, such as the photoacoustic imaging device, can generate imaging data, such as A-scans, in a scan pattern while the sensor 520 measures both the translational and rotational motion of the imaging device. The imaging data and motion data, including the position of the beam in the scan pattern, are time-stamped or otherwise correlated with each other so that the position and orientation of the probe device 501 can be associated with each A-scan of the photoacoustic imager. The imaging data and motion data may be processed by the 3D imager 401 and / or the control unit 410 to construct a 3D model. A 3D model can be constructed because the photoacoustic imaging device captures data from different parts of the patient's eye as it moves. The 3D imager 401 and / or control unit 410 can then assemble a 3D model by positioning each A-scan in a spatial orientation relative to other A-scans, based on sensor motion data, such as that discussed herein, including the use of tremor with respect to probe movement, as shown in Figure 5C.

[0113] Figures 5A to 6C refer to a combined imaging probe and a treatment probe, although in some embodiments the probe includes an imaging probe without a treatment channel. Alternatively, the probe may be configured for treatment without imaging. In some embodiments, the probe components shown in Figures 5A to 6C are provided as a plurality of probes that can be used for treatment, for example, together with a first treatment probe for treating tissue and a second imaging probe for imaging tissue. In some embodiments, the imaging probe and the treatment probe are inserted from different incisions. For example, the imaging probe may be inserted from a first incision and the treatment probe from a second incision, and the tissue and treatment probe may be imaged simultaneously using the imaging probe.

[0114] Referring to Figure 7, a system 400 for assisting a surgeon in performing a surgical procedure on an eye E is shown according to several embodiments. The surgical procedure involves inserting a portion of an elongated probe device 501 into the eye through the opening, across the anterior chamber, to a target tissue region including the trabecular meshwork and Schlemm's canal. In some embodiments, the system 400 may include an optical microscope 409 for the surgeon to observe the eye in real time during the procedure. The 3D imager 401 receives a feed as input from an imaging device of the elongated probe device 501, which may include a portion of the system described with reference to Figures 5A to 6C, positioned inside or near the eye. The 3D imager 401 is operably coupled to a processor 414 of a control unit 410. The processor of the control unit 410 may consist of instructions to locate the structures of the eye and to overlay markers or other marks onto the input camera image. In conjunction with the optical microscope 409, an imaging device coupled to a probe device 501, as described herein, can provide data to the 3D imager 401 and the control unit 410. In some embodiments, a second camera 416, including a detector array, is optically coupled to the microscope 409 and to a processor 414 of the control unit 410 to receive optical images from the surgical microscope 409. The control unit 410 receives image data from the camera 416, processes the image data, and provides visual image data to a head-up display 407, which can overlay the visual image data onto the forward optical image of the surgical microscope 409. The microscope 409 may include, for example, a binocular surgical microscope. The system 400 includes, for example, an imaging device of the probe device 501, or a part thereof, carried in situ with the treatment probe 23 or in close proximity to the treatment probe 23 and the eye, and can, for example, enable imaging of one or more target locations before, during, or after a procedure.The imaging device of the probe apparatus 501 includes one or more components, as described with reference to Figures 5A to 6C, and herein include sensors for measuring one or more of the following: OCT imaging, ultrasound imaging, and photoacoustic imaging, as well as the position or orientation of the probe as described herein. Images captured by the imaging device are processed by the image processing apparatus 412 of the control unit 410, which can generate multiple 2D images, and 3D images or models, and augmented images and models for real-time visualization by the surgeon.

[0115] The extended image can be displayed on the head-up display 407 and, using the internal beam splitter 420, can be combined with the optical image from the microscope 409 to form a monocular or binocular image, as is known to those skilled in the art. As described herein, a microscope view may include one or more microscope images with or without overlapping virtual images, for example, an optical microscope image, a microscope image and an overlapping virtual image, or one or more microscope images with overlapping virtual images, combined with one or more of 2D images, 3D images, or models generated from imaging captured by an imaging device. If a microscope view includes overlapping images, the overlapping images can be aligned with the microscope image using alignment-enabled elements. Similarly, if a view includes imaging from a camera and an overlapping virtual image, the overlapping images can be aligned with imaging from the camera using alignment-enabled elements.

[0116] Images can be provided to the surgeon in many ways. For example, the surgeon can observe the images using an augmented reality display such as glasses or goggles, and can also observe the surgical site through a surgical microscope 409. In some embodiments, the surgeon observes the images using a virtual reality display. Alternatively, or in combination, the eye is observed on an external monitor, and the image of the eye observed on the external monitor is marked as described herein. The images observed by the surgeon may include, for example, monocular images or stereoscopic images. In some embodiments, the images observed by the surgeon may include, for example, augmented reality (AR) images or virtual reality images.

[0117] In some embodiments, a surgeon can first observe a surgical instrument, such as a part of the probe device 501, in a video image from a microscope or surgical microscope. In some cases, the surgeon can alternatively or additionally observe images such as 2D or 3D images or models generated from data by the imaging device of the probe device 501. In some embodiments, the surgeon can observe images from the microscope 409 and 2D or 3D images or models generated from data by the imaging device of the probe through the eyepiece of the microscope 409. Alternatively, or in combination, the surgeon can observe an augmented image or augmented view in which additional information is superimposed on one or more of the optical microscope images or 2D or 3D images or models generated from data by the imaging device. When the 2D or 3D images or models generated from data by the imaging device are superimposed on the image from the microscope image, the surgeon can observe both the microscope image and the superimposed 2D or 3D images or models generated from data by the imaging device simultaneously. Furthermore, the image processing device 412 can detect anatomical features of the eye as described herein and overlay markers onto one or more of the microscopic image or 2D, 3D, or models generated from the data by the imaging device to assist the surgeon in identifying and locating these features. The augmented image can be presented to the surgeon via the microscope eyepiece (or multiple eyepieces) or eyepiece and / or the microscope display, and in some embodiments, can be observed on a monitor screen. This is beneficial for the surgeon to maintain a three-dimensional view of the surgical site through the microscope eyepiece while simultaneously observing the overlaid or adjacent images or information, for example, in a stereoscopic or monocular manner. Real-time 2D or 3D images or models, and real-time treatment information, generated in situ from the data by the imaging device of the probe device 501 can be overlaid onto the live view of one or both eyepieces.In some embodiments, the disclosed apparatus and methods enable real-time observation, including actual and augmented images, from both external and internal perspectives of the anterior chamber during these surgeries.

[0118] The optical microscope 409 can be operably coupled to the imaging device of the probe device 501 in several ways when it is inserted into the eye. The optical microscope 409 may include a binocular microscope, such as a stereomicroscope, which includes an imaging lens element for imaging an object in an eyepiece including an eyepiece 408. The imaging device of the probe device 501, positioned inside, on the surface of, or around the eye, is configured to capture an image of the eye and may include any of the imaging systems described herein, such as those described with respect to Figures 5A to 6C. The optical image is transmitted to the control unit 410 for processing.

[0119] The imaging device of the probe device 501 may include emitters and sensors as described herein. Part of the imaging device may be introduced together with the treatment probe device 501 and moved together with the treatment probe device 501, or the treatment probe device 501 may move independently of the imaging device while maintaining alignment with the treatment probe device 501. The probe device 501 may be configured with a handpiece as described herein so that the probe device 501 can be inserted, manipulated, or withdrawn by a user, actuator, robotic arm, or in other ways.

[0120] In some embodiments, the optical microscope 409 may be coupled to an electronic display device 407. The electronic display 407 may include a heads-up display device (HUD). The HUD may or may not be a component of the microscope system 409. The HUD may be optically coupled to one or both of the field-of-view (FOV) of the eyepiece 408. The display device may be configured to project an augmented image to the user or surgeon from an input 401 generated by the control unit 410. The display device 407 may, alternatively or additionally, be configured to project images captured by a camera and / or imaging device to the user or surgeon. The display device may be coupled to the microscope via one or more optical elements, such as a beam splitter or mirror 420, so that the surgeon looking through the eyepiece 408 can perceive, in addition to the real image, one or more of the following: camera imaging, augmented images, 2D images, 3D images, models generated using data from the imaging device, or any combination represented and presented by the display device 407. The display device may be made observable to the surgeon or user through a single eyepiece. Alternatively, the HUD may be made observable to the surgeon as a stereoscopic binocular image combined with, for example, the optical image formed by the components of the microscope, so that it can be observed through both eyepieces 408.

[0121] In some embodiments, the display device of the head-up display 407 communicates with a control unit 410. The display device can provide the user with an augmented image produced by the control unit 410 in real time. As described herein, real-time imaging includes capturing images or image data and generating 2D or 3D images or models with substantially no latency, enabling the surgeon to perceive a smooth motion flow that matches the tactile movements of the surgeon's surgical instruments during surgery. In some cases, the display device 407 may receive one or more control signals from the control unit 410 to adjust one or more parameters of the display, such as brightness, magnification, and alignment. The image observed by the surgeon or user through the eyepiece or eye lens 408 can be the direct optical view of the eye, the image displayed on the display 407, or a combination of both. Therefore, adjusting the brightness of the image on the HUD may affect the surgeon's view through the eyepiece. For example, processed information and markers shown on the display 407 can be balanced with the microscopic view of the object. The processor can process camera image data, such as by increasing the contrast of the image data to make it easier to detect or identify observable features.

[0122] The head-up display 407 is, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED), a scanning laser display, a CRT, or any similar device known to those skilled in the art.

[0123] Alternatively, or in combination, the display 407 may include an external display. For example, in some embodiments, the display 407 may not be visible through the eyepiece. The display 407 may include a monitor positioned near the optical microscope 409. The display 407 may include, for example, a display screen. The display 407 may include a light-emitting diode (LED) screen, an OLED screen, a liquid crystal display (LCD) screen, a plasma screen, or other types of screens. The display device 407 may or may not include a touchscreen. The surgeon can simultaneously observe a real-time optical image of the surgical site from the display 407 and the imaging provided by the imaging device 702.

[0124] The resolution of the imaging device of the probe apparatus 501 can be configured in many ways using appropriate optical and / or sensor resolutions to image the target tissue with appropriate resolution. The imaging system can produce 2D and 3D images and models with a resolution suitable for observing the tissue structure of the eye as described herein, and may include resolutions in the range of, for example, 1 to 10 microns, or in the range of, for example, about 3 to 6 microns. In some embodiments, the imaging system can produce 2D and 3D images and models and has a certain spatial resolution, for example, an image spatial resolution in the range of about 10 μm to about 80 μm, for the tissue in contact with the inclined distal end of the probe (or in contact with the implant). In some embodiments, the resolution is in the range of about 20 μm to about 40 μm.

[0125] The system 400 may further include a user interface 413. The user interface 413 may be configured to receive user input and provide output information to the user. User input may relate to the control of surgical instruments such as a probe device 501. The user interface 413 can receive input commands relating to the operation of the optical microscope (e.g., microscope settings, camera acquisition, etc.). The user interface 413 can receive indications relating to various operations or settings of the camera. For example, user input may include the selection of a target location, the selection of a treatment criterion marker, the display settings for augmented images, and customizable display settings. The user interface 413 may include a screen such as a touchscreen, and any other user-interactive external devices such as a handheld controller, mouse, joystick, keyboard, trackball, touchpad, buttons, voice commands, gesture recognition, posture sensors, thermal sensors, touch capacitive sensors, foot switches, or any other devices.

[0126] The control unit 410 may be configured to generate an extension layer containing extension information. The extension layer may be a substantially transparent image layer containing one or more graphical elements. The terms “graphical elements” and “graphical visual elements” may be used interchangeably throughout this application. The extension layer may be superimposed on the optical view of a microscope, an optical image, or a video stream, and / or displayed on a display device. In some embodiments, the extension layer is superimposed on the optical view of a microscope so that the transparency of the extension layer allows the user to observe the optical image with the graphical elements superimposed on it. In some embodiments, the extension layer may include real-time images, such as one or more real-time 2D images, 3D images, or models as described herein, acquired by one or more imaging devices of a probe device 501 positioned inside, on the surface of, or near the eye.

[0127] In some embodiments, graphical elements may be configured to change dynamically as the position or orientation of the probe or instrument changes relative to the target location. For example, graphical elements can indicate the location of the distal end of the probe shown in the optical image, or the relative location or spacing between tissues, such as the inner wall of a sclera cortex (SC) or tissue mass (TM). Graphical elements can be configured to dynamically show changes in the spacing between tissue walls, or changes in the distance between the tip and the target location, on the optical image in substantially real-time or near real-time, as the relative distance between the probe tip and the target location changes, and / or the probe tip presses against tissue (e.g., the probe tip contacts the surface of a trabecular meshwork).

[0128] An extension layer, or at least part of its graphical elements, can be mapped or matched to an optical image using object recognition techniques or pattern matching techniques, or a combination of techniques, such as feature point recognition, edge detection, classifiers, spatial pyramid pooling, convolutional neural networks, or any of several suitable object recognition algorithms. Feature points are parts of an image (e.g., scleral landmarks, collecting tube patterns, iris landmarks, etc.) that are uniquely distinguishable from the rest of the image and / or other feature points in the image. Feature points can be detected in parts of an image that are relatively stable even under disturbances (e.g., when the illumination and brightness of the image are changed).

[0129] Referring to Figure 8, an exemplary extended image providing an extended view 600 is shown. As described herein, the extended image 600 can be observed binocularly by a user or surgeon through the eyepiece of a microscope and can be displayed on a head-up display, an external display device, or a display coupled to a user interface. The extended image or extended view may include an optical image 505 or an optical path view through the eyepiece of an optical microscope. The optical image 505 may include a view looking down on the eye from above. The optical image 505 or optical view may show the anterior portion of the eye. The optical image 505 or optical view may further show a portion of an elongated probe device 501. The extended image or extended view 600 may include a plurality of graphical visual elements and / or one or more of 2D images, 3D images, or Model 802 from a probe device 501 as described herein, superimposed on the optical image, for example by optically coupling a display to the optical path of a microscope using a beam splitter. Multiple graphical visual elements may include different shapes and / or colors corresponding to different objects so that different objects shown in the optical image can be easily distinguished from one another. For example, one or more of the 2D image, 3D image, or model may be overlaid with Schlemm tube identification information and location, such as a Schlemm tube identifier. In some embodiments, one or more of the 2D image, 3D image, or model 802 from the probe device 501 includes one or more of the OCT image, ultrasound image, or photoacoustic image as described herein, superimposed on the optical image 505.

[0130] Multiple graphical visual elements may include one or more treatment reference markers 601, 602, 603 mapped to one or more target locations. As discussed elsewhere herein, treatment reference markers 601, 602, 603 may correspond to target locations that are not optically observable to the surgeon in optical images from a surgical microscope. According to some embodiments, the target locations may be located ab interno, and treatment of the target locations may involve ab interno techniques. Alternatively, or in combination, the target locations may be located ab externo, for example, using a femtosecond laser configured to guide laser energy through one or more of the sclera or cornea. Examples of laser light guide systems and femtosecond lasers suitable for integration in accordance with this disclosure are described in U.S. Patent Application No. US14 / 732,627, filed on June 5, 2015, entitled “Methods and apparatuses for the treatment of glaucoma using visible and infrared ultrashort laser pulses,” published on April 7, 2016, as U.S.20160095751, the entire disclosure of which is incorporated herein by reference.

[0131] Multiple graphical visual elements may also include a probe line 604 coaxial with the elongated probe. The probe line 604 indicates the orientation of the probe with respect to one or more target locations. Multiple graphical visual elements may also include a distal tip marker 605 that overlaps with the distal end of the elongated probe. Both the probe line and the distal tip marker may change position dynamically relative to the actual position and orientation of the elongated probe shown in the optical image or optical view 802 as the probe moves within the anterior chamber of the eye. Thus, for example, a surgeon can use a microscope to see where the probe enters the anterior chamber and observe where the probe moves relative to the eye. A detection mechanism can detect the probe, and an automated system or processor can generate the probe line 604 in response to the detection. Similarly, an automated system or processor can generate a guide arrow 612.

[0132] Multiple graphical visual elements may further include one or more guide arrows or markers 612 extending from the distal tip marker 605 toward one or more procedure reference markers (e.g., marker 601). One or more guide arrows 612 may be configured to guide the surgeon to align the distal end of the elongated probe toward one or more target locations during the procedure, and to guide the surgeon to advance the elongated probe toward one or more target locations during the procedure. As discussed elsewhere herein, one or more target locations may not be optically observable to the surgeon in the microscope view 505, and camera imaging is superimposed so that the surgeon can see a real-time image of the distal tip of the probe.

[0133] Figure 9 shows the movement of the probe 560 and the corresponding position 910 of the measurement beam or imaging beam. In some embodiments, the movement of the probe to multiple positions and orientations, which may result from tremors, causes the measurement beam or imaging beam to move to multiple positions and orientations corresponding to the positions and orientations of the probe 560 and probe device 501. The multiple positions may include a first position 912 corresponding to a first position and orientation of the probe 560 and probe device 501, a second position 914 corresponding to a second position and orientation of the probe 560 and probe device 501, a third position 916 corresponding to a third position and orientation of the probe 560 and probe device 501, a fourth position 918 corresponding to a fourth position and orientation of the probe 560 and probe device 501, and a fifth position 918 corresponding to a fifth position and orientation of the probe 560 and probe device 501. In some embodiments, each of the multiple positions corresponds to the measurement location of the beam during a scan, such as an A-scan, and the position and orientation of the probe measured for each of the multiple A-scans. The position and orientation data may include, for example, position and orientation data measured for each of the first axis 522, the second axis 524, and the third axis 526.

[0134] In some embodiments, multiple position and orientation measurements are fitted to a model. Research related to this disclosure suggests that certain types of motion may have periodic components that can be fitted to a model, such as a frequency-domain model. In some embodiments, position and orientation data are fitted to a model, and the position and orientation data are reconstructed according to the model.

[0135] Figure 10 shows a probe device 501 equipped with a sensor 520 that rotates around an opening such as an incision 14. In some embodiments, the probe is inserted through an incision in tissue such as corneal tissue 15, but the probe can be inserted through any suitable opening, such as an opening in tissue or an opening in any material sized to receive the probe 560. In some embodiments, movement of the proximal part of the probe device 501, such as movement of the handpiece including the housing 504, results in a rotation 1010 around the opening 14. In some embodiments, movement of the housing 504 in a first direction 1012 on the first side of the rotation 1010 results in movement in a second direction 1014 opposite to the first direction on the second side of the rotation. For example, translation of the housing 504 along a first axis 522 or a second axis 524 results in opposite movement of the probe tip on the second side of the rotation. In some embodiments, the rotation 1010 includes a two-dimensional rotation around the incision section 14 related to translation along axes 522 and 524, for example, with reference to the translation and rotation of the handpiece including the housing 504. In some embodiments, translation of the probe 560 along axis 526 results in translation of the probe tip along the incision section 14 without rotation. In some embodiments, rotation 527 of the probe 560 about axis 526 results in corresponding rotations of the imaging channel 552 and the treatment channel 554.

[0136] The probe device 501 may include any suitable probe device as described herein, for example with reference to Figures 5A to 6C. In some embodiments, the processor is configured to measure a plurality of measurement beam locations or imaging beam locations 910 in response to a movement around a pivot, such as a first measurement beam location or imaging beam location 912, a second measurement beam location or imaging beam location 914, a third measurement beam location or imaging beam location 916, a fourth measurement beam location or imaging beam location 918, and a fifth measurement beam location or imaging beam location 919, the plurality of measurement beam locations or imaging beam locations move in a second direction 1014 opposite to a first direction of movement 1012 of the handpiece including the housing 504.

[0137] In some embodiments, the processor is operably coupled to the sensor 520 and configured to detect the movement of the probe tip in response to the rotation. In some embodiments, the elongated probe 560 is coupled to the imager 502 and the elongated probe is sized for insertion into the aperture 14. The sensor 520 is configured to measure the orientation and translation of the probe, and the processor is configured to detect the rotation of the elongated probe around the aperture 14. In some embodiments, the processor is configured to determine the position and orientation of a measurement beam or imaging beam from the imaging device 502 in response to the elongated probe rotating around the aperture 14. In some embodiments, the processor is configured to determine the position and orientation of the tip 562 of the probe 560 from sensor data in response to the probe rotating around the aperture 14.

[0138] In some embodiments, the housing 504 includes an elongated probe 560, an imaging device 502, and a handpiece coupled to a sensor 520, and the processor is configured to determine one or more of the position or orientation of the probe tip in response to the movement of the probe tip and the opposite movement of the handpiece.

[0139] Figure 11 shows a probe device 501, which includes an endoscope 1110 in which a probe 560 is configured to determine the location of a measurement beam or imaging beam of a 3D imaging device 502. In some embodiments, an imaging channel 552 includes the endoscope and the 3D imaging device 502. The imaging channel 552 is coupled to one or more components of a 2D imager 1101, such as a sensor array, and a 3D imager 401, as described herein. In some embodiments, images from the endoscope 1110 are used to determine the location of the measurement beam or imaging beam of the 3D imager. In some embodiments, the probe 560 includes a treatment channel 554, as described herein. The probe device 501 can be configured in many ways, but in some embodiments, the probe 560 is dimensioned to fit inside an opening 14 through tissue such as corneal tissue 15. In some embodiments, the probe device 501 is configured to pivot around the opening 14. Alternatively, or in combination, the probe device 501 can be configured to perform measurements without being inserted through the opening 14, for example, in a free-hand configuration.

[0140] The imaging channel 552 of the probe 560 can be configured in various ways using the endoscope 1110, and may include, for example, one or more of adjacent, parallel, overlapping, coaxial, or concentric configurations. The endoscope 1110 of the probe device 501 can be combined with any 3D imaging device as described herein, for example, with reference to Figures 5A to 6C. The endoscope 1110 can be configured in many ways and may include one or more lenses for imaging tissue and an array of optical fibers, such as a regular array of optical fibers. In some embodiments, the endoscope includes a sensor array, for example, on one or more of the probes 560, within the housing 504, or within the console of the treatment station.

[0141] The sensor array of a two-dimensional (2D) imager may include any suitable sensor array capable of generating a 2D image, such as a charge-coupled device (CCD) array, a complementary metal-oxide-semiconductor (CMOS) array, or one or more other 2D sensor arrays as can be understood by those skilled in the art.

[0142] Figure 12 shows a probe device 501 in which the probe 560 includes a treatment channel 554 and an imaging channel including a 3D imager component and an endoscope 1110, and determines the location of the measurement beam or imaging beam of the 3D imager. In some embodiments, the endoscope 1110 includes a sensor array 1118 in which tissue images are formed. The sensor array can be located, for example, in the probe 560, within the housing 504 of the handpiece, or on an external console coupled to the control unit 410. In some embodiments, the sensor array 1118 is coupled to the endoscope 1110 by one or more optical fibers, such as a regular array of optical fibers.

[0143] The treatment channel 554 can be configured in many ways as described herein, but in some embodiments, the treatment channel 554 includes an optical fiber 1252 extending to the distal tip 562 of the probe 560. In some embodiments, the distal tip 562 of the treatment channel 554 extends beyond the distal tip 563 of the imaging channel 552 to image the distal tip 562 of the treatment channel and the tissue simultaneously, as described herein. In some embodiments, the treatment channel 554 includes a treatment axis 1210, such as one or more of a medium for propagating light energy or a tube or elongated element for carrying an implant. The optical fiber 1250 is coupled to a laser 1250. The laser can include any laser configured to emit treatment light energy, such as one or more of ultraviolet, visible, near-infrared, or visible light. In some embodiments, the laser 1250 includes, for example, a xenon chloride excimer laser. In some embodiments, the imaging channel 552 includes one or more optical fibers or components of an ultrasonic imaging probe as described herein. In some embodiments, the imaging channel 552 includes a first channel for a 3D measurement beam or imaging beam and a second channel for an endoscope that performs 2D measurements to determine the position of the measurement beam or imaging beam. The treatment channel 554 and one or more imaging channels can be arranged in any suitable manner, such as a side-by-side configuration or an adjacent configuration, as described herein. In some embodiments, the imaging channel 552 of the imaging device 502 extends along the imaging axis 564, and the endoscope 1110 extends along the endoscope axis 1210. The fields of view of the imaging device 502 and the fields of view of the endoscope 1110 can be configured to overlap so that the position of the measurement beam or imaging beam of the 3D imaging device can be determined as described herein.

[0144] The endoscope 1110 can be configured in many ways, but in some embodiments, the endoscope 1110 includes a first lens 1112, such as a gradient index (GRIN) lens, and a second lens 1116 having a propagation portion 1114 between them. The second lens 1116 may include any suitable lens, such as a camera lens or a GRIN lens. The propagation portion 1114 may include any suitable optical transmission medium, such as air or one or more optical fibers.

[0145] Figure 13A shows a probe including 2D and 3D imaging components, including a treatment channel 554 and optical paths that overlap along the imaging channel 552. In some embodiments, one or more components of the imaging device 502 are optically coupled to one or more components of the endoscope 1110 such that the optical path of the endoscope along axis 1115 overlaps with the optical path of the 3D imaging device 502. The endoscope 1110 can be optically coupled to the 3D imaging device 502 in many ways, but in some embodiments, a beam splitter 1310 is used to couple the endoscope to the 3D imaging device. Alternatively, or in combination, an optical fiber with a coupler can be used to couple the imaging device 502 to the endoscope 1110. The beam splitter 1310 can be positioned at any suitable location on the optical path of the endoscope, such as distal to or proximal to lens 1116, as shown. In some embodiments, in order to reduce parallax in the location of the measurement beam or imaging beam determined from the endoscopic image as described herein, the 3D measurement beam or imaging beam along the imaging axis 564 is substantially coaxial with the optical path of the endoscope along axis 1115.

[0146] Figure 13B shows a probe 560 including a 3D imaging optical fiber 1360 extending along a treatment channel 554 and an imaging channel 552, and a plurality of 2D imaging optical fibers 1350. In some embodiments, the plurality of 2D imaging optical fibers 1350 are arranged around the 3D imaging optical fiber 1360, thereby reducing parallax in the location of the 3D measurement beam or imaging beam determined from the endoscopic image as described herein. In some embodiments, the imaging optical fiber 1360 is positioned near the center of the plurality of imaging optical fibers near the axis 1210 of the endoscope 1115 of the endoscope. The plurality of endoscopic imaging optical fibers 1350 can be arranged in many ways with respect to the 3D imaging optical fiber 1360, for example, in an annular array or hexagonal array extending around the 3D imaging optical fiber 1350.

[0147] In some embodiments, the lens 1112 is configured to form an image of tissue at the distal ends of multiple imaging optical fibers 1350. In some embodiments, the lens 1112 is configured to project a measurement beam or imaging beam from a 3D imaging optical fiber 1360 onto the tissue and to image the light from the tissue at the location of the measurement beam or imaging beam onto the 3D imaging optical fiber 1360. The light collected by the 3D imaging optical fiber is transmitted to a 3D imaging device as described herein.

[0148] In some embodiments, light from multiple imaging optical fibers is supplied to the sensor array of the endoscope, for example, by imaging the proximal ends of the multiple optical fibers onto the sensor array using a second lens, to provide an endoscopic image to the sensor array. The sensor array can be located on the probe 560, within the housing of the handpiece, or within the console of the treatment system. In some embodiments, the multiple imaging optical fibers 1350 extend from the handpiece to the console of a treatment system, such as a laser treatment system, and are coupled to the sensor array with connectors. The multiple optical fibers 1350 may include, for example, a regular array of optical fibers to transmit the image formed at the proximal ends of the fibers to the sensor array.

[0149] The imaging channel and the treatment channel can be configured in various ways. In some embodiments, there is parallax between the axis of the imaging channel and the axis of the treatment channel. In some embodiments, there is parallax between the imaging channel and the treatment channel, and the processor is configured to adjust the constructed 3D image in response to the parallax angle and the distance between the distal end of the imaging channel and the distal end of the treatment channel, such as an implant or treatment fiber. In some embodiments, the processor is configured to adjust the position of the 3D constructed image in response to the distance between the distal end of the 3D imaging channel and the tissue. In some embodiments, the processor is configured to adjust one or more 3D images of the tissue or the distal end of the treatment channel using, for example, an artificial intelligence (AI) algorithm.

[0150] The treatment channel and imaging channel can be positioned on the probe in many ways. In some embodiments, referring to, for example, the imaging probe 25 and treatment probe 23 shown in Figure 3, the 3D imaging channel is positioned on the upper side of the probe and the treatment channel is positioned on the lower side of the probe. Alternatively, the 3D imaging channel can be positioned on the underside of the probe, as shown, for example, with reference to Figures 5A to 6C and Figures 9 to 13B. As will be understood by those skilled in the art, each of these drawings shows the imaging probe on the underside of the probe, but it is also possible to position the imaging probe on top of the treatment probe, as shown in Figure 3. Alternatively, the imaging probe and treatment probe can be positioned laterally relative to each other, such that the elongated axes of the probes are positioned substantially laterally to each other, for example, within 10 degrees horizontally from each other.

[0151] Each of the probes described herein may include a single-use, sterile, disposable probe. In some embodiments, the sterile probe and housing, including the handpiece, are housed in sterile, sealed packaging. In some embodiments, the sensor array includes a single-use sensor array within a housing that includes the handpiece, as described herein. In some embodiments, the sensor array includes, for example, a sterile sensor array, or a sterilizable sensor array, and combinations thereof.

[0152] While probes such as handheld probes are mentioned, the probes described herein can be used in surgical robotic systems, such as surgical robotic systems including robotic arms. Treatment channels, as described herein, can be configured in many ways, for example, for implant placement and creation of openings in the trabecular meshwork, involving one or more of tissue manipulation, incision, or excision. In some embodiments, the treatment channel includes an end effector, such as an end effector of a surgical robotic system configured to manipulate tissue. The end effector may include any suitable end effector, such as a blade or forceps.

[0153] Figure 14A shows an image 1412, such as an endoscopic image, which includes one or more tissue structures and the location 1432 of the measurement beam or imaging beam 1420 on the sensor array 1118 at a first time. In some embodiments, the location of the measurement beam or imaging beam 1420 in the image corresponds to one or more pixels 1435, such as a group of pixels in the image. In some embodiments, one or more pixels 1435 are used as reference locations for the measurement beam or imaging beam. In some embodiments, the sensor array 118 is aligned with the measurement beam or imaging beam such that the location of one or more pixels 1435 corresponds to the location 1432 of the measurement beam or imaging beam. Changes in the position of the tissue result in changes in the position of the measurement beam or imaging beam on the tissue, and these changes can be used to construct a 3D image. One or more tissue structures in the image may include any suitable tissue structure, such as one or more of the following: Schwalbe lines, trabecular meshwork, scleral promontory, ciliary body, iris, or Schlemm's canal. In some embodiments, the first image includes a first location of one or more tissue structures, and the second image includes a second location of one or more tissue structures. In some embodiments, the first image includes one or more of the following: a first location 1452 of the Schwalbe line, a first location 1454 of the trabecular meshwork, a first location 1456 of the scleral promontory, a first location 1458 of the ciliary body, or a first location 1459 of the iris.

[0154] Figure 14B shows a second endoscopic image 1414 including one or more tissue structures and a second location 1434 of the measurement beam or imaging beam 1420 at a second time. The second location 1434 of the measurement beam or imaging beam 1420 and one or more pixels 1435 remain in fixed locations on the sensor array 1118, but the location of one or more tissue structures may change between the first image 1412 and the second image 1414. One or more tissue structures may be offset between the first image and the second image by one or more of the rotations or translations between the first image and the second image. In some embodiments, the tissue structures are shifted between the first image and the second image by a motion vector 1490. In some embodiments, the second image includes one or more of the following locations: the second location of the Schwalbe line 1462, the second location of the trabecular meshwork 1464, the second location of the scleral promontory 1466, the second location of the ciliary body 1468, or the second location of the iris 1469. The locations of one or more tissue structures corresponding to those in the first image are indicated by dashed lines.

[0155] Figure 14C shows the displacement vector 1490 of the image and multiple corresponding locations 910 of the measurement beam or imaging beam. In some embodiments, the displacement vector 1490 includes a first component 1492, such as the X displacement along the tissue, and a second displacement 1494, such as the Y displacement along the tissue. The displacement of the measurement location relative to the tissue is shown by vector 1495, and is generally in the opposite direction to the movement of the tissue shown in the image. In some embodiments, multiple measurements of the 3D imaging beam are taken between the first and second images, and their values ​​are interpolated. For example, if the first image corresponds to the first measurement beam or imaging beam location 912 and the second image corresponds to the fifth measurement beam or imaging beam location 919, the second, third, and fourth measurement beams or imaging beams can be interpolated to intermediate locations along the displacement vector.

[0156] In some embodiments, the processor is configured to receive multiple 2D images of tissue, determine the position of a measurement beam or imaging beam for each of the multiple 2D images, and construct a 3D image in response to each of the multiple 2D images. In some embodiments, the processor is configured to assign the location of the measurement beam or imaging beam for each of a plurality of A scans in response to the multiple 2D images. The sampling rate of the A scans may be faster than the frame rate of the sensor array. In some embodiments, the processor is configured to interpolate the location of the measurement beam or imaging beam for each of the plurality of A scans in response to the multiple 2D images. For example, an A scan imager may be configured to sample a plurality of A scans at a sampling rate at least 100 times faster than the frame rate of the sensor array. In some embodiments, each of the plurality of 2D images includes tissue structure and a response to the location of a measurement beam or imaging beam away from the tissue structure.

[0157] In some embodiments, the processor is configured to detect the user's tremors and construct a tremor model in response to the movement of tissue structures between multiple 2D images.

[0158] Figures 14A to 14C refer to probe movement, but in some embodiments, image movement corresponds to tissue movement as described herein, such as periodic movement. In some embodiments, the tissue movement shown in Figures 14A to 14C is related to pulsatile blood flow, resulting in movement of one or more of Schlemm's canal, collecting ducts, trabecular meshwork, or retina. While references are made to imaging the anterior tissues of the eye, any tissue described herein, such as retinal tissue, can be imaged.

[0159] Figure 15 shows the probe 560 and the corresponding measurement locations 1510 of the measurement beam or imaging beam in response to tissue movement, such as periodic tissue movement as described herein. In some embodiments, the tissue movement results in tissue sampling at multiple locations 1510 of the measurement beam or imaging beam, while the measurement beam or imaging beam remains substantially fixed, as can be caused by tissue movement, such as periodic tissue movement as described herein. For example, the position and orientation of the probe 560 and probe device 501 may remain substantially fixed. Alternatively, or in combination, data on tissue movement can be combined with probe movement data as described herein. The multiple locations of tissue measured may include a first location 1512 corresponding to a first location of tissue, a second location 1514 corresponding to a second location of tissue, a third location 1516 corresponding to a third location of tissue, a fourth location 1518 corresponding to a fourth location of tissue, and a fifth location 1518 corresponding to a fifth location of tissue. In some embodiments, each of the multiple locations corresponds to a measurement location of the beam during a scan, such as an A-scan. The position can be determined by data such as sensor data or motion from endoscopic images. In some embodiments, tissue motion data is combined with probe position and orientation measured for each of a plurality of A scans. Position and orientation data may include, for example, position and orientation data measured for each of the first axis 522, second axis 524, and third axis 526 as described herein.

[0160] In some embodiments, multiple position and orientation measurements are fitted to a model. Research related to this disclosure suggests that certain types of motion may have periodic components that can be fitted to a model, such as a frequency-domain model. In some embodiments, position and orientation data are fitted to a model, and the position and orientation data are reconstructed according to the model.

[0161] Figure 16 shows a method 1600 for imaging tissue.

[0162] In step 1610, image data is acquired using an imager coupled to a sensor configured to measure one or more of the position or orientation of the imager.

[0163] In step 1620, sensor data is acquired from the sensor.

[0164] In step 1630, motion data is acquired. The motion data can be acquired by any suitable method and may include motion data generated or acquired from one or more of sensor data or imager data. The motion data may include periodic motion data, such as periodic motion associated with one or more of tremors, resonances, or pulsating tissues. In some embodiments, the periodic motion corresponds to harmonics that can be determined in response to the motion data. In some embodiments, the periodic motion corresponds to pulsation of tissue such as the choroid beneath the retina. This motion data can be used to construct a 3D image.

[0165] In some embodiments, the periodic motion is determined by a sensor configured to measure the cardiac cycle of the patient's tissues, such as the patient's cardiac cycle. In some embodiments, a processor is configured to construct a 3D image in response to the cardiac cycle. Cardiac signals can be measured in many ways, but in some embodiments, the sensor includes one or more of the following: an electrocardiogram (EKG) sensor, a pulse oximeter, or a blood oxygen sensor.

[0166] In step 1640, images such as 3D images are constructed in response to image data and sensor data.

[0167] Figure 16 shows a method 1600 for imaging tissue according to several embodiments, but those skilled in the art will recognize many adaptable and modified forms. For example, the steps may be performed in any order. Some steps may be omitted, and some steps may be repeated. Some steps may also be combined with sub-steps of other steps.

[0168] Figure 17 shows a femtosecond laser and OCT system 1700 including a sensor. An indirect angle lens 1750 is attached to the sclera 1717, with an internal mirror 1752, by mechanical means or by prongs 1756 or suction. The mirrors may be individual or segmented, fixed or movable, allowing scanning for both observation and treatment of the target. In the case of a movable mirror / mirror surface, the mirror 1752 can be controlled mechanically, pneumatically, or by Mylar-type surface reflection balloons. The mirrors may be planar, concave, convex, and single or segmented arrays.

[0169] In some embodiments, the system 1700 includes multiple laser beams, such as a first laser beam 1792 and a second laser beam 1794. The laser beams may include visible laser beams, such as red laser beams from one or more red laser diodes. The laser beams may be configured to substantially overlap in the tissue being treated, such as in the target tissue, such as the trabecular meshwork or Schlemm's canal, so that the beams appear as a single beam. If the tissue is not positioned at the correct distance from the optical path, the beam overlap may decrease, and the beams may appear as separate spots on the target tissue.

[0170] System 1700 may include additional components such as processors and arrays, as described herein with reference to Figure 7, for example. For example, System 1700 may include a surgical microscope, a display, a 3D imager, a control unit 400, and a femtosecond laser unit.

[0171] Examples of laser light guide systems and femtosecond lasers suitable for integration in accordance with this disclosure are described in U.S. Patent Application No. US14 / 732,627, filed on June 5, 2015, entitled “Methods and apparatuses for the treatment of glaucoma using visible and infrared ultrashort laser pulses,” which was published on April 7, 2016, as U.S.20160095751, and whose entire disclosure is already incorporated herein by reference.

[0172] Referring again to Figure 7, an imaging device coupled to a probe device 501, as described herein, in combination with the optical microscope 409, can provide data to the 3D imager 401 and control unit 410, such as the overlap of beams 1792 and 1794 on the trabecular meshwork. In some embodiments, a second camera 416, including a detector array, is optically coupled to the surgical microscope 409 and optically coupled to a processor 414 of the control unit 410 to receive optical images from the surgical microscope 409 and to determine beam overlap. The control unit 410 receives image data from the camera 416, processes the image data, and provides visual image data to a head-up display 407, which can superimpose visual image data onto the forward optical image of the surgical microscope 409, such as a front image showing beams 1792 and 1794, which may overlap in target tissue such as the trabecular meshwork.

[0173] Referring again to Figures 14A and 14B, the tissue image and the beams 1792 and 1794 illuminating the tissue appear on a two-dimensional sensor array, similar to the two-dimensional endoscope array 1118 shown in Figure 14. In some embodiments, the two-dimensional sensor array includes a two-dimensional sensor array of camera 416. Using the surgical microscope and the beam separation distance, the position of the tissue along the optical path of the imaging beam, and in some embodiments, along beam 1751, can be determined. To determine the position of the tissue in three dimensions, along the optical axis of the measurement beam and orthogonal to the measurement beam, as described herein with reference to, for example, Figures 14A, 14B, and 17, one or more pixels 1435 shown in Figures 14A and 14B can be used to determine the position of the measurement beam, and the separation distance between the first beam 1792 and the second beam 1794 can be used to determine the position of tissue, such as the trabecular meshwork, along the optical axis.

[0174] In some embodiments, the processor is configured to image tissue using multiple scanning A scans and to measure the separation distance for each of the A scans, and to construct an image in response to each of the A scans and the separation distance for each of the A scans. In some embodiments, the processor is configured to construct an image in response to each of the A scans, multiple separation distances, and the lateral position of the tissue, as described herein, for example, with reference to Figures 9, 10, 14A, and 14B.

[0175] In some embodiments, the pulsed radiation beam 1751 is generated by a femtosecond laser and guided to the eye by a light guide system including an angle lens 1750. In some embodiments, the beam 1751 includes a bidirectional beam in which the OCT beam is guided to the tissue and the OCT beam receives light from the tissue to which it is focused. In some embodiments, the OCT beam has sufficient resolution to measure the separation distance between the inner wall and the outer wall of the Schlemm tube.

[0176] In some embodiments, beam 1751 comprises multiple beams. In some embodiments, beam 1751 comprises a femtosecond laser beam and an imaging beam, as described herein. Alternatively, the imaging beam may include a separate beam directed to the tissue. The imaging beam may include any imaging beam, as described herein. In some embodiments, the system's processor is configured to image the tissue by scanning multiple A-scans of the imaging beam along the tissue, such as the trabecular meshwork.

[0177] In some embodiments, beam 1751 includes multiple beams, such as an OCT imaging beam and a treatment laser beam, such as a bidirectional laser beam. Although shown substantially coaxially, in some embodiments, the treatment beam and imaging beam are offset from each other as they approach the eye, as are the first laser beam 1792 and the second laser beam 1794.

[0178] Beam 1751 is reflected by mirror 1752, which can be controlled by a servo system 1753 connected to controller 1758, to concentrate scanning light-disruption energy onto the curved surface of the target tissue. This optical system is bidirectional, with one direction used to treat the target tissue and the other direction used to observe and / or sense the x, y, z coordinates of the target tissue, enabling precise treatment and removal of the target area. Beam 1751 has a set of pulse parameter ranges specifically selected to photodisrupt the target tissue of the trabecular meshwork while minimizing damage to surrounding tissue. Thus, the beam has wavelengths between 0.4 and 2.5 microns. The precise wavelength to use for a particular target site depends on the trade-off between strong absorption by the trabecular meshwork and transmission of the eye structure and aqueous humor in front of it.

[0179] In some embodiments, the indirect gonioscopy lens 1750 is bonded to the cornea 1715 by suction or to an internal mirror 1752. In some embodiments, the gonioscopy lens is bonded to the sclera 1717 by suction 1757 or mechanically to a mirror system 1752 located outside the gonioscopy lens.

[0180] In some embodiments, the pulse duration of the laser beam is selected to maximize the probability of photodisrupting material in the corneal-scleral keratin outflow tissue. In some embodiments, there is an inverse relationship between the laser pulse duration and the energy required to produce optical disruption with each pulse. In some embodiments, the pulse duration is selected to be shorter than the thermal relaxation of the target, so that only the target material is heated and the surrounding tissue is unaffected. Thus, the pulse duration is between 20 fs and 300 ps. The pulse rate is between 1 and 500 kHz.

[0181] In some embodiments, the pulse energy is selected to promote photodestruction and minimize the effects of the laser light shock wave. Typical pulse energy values ​​are between 300 and 1500 nJ.

[0182] In some embodiments, the spot diameter is selected to achieve a laser energy density sufficient to facilitate the photodestruction of one or more tissues, such as one or more trabecular meshworks or circumferential trabecular meshworks. In some embodiments, the spot size is between 1 and 10 microns.

[0183] In some embodiments, the gonioscopy lens 1750 is anchored to either the sclera 17 or the cornea 15 by, for example, a suction ring 57 or prongs 56. The anchoring system is attached to a pressure adjustment system 55 and an ocular pulse detection system 54. The anchoring system is either concentric 57 or segmented 56. Scanning the spot in the x, y, and z directions produces a pattern of tissue removal. Although a gonioscopy lens is mentioned, the lens coupled to the eye can include any suitable lens shaped to accept the cornea, and the lens may include one or more components of the patient interface. The mirror can be coupled to the lens in many ways. In some embodiments, the lens includes an external mirror positioned outside the gonioscopy lens. Alternatively, or in combination, the lens may include an internal mirror positioned inside the gonioscopy lens. Examples of appropriate lens and mirror combinations are described in Figures 4, 5, and 6 of U.S. Patent Application No. US14 / 732,627, filed on June 5, 2015, entitled “Methods and apparatuses for the treatment of glaucoma using visible and infrared ultrashort laser pulses,” which was published on April 7, 2016, as U.S.20160095751, and the entire disclosure is already incorporated herein by reference.

[0184] While the detection of the cardiac cycle by EKG, pulse oximeter, or oxygen sensor is mentioned, in some embodiments the OCT system has sufficient resolution to detect tissue pulsations, such as dilation and stenosis of Schlemm's canal. In some embodiments, the OCT system has sufficient resolution to determine changes in the distance between the inner and outer walls of Schlemm's canal using multiple A-scans, as described herein. In some embodiments, the distance across Schlemm's canal increases and decreases in response to the cardiac cycle, and the OCT system is configured to measure the change in distance using, for example, multiple A-scans.

[0185] As detailed above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions, including those contained within the modules described herein. In their most basic configuration, each of these computing devices may include at least one memory device and at least one physical processor.

[0186] As used herein, the terms “memory” or “memory device” generally refer to any type or form of volatile or non-volatile storage device or storage medium capable of storing data and / or computer-readable instructions. In one example, a memory device may store, load, and / or maintain one or more of the modules described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drives (HDD), solid-state drives (SSD), optical disk drives, caches, variations or combinations of one or more of these, or any other suitable storage memory.

[0187] In addition, as used herein, the terms “processor” or “physical processor” generally refer to any type or form of hardware implementation processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor may access and / or modify one or more modules stored in the memory devices described above. Examples of physical processors include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), parts of one or more of these, variations or combinations of one or more of these, or any other suitable physical processor.

[0188] The method steps described and / or illustrated herein, although shown as separate elements, may also represent part of a single application. In addition, in some embodiments, one or more of these steps may represent, or correspond to, one or more software applications or programs that, when executed by a computing device, cause that computing device to perform one or more tasks, such as the method steps.

[0189] In addition, one or more of the devices described herein can convert data, physical devices, and / or representations of physical devices from one form to another. For example, one or more of the devices described herein can receive image data of a sample to be converted, convert the image data, output the result of the conversion to determine a process, execute the process using the result of the conversion, and store the result of the conversion to produce an output image of the sample. In addition, or / or, one or more of the modules described herein can convert processors, volatile memory, non-volatile memory, and / or any other parts of a physical computing device from one form of computing device to another form of computing device by running on a computing device, storing data in a computing device, and / or otherwise interacting with a computing device.

[0190] As used herein, the term “computer-readable medium” generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable mediums include, but are not limited to, transmission-type media such as carrier waves, and non-transient-type media such as magnetic storage media (e.g., hard disk drives, tape drives, floppy disks), optical storage media (e.g., compact discs (CDs), digital video discs (DVDs), and Blu-ray discs), electronic storage media (e.g., solid-state drives and flash media), and other distribution systems.

[0191] Those skilled in the art will recognize that any process or method disclosed herein can be modified in many ways. The order of process parameters and steps described and / or illustrated herein is given only as an example and can be changed as desired. For example, the steps illustrated and / or described herein may be shown or considered in a particular order, but these steps do not necessarily have to be performed in the order shown or considered.

[0192] The various exemplary methods described and / or illustrated herein may also omit one or more of the steps described or illustrated herein, or may include additional steps in addition to the disclosed steps. Furthermore, any step of any of the methods disclosed herein may be combined with any one or more steps of any of the other methods disclosed herein.

[0193] A processor as described herein may be configured to perform one or more steps of any of the methods described herein.

[0194] Unless otherwise stated, the terms “connected” and “combined” (and their derivatives) as used in the specification and claims shall be construed to allow both direct and indirect (i.e., through other elements or components) connection. In addition, as used in the specification and claims, the terms “a” or “an” shall be construed to mean “at least one of.” Finally, for ease of use, the terms “including” and “having” (and their derivatives) as used in the specification and claims shall be interchangeable with and have the same meaning as “comprising.”

[0195] A processor as disclosed herein may consist of instructions for performing one or more steps of any of the methods disclosed herein.

[0196] In this specification, terms such as “first,” “second,” and “third” may be used to describe various layers, elements, components, areas, or sections without referring to a specific order or sequence of events. These terms are used solely to distinguish one layer, element, component, area, or section from another. A first layer, element, component, area, or section as described herein may be referred to as a second layer, element, component, area, or section without departing from the teachings of this disclosure.

[0197] As used herein, the term "or" is used to comprehensively refer to items as alternatives and combinations.

[0198] Where used herein, symbols such as numbers refer to similar elements.

[0199] This disclosure includes the following clauses.

[0200] Clause 1. An apparatus for imaging tissue, comprising: an imager configured to generate an imaging beam for imaging tissue; a sensor coupled to the imager for acquiring sensor data relating to one or more of the position or orientation of the imaging beam; and a processor coupled to the imager and the sensor for acquiring image data and sensor data, the processor comprising instructions for constructing a 3D image of the tissue in response to the image data and sensor data.

[0201] Clause 2. The apparatus described in Clause 1, wherein the imaging beam includes one or more of the following: an optical imaging beam, an optical coherence tomography (OCT) imaging beam, an ultrasound (US) imaging beam, or a photoacoustic imaging beam.

[0202] Clause 3. The apparatus according to Clause 1 or 2, wherein the imager includes a sensor configured to generate motion data, and the processor configured to reconstruct a 3D image in response to the motion data.

[0203] Clause 4. The apparatus according to any one of Clauses 1 to 3, wherein the optical imager is a two-dimensional imager including a two-dimensional sensor array, and the two-dimensional imager includes one or more of an endoscope, a microscope, a stereomicroscope, and a stereoendoscopy.

[0204] Clause 5. An apparatus according to any one of Clauses 1 to 4, wherein the imager includes an imaging axis and emits a beam along the axis, and the sensor includes multiple axes for measuring one or more of the position or orientation of the beam of imaging energy.

[0205] Clause 6. The apparatus according to any one of Clauses 1 to 5, wherein the beam axis is aligned with at least one of a plurality of axes for measuring one or more of the imaging energy position beam or orientation beam.

[0206] Clause 7. The apparatus described in any one of Clauses 1 to 6, wherein the sensor is configured to measure the motion of an imager in a direction along the axis of the beam, corresponding to the motion of the imaging beam along the axis, and the processor is configured to construct a 3D image in response to the motion of the imaging beam along the axis.

[0207] Clause 8. The apparatus described in any one of Clauses 1 to 7, wherein the sensor is configured to measure the motion of an imager perpendicular to the axis of the imaging beam, corresponding to the motion of the imaging beam perpendicular to the axis, and the processor is configured to construct a 3D image in response to the motion of the imaging beam perpendicular to the axis.

[0208] Clause 9. The apparatus according to any one of Clauses 1 to 8, wherein the motion of an imaging beam perpendicular to the axis corresponds to the motion of an imaging beam in a first direction perpendicular to the axis and a second direction perpendicular to the axis, and the processor is configured to construct a 3D image in response to the motion of the imaging beam in the first and second directions.

[0209] Clause 10. An apparatus as described in any one of Clauses 1 to 9, wherein the motion of the imager includes rotation of the imager corresponding to the motion of the imaging beam.

[0210] Clause 11. An apparatus as described in any one of Clauses 1 to 10, wherein the motion of the imager includes the translation of the imager corresponding to the motion of the imaging beam.

[0211] Clause 12. The apparatus described in any one of Clauses 1 to 11, wherein a first direction perpendicular to the axis and a second direction perpendicular to the axis define a plane perpendicular to the axis, and optionally the plane extends substantially perpendicular to the axis.

[0212] Clause 13. The apparatus according to any one of Clauses 1 to 12, wherein the processor is configured to store multiple positions and orientations of the imager while a beam of imaging energy is directed into the tissue, and to construct a 3D image in response to the multiple positions and orientations.

[0213] Clause 14. The apparatus described in any one of Clauses 1 to 13, wherein the processor is configured to determine multiple locations of the imaging beam from sensor data, and the processor is configured to construct a 3D image from multiple positions and orientations of the sensor.

[0214] Clause 15. The apparatus according to any one of Clauses 1 to 14, comprising an elongated probe coupled to an imager, the elongated probe further comprising an elongated probe that is dimensioned for insertion into an opening, and a sensor configured to measure the orientation and translation of the probe.

[0215] Clause 16. The apparatus according to any one of Clauses 1 to 15, further comprising a handpiece coupled to an elongated probe, an imager, and a sensor, wherein a processor is configured to determine one or more of the positions or orientations of the probe tip in response to the movement of the handpiece in the same direction as the movement of the probe tip.

[0216] Clause 17. The apparatus according to any one of Clauses 1 to 16, further comprising a second probe, wherein the second probe includes an elongated treatment probe, and optionally the elongated treatment probe is not coupled to the elongated probe coupled to the imager and sensor.

[0217] Clause 18. An apparatus according to any one of Clauses 1 to 17, comprising an elongated probe coupled to an imager, the elongated probe further comprising an elongated probe that is sized for insertion into an opening, the sensor configured to measure the orientation and translation of the probe, and the processor configured to detect the rotation of the elongated probe about an opening.

[0218] Clause 19. The apparatus according to any one of Clauses 1 to 18, wherein the processor is configured to determine the position and orientation of an imaging beam from an imager in response to an elongated probe that revolves around an aperture.

[0219] Clause 20. The apparatus according to any one of Clauses 1 to 19, wherein the processor is configured to determine the position and orientation of the probe tip from sensor data in response to the probe rotating around an opening.

[0220] Clause 21. The apparatus according to any one of Clauses 1 to 20, further comprising an elongated probe, an imager, and a handpiece coupled to a sensor, wherein a processor is configured to determine one or more of the positions or orientations of the probe tip in response to the movement of the handpiece opposite to the movement of the probe tip.

[0221] Clause 22. The apparatus described in any one of Clauses 1 to 21, wherein the sensor includes a sensor array of an endoscope or microscope, and optionally the sensor array includes a two-dimensional sensor array.

[0222] Clause 23. The apparatus according to any one of Clauses 1 to 22, wherein the endoscope is configured to generate a two-dimensional (2D) image of tissue using a sensor array, and the processor is configured to determine the position of the imaging beam in response to the 2D image of tissue.

[0223] Clause 24. The apparatus according to any one of Clauses 1 to 23, wherein the processor is configured to receive a plurality of 2D images of an organization, determine the position of an imaging beam for each of the plurality of 2D images, and construct a 3D image in response to each of the plurality of 2D images.

[0224] Clause 25. The apparatus according to any one of Clauses 1 to 24, wherein the processor is configured to construct a 3D image in response to the movement of tissue structures between a plurality of 2D images.

[0225] Clause 26. The apparatus according to any one of Clauses 1 to 25, wherein the imaging beam in each of the multiple images is located away from the tissue structure, the processor is configured to construct a 3D image, and the processor is configured to assign the imaging beam location for each of the multiple A scans in response to the multiple 2D images.

[0226] Clause 27. The apparatus according to any one of Clauses 1 to 26, wherein the processor is configured to interpolate the location of the imaging beam for each of a plurality of A scans in response to a plurality of 2D images.

[0227] Clause 28. The apparatus described in any one of Clauses 1 to 27, wherein the imager can be configured to sample multiple A scans at a sampling rate at least 100 times faster than the frame rate of the sensor array.

[0228] Clause 29. The apparatus according to any one of Clauses 1 to 28, wherein each of the plurality of 2D images includes tissue structure and a response to the position of the imaging beam away from the tissue structure.

[0229] Clause 30. The apparatus described in any one of Clauses 1 to 29, wherein the processor is configured to detect the movement of one or more of a user, a probe, a robotic arm, or tissue, and to construct a motion model in response to the movement of tissue structures between multiple 2D images.

[0230] Clause 31. An apparatus described in any one of Clauses 1 to 30, wherein the motion model corresponds to the periodic motion of one or more of the user, probe, robotic arm, or tissue.

[0231] Clause 32. The apparatus according to any one of Clauses 1 to 31, wherein the processor is configured to detect user tremors and construct a tremor model in response to the movement of tissue structures between multiple 2D images.

[0232] Clause 33. The apparatus according to any one of Clauses 1 to 32, wherein the processor is configured to detect tissue movement and construct a tissue motion model in response to the movement of tissue structures between multiple 2D images.

[0233] Clause 34. A device described in any one of Clauses 1 to 33, wherein the motion model corresponds to the movement of the eye tissue in response to the patient's pulsating blood flow.

[0234] Clause 35. The apparatus described in any one of Clauses 1 to 34, wherein the processor is configured to detect the resonance mode of a robotic arm and to construct a resonance model in response to the movement of tissue structures between multiple 2D images.

[0235] Clause 36. The apparatus described in any one of Clauses 1 to 35, wherein the tissue structure includes one or more of the Schwalbe lines, ciliary zone, scleral promontory, Schlemm's canal, trabecular meshwork, or iris of the eye.

[0236] Clause 37. The apparatus described in any one of Clauses 1 to 36, wherein an imaging beam is aligned with one or more pixels of a sensor array, and a processor is configured to construct a 3D image in response to the location of one or more pixels in a plurality of 2D images.

[0237] Clause 38. An apparatus according to any one of Clauses 1 to 37, wherein one or more pixels are aligned coaxially with the imaging beam.

[0238] Clause 39. An apparatus as described in any one of Clauses 1 to 38, wherein one or more pixels correspond to a reference location of the imaging beam.

[0239] Clause 40. The apparatus described in any one of Clauses 1 to 39, wherein the imager is configured to generate a plurality of A scans using an imaging beam, and the processor is configured to determine the position of the imaging beam for each of the plurality of A scans in response to sensor data, and to construct an image in response to the plurality of positions of the imaging beam.

[0240] Clause 41. The apparatus described in any one of Clauses 1 to 40, further comprising an imaging optical fiber, wherein the imager includes one or more of OCT imaging or photoacoustic imaging, and the imaging beam is guided along the optical fiber.

[0241] Clause 42. The apparatus according to any one of Clauses 1 to 41, further comprising an acoustic sensor located near the distal end of an optical fiber, wherein the imager includes a photoacoustic imager, and the sensor is configured to receive acoustic pulses in response to an imaging beam from an optical fiber that illuminates tissue and generates acoustic pulses.

[0242] Clause 43. The apparatus described in any one of Clauses 1 to 42, further comprising a scanner for scanning an imaging beam.

[0243] Clause 44. The apparatus described in any one of Clauses 1 to 43, wherein the scanner is configured to deflect the distal end of an imaging optical fiber to scan the imaging beam.

[0244] Clause 45. The apparatus described in any one of Clauses 1 to 44, further comprising an ultrasonic transducer for generating multiple A scans.

[0245] Clause 46. The apparatus described in any one of Clauses 1 to 45, wherein the optical imager includes one or more of an endoscope, microscope, stereomicroscope, or stereoendoscopy, and the processor is configured to generate motion data from multiple images from the optical imager, and the processor is configured to generate a 3D image of tissue in response to the motion data and the multiple images.

[0246] Clause 47. The apparatus described in any one of Clauses 1 to 46, wherein the imager includes an ultrasonic (US) imager, and the US imager includes one or more arrays.

[0247] Clause 48. The apparatus according to any one of Clauses 1 to 47, further comprising a treatment probe for treating tissue, wherein the treatment probe is coupled to a sensor and an imager.

[0248] Clause 49. The apparatus described in any one of Clauses 1 to 48, wherein the treatment probe includes a treatment channel for treating tissue.

[0249] Clause 50. The apparatus described in any one of Clauses 1 to 49, wherein the treatment channel is configured to perform one or more of the following: carrying an implant, guiding treatment energy, or manipulating tissue.

[0250] Clause 51. The apparatus described in any one of Clauses 1 to 50, wherein the treatment channel includes an optical fiber for guiding laser energy into the tissue.

[0251] Clause 52. The apparatus described in any one of Clauses 1 to 51, wherein the treatment channel includes a working channel for delivering the implant to the tissue.

[0252] Clause 53. The apparatus according to any one of Clauses 1 to 52, further comprising a handpiece coupled to an imager and a sensor, wherein a processor is configured to detect user tremors and construct a tremor model in response to sensor data.

[0253] Clause 54. The apparatus described in any one of Clauses 1 to 53, wherein the processor is configured to determine the position of an imaging beam within tissue in response to sensor data and a tremor model.

[0254] Clause 55. An apparatus described in any one of Clauses 1 to 54, wherein the tremor corresponds to the periodic movement of the handpiece.

[0255] Clause 56. An apparatus according to any one of Clauses 1 to 55, wherein the imager is configured to emit an imaging beam, and the processor is configured to determine the location of the imaging beam in response to a tremor detector equipped with a sensor.

[0256] Clause 57. The apparatus according to any one of Clauses 1 to 56, further comprising a sensor for measuring the cardiac cycle of tissue, wherein a processor is configured to construct a 3D image in response to the cardiac cycle.

[0257] Clause 58. The apparatus described in any one of Clauses 1 to 57, wherein the sensor includes one or more of the following: an electrocardiogram (EKG) sensor, a pulse oximeter, or a blood oxygen sensor.

[0258] Clause 59. The apparatus described in any one of Clauses 1 to 58, further comprising multiple visible laser beams for measuring the position of tissue along the optical path of an imaging beam.

[0259] Clause 60. The apparatus according to any one of Clauses 1 to 59, wherein the sensor includes a two-dimensional sensor array, and the processor is configured to determine the position of tissue along the optical path in response to the separation distance between a plurality of visible laser beams on the sensor array.

[0260] Clause 61. The apparatus according to any one of Clauses 1 to 60, wherein the processor is configured to construct a 3D image in response to the separation distance between a first laser beam and a second laser beam.

[0261] Clause 62. The apparatus according to any one of Clauses 1 to 61, wherein the processor is configured to image tissue with multiple OCT A scans and to measure the separation distance for each of the multiple A scans.

[0262] Clause 63. The apparatus according to any one of Clauses 1 to 62, wherein the processor is configured to construct a 3D image in response to the separation distance between a first measurement beam and a second measurement beam for each of a plurality of A scans.

[0263] Clause 64. The apparatus according to any one of Clauses 1 to 63, wherein the processor is configured to determine the lateral position of a measurement beam on tissue in response to an image of tissue on a two-dimensional sensor array.

[0264] Clause 65. The apparatus described in any one of Clauses 1 to 64, wherein one or more pixels of the two-dimensional sensor array correspond to the location of the measurement beam in the image.

[0265] Clause 66. The apparatus according to any one of Clauses 1 to 65, wherein the processor is configured to determine the three-dimensional position of the measurement beam on tissue in response to the separation distance between the first beam and the second beam, and the separation distance between the first measurement beam and the second measurement beam.

[0266] Clause 67. The apparatus described in any one of Clauses 1 to 66, further comprising a treatment laser, wherein the treatment laser includes one or more of ultraviolet lasers, femtosecond lasers, visible lasers, or infrared lasers.

[0267] Clause 68. A method for imaging tissue, comprising: acquiring image data using an imager coupled to a sensor configured to measure one or more of the position or orientation of the imager; acquiring sensor data from the sensor together with the image data; and constructing a 3D image of the tissue in response to the image data and sensor data.

[0268] Clause 69. An apparatus for imaging tissue, comprising an imager for imaging tissue using one or more of the following: an optical imager, an optical coherence tomography (OCT) imager, an ultrasound (US) imager, or a photoacoustic imager; and a processor coupled to the imager, the processor comprising a motion model and instructions for constructing a 3D image of the tissue in response to imager data and the motion model.

[0269] Apparatus or method according to any one of Clauses 1 to 69, wherein the D imaging channel is positioned proximal to the distal end of the treatment channel to observe one or more implants, optical fibers, tissue manipulators, or end effectors positioned at the distal end of the treatment channel.

[0270] The embodiments of this disclosure are shown and described herein and are provided for illustrative purposes only. Those skilled in the art will recognize numerous adaptable, modified, altered, and substitute forms without departing from the scope of this disclosure. Multiple alternative forms and combinations of the embodiments disclosed herein can be utilized without departing from the scope of this disclosure and the inventions disclosed herein. Accordingly, the scope of the inventions currently disclosed is defined solely by the scope of the appended claims and their equivalents.

Claims

1. A device for imaging tissue, An imager configured to generate an imaging beam for imaging the aforementioned tissue, A sensor coupled to the imager to acquire sensor data related to one or more of the position or orientation of the imaging beam, A processor coupled to the imager and the sensor for acquiring image data and sensor data, wherein the processor comprises instructions for constructing a 3D image of the tissue in response to the image data and the sensor data. A device equipped with the following features.

2. The apparatus according to claim 1, wherein the imaging beam includes one or more of the following: an optical imaging beam, an optical coherence tomography (OCT) imaging beam, an ultrasonic (US) imaging beam, or a photoacoustic imaging beam.

3. The apparatus according to claim 1, wherein the imager includes the sensor, the sensor is configured to generate motion data, and the processor is configured to reconstruct the 3D image in response to the motion data.

4. The apparatus according to claim 1, wherein the optical imager is a two-dimensional imager including a two-dimensional sensor array, and the two-dimensional imager includes one or more of an endoscope, a microscope, a stereomicroscope, and a stereoendoscopy.

5. The apparatus according to claim 1, wherein the imager includes an imaging axis and emits the beam along the axis, and the sensor includes a plurality of axes for measuring one or more of the position or orientation of the beam of imaging energy.

6. The apparatus according to claim 5, wherein the axis of the beam is aligned with at least one of the plurality of axes for measuring one or more of the position beam or orientation beam of the imaging energy.

7. The apparatus according to claim 5, wherein the sensor is configured to measure the movement of the imager in a direction along the axis of the beam corresponding to the movement of the imaging beam along the axis, and the processor is configured to construct the 3D image in response to the movement of the imaging beam along the axis.

8. The apparatus according to claim 5, wherein the sensor is configured to measure the movement of the imager perpendicular to the axis of the imaging beam, corresponding to the movement of the imaging beam perpendicular to the axis, and the processor is configured to construct the 3D image in response to the movement of the imaging beam perpendicular to the axis.

9. The apparatus according to claim 8, wherein the movement of the imaging beam perpendicular to the axis corresponds to the movement of the imaging beam in a first direction perpendicular to the axis and a second direction perpendicular to the axis, and the processor is configured to construct the 3D image in response to the movement of the imaging beam in the first direction and the second direction.

10. The apparatus according to claim 9, wherein the motion of the imager includes rotation of the imager corresponding to the motion of the imaging beam.

11. The apparatus according to claim 9, wherein the movement of the imager includes translation of the imager corresponding to the movement of the imaging beam.

12. The apparatus according to claim 9, wherein the first direction perpendicular to the axis and the second direction perpendicular to the axis define a plane perpendicular to the axis, and optionally the plane extends substantially perpendicular to the axis.

13. The apparatus according to claim 5, wherein the processor is configured to store a plurality of positions and orientations of the imager while the beam of imaging energy is guided to the tissue, and to construct the 3D image in response to the plurality of positions and orientations.

14. The apparatus according to claim 13, wherein the processor is configured to determine a plurality of locations of the imaging beam from the sensor data, and the processor is configured to construct the 3D image from the plurality of positions and orientations of the sensor.

15. An elongated probe coupled to the imager, wherein the elongated probe is sized for insertion into an opening. Furthermore, The apparatus according to claim 1, wherein the sensor is configured to measure the orientation of the probe and the translation of the probe.

16. The elongated probe, the imager, and the sensor are combined into a handpiece. Furthermore, The apparatus according to claim 15, wherein the processor is configured to determine one or more positions or orientations of the tip of the probe in response to the movement of the handpiece in the same direction as the movement of the tip of the probe.

17. The apparatus according to claim 15, further comprising a second probe, wherein the second probe includes an elongated treatment probe, and optionally the elongated treatment probe is not coupled to the elongated probe coupled to the imager and the sensor.

18. An elongated probe coupled to the imager, wherein the elongated probe is sized for insertion into an opening. Furthermore, The sensor is configured to measure the orientation and translation of the probe. The apparatus according to claim 1, wherein the processor is configured to detect the rotation of the elongated probe around the opening.

19. The apparatus according to claim 18, wherein the processor is configured to determine the position and orientation of the imaging beam from the imager in response to the elongated probe that rotates about the aperture.

20. The apparatus according to claim 18, wherein the processor is configured to determine the position and orientation of the tip of the probe from sensor data in response to the probe rotating about the opening.

21. The elongated probe, the imager, and the sensor are combined into a handpiece. Furthermore, The apparatus according to claim 18, wherein the processor is configured to determine one or more positions or orientations of the tip of the probe in response to the movement of the handpiece opposite to the movement of the tip of the probe.

22. The apparatus according to claim 1, wherein the sensor includes a sensor array of an endoscope or microscope, and optionally the sensor array includes a two-dimensional sensor array.

23. The apparatus according to claim 22, wherein the endoscope is configured to generate a two-dimensional (2D) image of the tissue using the sensor array, and the processor is configured to determine the position of the imaging beam in response to the 2D image of the tissue.

24. The apparatus according to claim 23, wherein the processor is configured to receive a plurality of 2D images of the tissue, determine the position of the imaging beam for each of the plurality of 2D images, and construct the 3D image in response to each of the plurality of 2D images.

25. The apparatus according to claim 24, wherein the processor is configured to construct the 3D image in response to the movement of the tissue structure between the plurality of 2D images.

26. The apparatus according to claim 25, wherein the position of the imaging beam in each of the plurality of images is located away from the tissue structure, the processor is configured to construct the 3D image, and the processor is configured to assign the position of the imaging beam in each of the plurality of A scans in response to the plurality of 2D images.

27. The apparatus according to claim 25, wherein the processor is configured to interpolate the location of the imaging beam for each of the plurality of A scans in response to the plurality of 2D images.

28. The apparatus according to claim 27, wherein the imager can be configured to sample the plurality of A scans at a sampling rate at least 100 times faster than the frame rate of the sensor array.

29. The apparatus according to claim 23, wherein each of the plurality of 2D images includes a tissue structure and a response of the imaging beam to the position away from the tissue structure.

30. The apparatus according to claim 25, wherein the processor is configured to detect the movement of one or more of the user, probe, robot arm, or tissue, and to construct a motion model in response to the movement of the tissue structure between the plurality of 2D images.

31. The apparatus according to claim 30, wherein the motion model corresponds to the periodic motion of one or more of the user, the probe, the robot arm, or the tissue.

32. The apparatus according to claim 30, wherein the processor is configured to detect user tremors and construct a tremor model in response to the movement of the tissue structure between the plurality of 2D images.

33. The apparatus according to claim 30, wherein the processor is configured to detect tissue movement and construct a tissue movement model in response to the movement of the tissue structure between the plurality of 2D images.

34. The apparatus according to claim 33, wherein the motion model corresponds to the movement of eye tissue in response to the patient's pulsating blood flow.

35. The apparatus according to claim 25, wherein the processor is configured to detect the resonance mode of the robot arm and to construct a resonance model in response to the movement of the tissue structure between the plurality of 2D images.

36. The apparatus according to claim 25, wherein the tissue structure includes one or more of the Schwalbe lines, ciliary zone, scleral promontory, Schlemm's canal, trabecular meshwork, or iris of the eye.

37. The apparatus according to claim 1, wherein the imaging beam is aligned with one or more pixels of the sensor array, and the processor is configured to construct the 3D image in response to the location of the one or more pixels in a plurality of 2D images.

38. The apparatus according to claim 37, wherein one or more pixels are aligned coaxially with the imaging beam.

39. The apparatus according to claim 37, wherein one or more of the pixels correspond to a reference location of the imaging beam.

40. The apparatus according to claim 1, wherein the imager is configured to generate a plurality of A scans using the imaging beam, and the processor is configured to determine the position of the imaging beam for each of the plurality of A scans in response to the sensor data, and to construct the image in response to the plurality of positions of the imaging beam.

41. The apparatus according to claim 40, further comprising an imaging optical fiber, wherein the imager includes one or more of the OCT imaging or the photoacoustic imaging, and an imaging beam is guided along the optical fiber.

42. The apparatus according to claim 41, further comprising an acoustic sensor positioned near the distal end of the optical fiber, wherein the imager includes the photoacoustic imager, and the sensor is configured to receive acoustic pulses in response to an imaging beam from the optical fiber that illuminates the tissue and generates acoustic pulses.

43. The apparatus according to claim 41, further comprising a scanner for scanning the imaging beam.

44. The apparatus according to claim 43, wherein the scanner is configured to scan the imaging beam by deflecting the distal end of the imaging optical fiber.

45. The apparatus according to claim 40, further comprising an ultrasonic transducer for generating the plurality of A scans.

46. The apparatus according to claim 1, wherein the optical imager includes one or more of an endoscope, a microscope, a stereomicroscope, and a stereoendoscopy, the processor is configured to generate motion data from a plurality of images from the optical imager, and the processor is configured to generate a 3D image of the tissue in response to the motion data and the plurality of images.

47. The apparatus according to claim 1, wherein the imager includes an ultrasonic (US) imager, and the US imager includes one or more arrays.

48. The apparatus according to claim 1, further comprising a treatment probe for treating the tissue, wherein the treatment probe is coupled to the sensor and the imager.

49. The apparatus according to claim 48, wherein the treatment probe includes a treatment channel for treating the tissue.

50. The apparatus according to claim 49, wherein the treatment channel is configured to perform one or more of the following: transporting an implant, guiding treatment energy, or manipulating tissue.

51. The apparatus according to claim 50, wherein the treatment channel includes an optical fiber for guiding laser energy to the tissue.

52. The apparatus according to claim 50, wherein the treatment channel includes a working channel for transporting the implant to the tissue.

53. The apparatus according to claim 1, further comprising a handpiece coupled to the imager and the sensor, wherein the processor is configured to detect user tremors and construct a tremor model in response to the sensor data.

54. The apparatus according to claim 53, wherein the processor is configured to determine the position of the imaging beam in the tissue in response to the sensor data and the tremor model.

55. The apparatus according to claim 54, wherein the tremor corresponds to the periodic movement of the handpiece.

56. The apparatus according to claim 53, wherein the imager is configured to emit an imaging beam, and the processor is configured to determine the location of the imaging beam in response to the vibration detector having the sensor.

57. The apparatus according to claim 1, further comprising a sensor for measuring the cardiac cycle of the tissue, wherein the processor is configured to construct the 3D image in response to the cardiac cycle.

58. The apparatus according to claim 57, wherein the sensor includes one or more of the following: an electrocardiogram (EKG) sensor, a pulse oximeter, or a blood oxygen sensor.

59. The apparatus according to claim 1, further comprising a plurality of visible laser beams for measuring the position of the tissue along the optical path of the imaging beam.

60. The apparatus according to claim 59, wherein the sensor includes a two-dimensional sensor array, and the processor is configured to determine the position of the tissue along the optical path in response to the separation distance between the plurality of visible laser beams on the sensor array.

61. The apparatus according to claim 60, wherein the processor is configured to construct the 3D image in response to the separation distance between the first laser beam and the second laser beam.

62. The apparatus according to claim 60, wherein the processor is configured to image the tissue with a plurality of OCT A scans and to measure the separation distance for each of the plurality of A scans.

63. The apparatus according to claim 62, wherein the processor is configured to construct the 3D image in response to the separation distance between the first measurement beam and the second measurement beam for each of the plurality of A scans.

64. The apparatus according to claim 60, wherein the processor is configured to determine the lateral position of the measurement beam on the tissue in response to an image of the tissue on the two-dimensional sensor array.

65. The apparatus according to claim 64, wherein one or more pixels of the two-dimensional sensor array correspond to the location of the measurement beam in the image.

66. The apparatus according to claim 64, wherein the processor is configured to determine the three-dimensional position of the measurement beam on the tissue in response to the separation distance between the first beam and the second beam, and the separation distance between the first measurement beam and the second measurement beam.

67. The apparatus according to claim 1, further comprising a treatment laser, wherein the treatment laser includes one or more of ultraviolet lasers, femtosecond lasers, visible lasers, or infrared lasers.

68. A method for imaging tissue, Acquiring image data using an imager coupled to a sensor configured to measure one or more of the position or orientation of the imager, To acquire sensor data from the sensor along with the aforementioned image data, To construct a 3D image of the tissue in response to the aforementioned image data and sensor data. Methods that include...

69. A device for imaging tissue, An imager for imaging the tissue using one or more of the following: an optical imager, an optical coherence tomography (OCT) imager, an ultrasound (US) imager, or a photoacoustic imager. A processor coupled to the imager, wherein the processor is composed of a motion model, and the processor is composed of instructions for constructing a 3D image of the tissue in response to the imager data and the motion model. A device equipped with the following features.

70. The apparatus or method according to any one of claims 1 to 69, wherein a 3D imaging channel is positioned proximal to the distal end of a treatment channel to observe one or more implants, optical fibers, tissue manipulators, or end effectors positioned at the distal end of the treatment channel.

Images