Augmented reality eyewear for image-guided medical interventions

Abstract

The head-mounted display includes an adjustable strap assembly including a frame, a first side strap connected to the frame, a second side strap connected to the frame, and an adjustment mechanism configured to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; a see-through display assembly pivotally connected to the frame; a first temple housing pivotally connected to the frame and slidably connected to the first side strap; and a second temple housing pivotally connected to the frame and slidably connected to the second side strap.

Description

[Technical Field]

[0001] The present disclosure generally relates to augmented reality systems, devices and methods, including head-mounted display (HMD) devices, for augmented reality (AR)-based image-guided systems to facilitate surgery or other medical interventions (e.g., therapeutic and / or diagnostic procedures), and other uses. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 405,901, entitled "AUGMENTED REALITY EYEWEAR FOR IMAGE-GUIDED MEDICAL INTERVENTION," filed September 13, 2022, the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0003] Near-eye displays can be used in various types of AR applications. For example, the present applicant has previously demonstrated that a head-mounted device with AR capabilities can be used to perform image-guided surgery (see, for example, the present applicant's U.S. Pat. Nos. 11,382,712, 11,389,252, 10,939,977, and 9,928,629, the present applicant's U.S. Patent Application Publication No. 2020 / 0163723, and the present applicant's International Publication Nos. 2023 / 021448 and 2023 / 021450, which describe methods and systems for applying augmented reality and near-eye display technology to image-guided surgery systems). The disclosures of these patents and published applications are incorporated herein by reference. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 11,382,712 [Patent Document 2] U.S. Patent No. 11,389,252 [Patent Document 3] U.S. Patent No. 10,939,977 [Patent Document 4] U.S. Patent No. 9,928,629 [Patent Document 5] US Patent Application Publication No. 2020 / 0163723 [Patent Document 6] International Publication No. 2023 / 021448 [Patent Document 7] International Publication No. 2023 / 021450 Summary of the Invention

[0005] Some embodiments of the present disclosure described below provide improved methods and systems for applying augmented reality (AR) technology, peripheral devices, and their operation methods to head-mounted displays (HMDs), for example, in image-guided medical (e.g., surgical) systems. In the context of this disclosure and the claims, the term head-mounted display or HMD shall be given its ordinary meaning and shall refer to any suitable display device configured to display information (e.g., images) superimposed on a scene (e.g., a portion of a patient's body for therapeutic or diagnostic intervention or evaluation), such as a patient's organs, skin, bones, or joints, using AR technology and / or other suitable display technology. For example, the term HMD may refer to helmets, AR glasses, goggles, eyeglasses, monoculars, eyewear, headsets, visors, head-up displays, and any other suitable type of display device attached to or worn by any part of a user's or wearer's head, including, but not limited to, the face, top of the head, forehead, nose, and ears. In some embodiments, a head-mounted display is not used or is used in conjunction with a standalone display, such as a monitor, handheld device, or tablet. The display may be a hands-free display such that the operator does not need to hold the display.

[0006] According to some embodiments, the head-mounted display devices described herein reduce stress or fatigue for the wearer and / or provide additional comfort features. The head-mounted display devices may provide improved ergonomics, comfort, and / or functionality that allows a wearer, such as a surgeon, to wear the device for relatively long periods of time (e.g., two, four, six, or more hours in one embodiment) without unnecessary fatigue and / or other adverse effects. For example, the head-mounted display device may be designed and adapted to distribute weight around the wearer's head, including the wearer's forehead and the back of the wearer's head, to reduce at least a portion of the weight being applied to the wearer's nose or other undesirable areas. Such a configuration may also reduce pressure on the wearer's temples, which may be another relatively weight-sensitive area in addition to the nose. Stated differently, such a head-mounted display may distribute pressure more widely over larger and / or less sensitive areas, such as the forehead and back of the head. While medical applications are well suited to some embodiments, non-medical applications also benefit from many of the embodiments described herein. For example, non-medical applications may include consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, and the like.

[0007] The head-mounted display device may be substituted with an alternative hands-free device that is not worn by the operator, such as a portal, monitor, or tablet. The display may be a head-up display or heads-up display.

[0008] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first temporal region of a wearer (e.g., a surgeon or other user) and the second end configured to be positioned adjacent a second temporal region of the wearer. The device further includes an adjustable strap assembly including: a first side strap having a first end coupled to the first end of the frame; a second side strap having a first end coupled to the second end of the frame; and an adjustment mechanism (e.g., a knob, rack and pinion, a mechanism using friction, a sliding mechanism, a ratchet mechanism, a snap or locking mechanism, etc.) configured to adjust the position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame. The device also includes a see-through display assembly (e.g., including a near-eye display, an augmented reality display, a stereoscopic display, glasses, a visor, a head-up display, etc.) coupled (e.g., rotatably, pivotably, movable, or slidably coupled) to the frame such that the tilt angle (e.g., pantoscopic tilt angle) can be adjusted. The device further includes a first temple housing coupled (e.g., rotatably or pivotably coupled) to a first end of the frame and slidably coupled to a first side strap of the adjustable strap assembly, and a second temple housing coupled (e.g., rotatably or pivotably coupled) to a second end of the frame and slidably coupled to a second side strap of the adjustable strap assembly.

[0009] In some implementations, the see-through display assembly is coupled to the frame (e.g., rotatably, pivotably, movably, or slidably coupled) using a pantoscopic tilt assembly that includes an arc-shaped slot that rotatably, pivotably, movably, or slidably couples a portion of the see-through display assembly to a portion of the frame.

[0010] In some implementations, the pantoscopic tilt assembly further includes a detent mechanism including a spring-loaded pin or ball and multiple detents (e.g., two, three, four, five, or six or more detents), configured to selectively hold the see-through display assembly in one of multiple predetermined positions relative to the frame.

[0011] In some implementations, the detent mechanism further includes a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

[0012] In some implementations, the see-through display assembly includes a removable lens assembly (e.g., a clip-on lens assembly, a snap-on lens assembly, a friction-fit lens assembly, a magnetic attachment assembly, etc.) that can be removed and replaced with a second removable lens assembly to change the prescription of the lenses of the see-through display assembly (e.g., the removable lens assemblies may be customized for a particular wearer so that they are interchangeable and the same device can be easily used interchangeably by multiple different wearers). In some implementations, the removable lens assembly may be provided along with the head-mounted display device by the manufacturer and / or provider of the head-mounted display device so that the user does not need to obtain it separately from an optometrist or third-party provider. In some implementations, the removable lens assembly can be coupled to and detached from the head-mounted display device without the use of any tools (e.g., "tool-less"). In some implementations, the removable lens assembly comprises or functions as an adapter. In some implementations, the removable lens assembly is non-magnetically coupled to the head-mounted display device.

[0013] In some implementations, the see-through display assembly includes a display assembly frame, a waveguide lens coupled to the display assembly frame, a front lens secured to the waveguide lens or to the display assembly frame in front of the waveguide lens, a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, or a clip, and a rear lens secured to the rear lens frame.

[0014] In some implementations, the head-mounted display device further includes a flashlight assembly (e.g., a headlamp assembly, a headlight assembly, etc.) that can be removably coupled to the frame. In some implementations, the flashlight can be permanently coupled to the frame.

[0015] In some implementations, the head-mounted display device further includes a first follower (e.g., a protrusion, a slidable member, etc.) that slidably connects the first temple housing to the first side strap, and a second follower (e.g., a protrusion, a slidable member, etc.) that slidably connects the second temple housing to the second side strap.

[0016] In some implementations, the frame further includes nose pads (e.g., nose supports, etc.) configured to engage the wearer's nose. The frame may optionally not include nose pads or may be configured not to engage the wearer's nose.

[0017] In some implementations, the head-mounted display device further includes a forehead support including a first end connected (e.g., rotatably or pivotally connected) to the first side strap, a second end connected (e.g., rotatably or pivotally connected) to the second side strap, and a central support connected to the frame.

[0018] In some implementations, the first side strap includes a connector connecting (e.g., rotatably or pivotally) a front portion of the first side strap to a rear portion of the first side strap and connecting (e.g., rotatably or pivotally) a first end of the forehead support to the first side strap, and the second side strap includes a connector connecting (e.g., rotatably or pivotally) a front portion of the second side strap to a rear portion of the second side strap and connecting (e.g., rotatably or pivotally) a second end of the forehead support to the second side strap.

[0019] In some implementations, the head-mounted display device further includes a first follower (e.g., a protrusion, a slidable member, etc.) slidably coupled to the first temple housing and coupled to the first side strap at a position between the connector of the first side strap and the first end of the first side strap, and a second follower (e.g., a protrusion, a slidable member, etc.) slidably coupled to the second temple housing and coupled to the second side strap at a position between the connector of the second side strap and the first end of the second side strap.

[0020] In some implementations, the head-mounted display device optionally further includes an upper strap removably coupled at a first end to the forehead support and removably coupled at a second end to an adjustment mechanism of the adjustable strap assembly.

[0021] In some implementations, the first side strap and the second strap each include a rack, and the adjustment mechanism of the adjustable strap assembly includes a pinion engaged with the rack of the first side strap and the rack of the second side strap, and a knob configured to rotate the pinion to adjust the circumferential size defined by the first side strap, the second side strap, and the frame (e.g., to customize the fit to a particular wearer's circumferential head size).

[0022] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a tensioning mechanism (e.g., a stop mechanism, one or more gears engaged with the tensioning member, etc.) that resists rotation of the knob until a threshold force is overcome.

[0023] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a pad configured to engage the back of the wearer's head.

[0024] In some implementations, the see-through display assembly is configured to display to the wearer augmented reality (AR) images, including virtual reality (VR) images presented superimposed on a portion of the patient's body, and the head-mounted display device further includes, or consists essentially of, one or more processors configured, for example, upon execution of program instructions stored on a non-transitory computer-readable medium, to receive signals indicating at least a position of the see-through display assembly relative to one or more anatomical images and scenes of the patient, and to render the AR images on the see-through display assembly. The see-through display assembly can be configured to enable the wearer to view the AR images from both standing and sitting positions without inconvenience or manual adjustments.

[0025] In some implementations, at least one of the one or more processors is located within the first temple housing or the second temple housing, hi some implementations, at least one of the one or more processors may be located elsewhere on the head-mounted display device or may be separate from and in wireless communication with the head-mounted display device.

[0026] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first temporal region of a wearer (e.g., a surgeon or other user) and the second end configured to be positioned adjacent a second temporal region of the wearer. The frame further includes a nose pad (e.g., a nose support member, etc.) configured to engage the wearer's nose, and an adjustable strap assembly including a first side strap having a first end coupled to the first end of the frame, a second side strap having a first end coupled to the second end of the frame, and an adjustment mechanism (e.g., a knob, a rack and pinion, a mechanism using friction, etc.) configured to adjust the position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame. The head-mounted display device further includes a forehead support including a first end connected (e.g., rotatably or pivotally connected) to the first side strap, a second end connected (e.g., rotatably or pivotally connected) to the second side strap, and a central support connected to the frame. The head-mounted display device further includes a see-through display assembly (e.g., an augmented reality display, a near-eye display, a stereoscopic display, glasses, a visor, a headset, goggles, a head-up display, etc.) connected (e.g., rotatably, pivotably, movably, or slidably connected) to the frame so that a tilt angle (e.g., a pantoscopic tilt angle) can be adjusted. The see-through display assembly includes a removable lens assembly (e.g., a clip-on lens assembly, a snap-on lens assembly, a friction-fit lens assembly, etc.) that can be removed and replaced with a second removable lens assembly to change the lens prescription of the see-through display assembly.The head-mounted display device also includes a first temple housing pivotally coupled to a first end of the frame and slidably coupled to a first side strap of the adjustable strap assembly, and a second temple housing pivotally coupled to a second end of the frame and slidably coupled to a second side strap of the adjustable strap assembly. The head-mounted display device further includes a flashlight assembly (e.g., a headlamp assembly, a headlight assembly, etc.) removably coupled to the frame.

[0027] In some implementations, the first side strap includes a connector that pivotally connects a front portion of the first side strap to a rear portion of the first side strap and a first end of the forehead support to the first side strap, and the second side strap includes a connector that pivotally connects a front portion of the second side strap to a rear portion of the second side strap and a second end of the forehead support to the second side strap.

[0028] In some implementations, the head-mounted display device further includes a first follower (e.g., a protrusion, a slidable member, etc.) slidably coupled to the first temple housing and coupled to the first side strap at a position between the connector of the first side strap and the first end of the first side strap, and a second follower (e.g., a protrusion, a slidable member, etc.) slidably coupled to the second temple housing and coupled to the second side strap at a position between the connector of the second side strap and the first end of the second side strap.

[0029] In some implementations, each of the first and second side straps includes a rack, and the adjustment mechanism of the adjustable strap assembly may include a pinion engaged with the rack of the first and second side straps and a knob configured to rotate the pinion to adjust a circumferential size defined by the first and second side straps and the frame.

[0030] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a tensioning mechanism (e.g., a stop mechanism, one or more gears engaged with the tensioning member, etc.) that resists rotation of the knob until a threshold force is overcome.

[0031] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a pad configured to engage the back of the wearer's head.

[0032] In some implementations, the head-mounted display device further includes an upper strap removably coupled at a first end to the forehead support and removably coupled at a second end to an adjustment mechanism of the adjustable strap assembly.

[0033] In some implementations, the see-through display assembly is configured to display to the wearer an augmented reality (AR) image including a virtual reality (VR) image presented superimposed on a scene of the patient's body, and the head-mounted display device further includes one or more processors configured to receive a signal indicating at least a position of the see-through display assembly relative to one or more anatomical images of the patient and the scene, and to render the AR image on the see-through display assembly.

[0034] In some implementations, at least one of the one or more processors is located within the first temple housing or the second temple housing, hi some implementations, at least one of the one or more processors may be located elsewhere on the head-mounted display device or may be separate from and in wireless communication with the head-mounted display device.

[0035] In some implementations, each of the first temple housing and the second temple housing includes a plurality of heat dissipation fins (eg, protrusions, heat sinks, etc.).

[0036] In some implementations, the see-through display assembly is rotatably, pivotably, movably, or slidably coupled to the frame using a pantoscopic tilt assembly that includes an arcuate slot that rotatably, pivotably, movably, or slidably couples a portion of the see-through display assembly to a portion of the frame, and a detent mechanism that includes a spring-loaded pin or ball and multiple detents. The detent mechanism can be configured to selectively hold the see-through display assembly in one of multiple predetermined positions relative to the frame.

[0037] In some implementations, the detent mechanism further includes a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

[0038] In some implementations, the frame includes a flashlight mounting socket including a first rod (e.g., a post, protrusion, shaft, etc.) defining a pivot or rotation axis and a second rod (e.g., a post, protrusion, shaft, etc.) disposed parallel to the first rod. The flashlight assembly may include a first recess (e.g., an opening, socket, recess, etc.) shaped to engage the first rod of the flashlight mounting socket and pivot about the first rod, a second recess (e.g., an opening, socket, recess, etc.) shaped to engage the second rod of the flashlight mounting socket, the second recess being oriented such that the first recess cannot disengage the first rod when engaged with the second rod, and a movable latch (e.g., a hook, coupler, etc.) configured to selectively hold the second recess in engagement with the second rod.

[0039] In some implementations, the head-mounted display device further includes a biasing mechanism, such as a spring, that biases the movable latch toward a position that holds the second recess in engagement with the second rod.

[0040] In some implementations, the flashlight assembly includes a flashlight (e.g., a headlight, a headlamp, etc.), a mounting base including a first recess, a second recess, and a movable latch, and one or more arms that pivotally or otherwise rotatably connect the flashlight to the mounting base.

[0041] In some implementations, the see-through display assembly includes a display assembly frame, a waveguide lens coupled to the display assembly frame, a front lens secured to the waveguide lens or in front of the waveguide lens to the display assembly frame, a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, a magnetic attachment, a hook-and-loop attachment, or a clip, and a rear lens secured to the rear lens frame.

[0042] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of a wearer's (e.g., a surgeon or other user's) head and the second end configured to be positioned adjacent a second side of the wearer's head; a head-mounted assembly configured to hold the frame in position on the wearer's head; a see-through display; and a tilt assembly (e.g., a pantoscopic tilt assembly) that rotatably, pivotably, movably, or slidably couples the see-through display to the frame so that a tilt angle (e.g., a pantoscopic tilt angle) can be adjusted. The tilt assembly (e.g., the pantoscopic tilt assembly) includes an arc-shaped slot that rotatably, pivotably, movably, or slidably couples a portion of the see-through display assembly to a portion of the frame. The pantoscopic tilt assembly also includes a detent mechanism including a spring-loaded pin or ball and multiple detents configured to selectively hold the see-through display assembly in one of a plurality of predetermined positions relative to the frame.

[0043] In some implementations, the detent mechanism further includes a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

[0044] In some implementations, the arcuate slot defines a virtual hinge that includes an axis of rotation configured to be aligned with the center of the wearer's eyeball. In some implementations, the virtual hinge, as opposed to a physical hinge, advantageously allows the wearer's peripheral vision to be unobstructed or distorted.

[0045] In some implementations, the head-mounted assembly includes an adjustable strap assembly including: a first side strap having a first end coupled to a first end of the frame; a second side strap having a first end coupled to a second end of the frame; and an adjustment mechanism (e.g., a knob, a rack and pinion, a mechanism using friction, etc.) configured to adjust the position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame.

[0046] In some implementations, the head-mounted display device further includes a forehead support including a first end pivotally connected to the first side strap, a second end pivotally connected to the second side strap, and a central support connected to the frame.

[0047] In some implementations, the head-mounted display device further includes a first temple housing pivotally coupled to a first end of the frame and slidably coupled to the head-mounted assembly, a second temple housing pivotally coupled to a second end of the frame and slidably coupled to the head-mounted assembly, and one or more processors configured to render images (e.g., upon execution of program instructions stored on a non-transitory computer-readable medium) for display by the see-through display. In some implementations, at least one of the one or more processors is located within the first temple housing or the second temple housing, but may be located elsewhere.

[0048] In some implementations, the head-mounted assembly includes a first temple arm coupled to the frame and configured to be placed over a first ear of the wearer, and a second temple arm coupled to the frame and configured to be placed over a second ear of the wearer.

[0049] In some implementations, the frame further includes nose pads (eg, nose support members, etc.) configured to engage the wearer's nose.

[0050] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first temporal region of a wearer (e.g., a surgeon or other user) and the second end configured to be positioned adjacent a second temporal region of the wearer. The head-mounted display device also includes an adjustable strap assembly including a first side strap having a first end coupled to the first end of the frame, a second side strap having a first end coupled to the second end of the frame, and an adjustment mechanism (e.g., a knob, a rack and pinion, a mechanism using friction, etc.) configured to adjust the position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame. The head-mounted display device further includes a first temple housing pivotally coupled to a first end of the frame and slidably coupled to a first side strap of the adjustable strap assembly, and a second temple housing pivotally coupled to a second end of the frame and slidably coupled to a second side strap of the adjustable strap assembly. The head-mounted display device also includes a see-through display coupled to the frame.

[0051] In some implementations, the head-mounted display device further includes a first follower (e.g., a protrusion, a slidable member, etc.) that slidably connects the first temple housing to the first side strap, and a second follower (e.g., a protrusion, a slidable member, etc.) that slidably connects the second temple housing to the second side strap.

[0052] In some implementations, the first follower includes an elongated protrusion that can slide back and forth within an elongated slot in the first temple housing in response to pivoting of the first temple housing relative to the frame. The second follower may include an elongated protrusion that can slide back and forth within an elongated slot in the second temple housing in response to pivoting of the second temple housing relative to the frame.

[0053] In some implementations, the first side strap and the second strap each include a rack, and the adjustment mechanism of the adjustable strap assembly includes a pinion engaged with the rack of the first side strap and the rack of the second side strap, and a knob configured to rotate the pinion to adjust a circumferential size defined by the first side strap, the second side strap, and the frame.

[0054] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a tensioning mechanism (e.g., a stop mechanism, one or more gears engaged with the tensioning member, etc.) that resists rotation of the knob until a threshold force is overcome.

[0055] In some implementations, the adjustment mechanism of the adjustable strap assembly further includes a pad configured to engage the back of the wearer's head.

[0056] In some implementations, the head-mounted display device further includes a forehead support including a first end pivotally connected to the first side strap, a second end pivotally connected to the second side strap, and a central support connected to the frame.

[0057] In some implementations, the head-mounted display device further includes an upper strap removably coupled at a first end to the forehead support and removably coupled at a second end to an adjustment mechanism of the adjustable strap assembly.

[0058] In some implementations, the frame further includes nose pads (eg, nose support members, etc.) configured to engage the wearer's nose.

[0059] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first temporal region of a wearer (e.g., a surgeon or other user) and the second end configured to be positioned adjacent a second temporal region of the wearer. The head-mounted display device further includes a head-mounted assembly configured to hold the frame in position on the wearer's head, a see-through display, and a flashlight assembly (e.g., a headlamp assembly, a headlight assembly, etc.) removably coupled to the frame. The frame includes a flashlight mounting socket including a first rod (e.g., a post, protrusion, shaft, etc.) defining a pivot axis and a second rod (e.g., a post, protrusion, shaft, etc.) positioned parallel to the first rod. The flashlight assembly includes a first recess (e.g., opening, socket, recess, etc.) shaped to engage a first rod of the flashlight mounting socket and pivot about the first rod, a second recess (e.g., opening, socket, recess, etc.) shaped to engage a second rod of the flashlight mounting socket and oriented such that the first recess cannot disengage the first rod when the second recess engages the second rod, and a movable latch (e.g., hook, coupler, etc.) configured to selectively hold the second recess in engagement with the second rod.

[0060] In some implementations, the head-mounted display device further includes a biasing structure, such as a spring, that biases the movable latch toward a position that holds the second recess in engagement with the second rod.

[0061] In some implementations, the flashlight assembly includes a flashlight (e.g., a headlight, a headlamp, etc.), a mounting base including a first recess, a second recess, and a movable latch, and one or more arms pivotally connecting the flashlight to the mounting base.

[0062] In some implementations, the flashlight mounting socket further includes one or more electrical contacts configured to electrically couple to corresponding one or more electrical contacts of the flashlight assembly.

[0063] In some implementations, the frame further includes nose pads (eg, nose support members, etc.) configured to engage the wearer's nose.

[0064] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of a wearer's (e.g., a surgeon or other user's) head and the second end configured to be positioned adjacent a second side of the wearer's head. The head-mounted display device further includes a head-mounted assembly configured to hold the frame in position on the wearer's head. The head-mounted display device also includes a see-through display assembly (e.g., an augmented reality display, a stereoscopic display, glasses, a visor, etc.) coupled to the frame. The see-through display assembly includes a display assembly frame, a waveguide lens coupled to the display assembly frame, a front lens secured to the waveguide lens or in front of the waveguide lens and to the display assembly frame, a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, or a clip, and a rear lens secured to the rear lens frame.

[0065] In some implementations, the head mounted display device further includes a first seal between the front lens and the waveguide lens, and a second seal between the rear lens frame and the waveguide lens.

[0066] In some implementations, the rear lens frame includes a first protrusion (e.g., clip, snap, etc.) at a top of the rear lens frame that fits into a first corresponding recess (e.g., opening, hole, slot, etc.) in the display assembly frame. The rear lens frame includes a second protrusion (e.g., clip, snap, etc.) at a bottom of the rear lens frame that forms a snap fit with a second corresponding recess (e.g., opening, hole, slot, etc.) in the display assembly frame.

[0067] In some implementations, the see-through display assembly is coupled to the frame (e.g., rotatably, pivotably, movably, or slidably coupled) so that the tilt angle (e.g., pantoscopic tilt angle) can be adjusted.

[0068] In some implementations, the see-through display assembly is coupled (rotatably, pivotably, movably, or slidably coupled) to the frame using a pantoscopic tilt assembly that includes an arcuate slot that rotatably, pivotably, movably, or slidably couples a portion of the see-through display assembly to a portion of the frame, and a detent mechanism including a spring-loaded pin or ball and multiple detents configured to selectively hold the see-through display assembly in one of multiple predetermined positions relative to the frame.

[0069] In some implementations, the detent mechanism further includes a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

[0070] In some implementations, the frame further includes nose pads (eg, nose support members, etc.) configured to engage the wearer's nose.

[0071] According to some implementations, the system includes a head-mounted display (HMD) including a frame configured to be worn on the head of a user (e.g., a surgeon or other user); and a display (e.g., including an augmented reality display, a stereoscopic display, glasses, goggles, a head-up display, a visor, etc.) (i) connected to the frame and configured to rotate relative to the user's frontal plane to set a tilt angle (e.g., a pantoscopic tilt angle) of the display, and (ii) at least partially transparent and configured to display augmented reality (AR) images to the user, including virtual reality (VR) images presented superimposed on a scene of the patient's body; and at least one processor configured to (i) receive a signal indicating at least a position of the display relative to one or more anatomical images and scenes of the patient, and (ii) render the AR images on the display.

[0072] In some implementations, the frame includes an adjustable head-mounted assembly including adjustable temple arms configured to rest over the user's ears and adjust to fit along at least a portion of the user's temples, adjustable nose pads (e.g., nose support members, etc.) configured to rest over the user's nose and fit the shape of at least a portion of the nose, and a housing connected to the temple arms and nose pads.

[0073] In some implementations, at least one of the temple arms includes a first section and a second section and a first tilt assembly and a second tilt assembly, the first tilt assembly configured to tilt the first section relative to the frame, and the second tilt assembly configured to rotate the second section relative to the first section.

[0074] In some implementations, the second tilt assembly includes a rocker arm configured to rotate about a hinge relative to the longitudinal axis of the first section.

[0075] In some implementations, the first section has an opening, the opening configured to accommodate at least a portion of the rocker arm when the rocker arm is rotating.

[0076] In some implementations, the rocker arm includes a cushion made from viscoelastic foam that conforms to the user's nape and is shaped to improve clamping between the frame of the HMD and the user's head.

[0077] In some implementations, the cushion includes a material selected from the list of materials consisting essentially of at least one of: (i) silicone, (ii) neoprene, and (iii) polyurethane.

[0078] In some implementations, the cushion includes a sponge.

[0079] In some implementations, the second tilt assembly includes an alloy coated with a viscoelastic foam and molded to conform to the user's nape.

[0080] In some implementations, (i) the first section and the second section, (ii) the first tilt assembly and the second tilt assembly, and (iii) the nose pads are adapted to provide the user with two or more degrees of freedom (DOF) for adjusting the frame to fit the contours of the user's head.

[0081] In some implementations, the second tilt assembly includes an array of rocker arms, each of the rocker arms configured to rotate about a respective hinge.

[0082] In some implementations, the arrangement of rocker arms is attached to a common bar and includes an additional hinge, and the common bar is configured to rotate about the additional hinge.

[0083] In some implementations, the frame includes an adjustable head-mounted assembly including: a first side strap having a first end coupled to a first end of the frame; a second side strap having a first end coupled to a second end of the frame; and an adjustment mechanism (e.g., a knob, rack and pinion, mechanism using friction, etc.) configured to adjust the position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame.

[0084] In some implementations, the system further includes a first temple housing pivotally coupled to the first end of the frame and slidably coupled to the first side strap, and a second temple housing pivotally coupled to the second end of the frame and slidably coupled to the second side strap.

[0085] In some implementations, at least one eye has a first optical axis, the display's optical engine has a second optical axis, and the pantoscopic tilt angle is set to align the second optical axis with the first optical axis.

[0086] In some implementations, the HMD includes a pantoscopic tilt assembly (PTA), which is connected to the frame and the display and configured to rotate the second optical axis relative to the first optical axis to adjust the pantoscopic tilt angle.

[0087] In some implementations, the PTA includes a hinge connecting the first optical axis and the second optical axis.

[0088] In some implementations, the PTA includes a bar coupled to an optical engine including a display, an edge of the bar coupled to a hinge, and the hinge adapted to rotate the bar relative to a first axis of the frame.

[0089] In some implementations, the PTA includes at least two portions other than the hinge and includes a virtual axis adapted to rotate the second optical axis relative to the first optical axis.

[0090] In some implementations, the virtual axis includes (i) a bar coupled to an optical engine including a display and a rotatable section of a disk having a slit, and (ii) an element configured to be inserted into the slit and moved along the slit in a direction tangential to the slit.

[0091] In some implementations, the virtual axis includes an arm including a first section, a second section, and a third section, the first section coupled to an optical engine including a display, the second section coupled to the frame, and the third section coupled between the first and second sections and configured to bend in response to a force applied to the PTA to adjust the pantoscopic tilt angle.

[0092] In some implementations, the virtual axis includes (i) a rigid arm connected to the frame and (ii) a flexible arm connected to an optical engine including a display, the flexible arm adapted to convert from elastic to plastic deformation and retain the resulting shape in response to a force applied to the PTA to adjust the pantoscopic tilt angle.

[0093] In some implementations, the PTA further includes a detent mechanism including a spring-loaded pin or ball and multiple detents, the detent mechanism configured to selectively hold the see-through display assembly in one of multiple predetermined positions relative to the frame.

[0094] According to some implementations, a head-mounted display device includes a frame extending from a first end to a second end, an adjustable head-mounted assembly configured to hold the frame in position on a user's (e.g., a surgeon or other user's) head, a display (e.g., including an augmented reality display, a stereoscopic display, glasses, a visor, etc.) that is at least partially transparent and configured to display to the user augmented reality (AR) images including virtual reality (VR) images presented superimposed on a scene of the patient's body, a pantoscopic tilt assembly that pivotally connects (e.g., rotatably, pivotably, movable, or slidably connects) the display to the frame so that the pantoscopic tilt angle can be adjusted, a first temple housing pivotally connected to the first end of the frame by the first tilt assembly, and a second temple housing pivotally connected to the second end of the frame by the second tilt assembly. At least one of the first temple housing or the second temple housing includes at least one processor configured to receive a signal indicating one or more anatomical images of the patient and at least a position of the display relative to the scene, and to render an AR image on the display.

[0095] In some implementations, the head-mounted assembly includes an adjustable head strap configured to engage at least the back of the user's head and the user's forehead.

[0096] In some implementations, the first temple housing and the second temple housing are each part of a head-mounted assembly.

[0097] In some implementations, the head-mounted assembly further includes an adjustable nose pad (e.g., a nose support member, etc.) configured to conform to the shape of at least a portion of the user's nose.

[0098] In some implementations, the pantoscopic tilt assembly includes a slot arranged to allow a pivot element to slide tangentially relative to the slot, and the pivot element configured to be locked in one or more predetermined positions relative to the slot.

[0099] According to some implementations, a method for pairing a head-mounted display and a workstation in a medical center network includes introducing a communication device of the head-mounted display to a first network and, if the communication device of the head-mounted display is previously known to the first network, initiating pairing of the communication device of the head-mounted display with a first workstation of the first network using previous connection parameters. The method also includes, if the communication device of the head-mounted display is not previously known to the first network, initiating pairing of the communication device of the head-mounted display with the first workstation of the first network using a key exchange process that generates new connection parameters. The method further includes exchanging data between the first workstation and the communication device of the head-mounted display during a surgical procedure to enable the head-mounted display to display augmented reality (AR) images, including virtual reality (VR) images, to a user, superimposed on a scene of a patient's body. In some implementations, the head-mounted display comprises any of the head-mounted displays or head-mounted display devices disclosed herein.

[0100] In some implementations, the method further includes initiating pairing of the communication device of the head mounted display with a second workstation of the first network in response to the pairing not being successfully completed within a predetermined time limit.

[0101] In some implementations, the method further includes unpairing the communication device of the head-mounted display from a first workstation of the first network, initiating pairing of the communication device of the head-mounted display with a second workstation of the second network, and exchanging data between the second workstation and the communication device of the head-mounted display during the surgical procedure to enable the head-mounted display to display to the user augmented reality (AR) images including virtual reality (VR) images presented overlaid on a scene of the patient's body.

[0102] Also described and contemplated herein is the use of any of the devices, systems, or methods for the treatment of the spine by surgical intervention.

[0103] Also described and contemplated herein is the use of any of the devices, systems, or methods for the treatment of orthopedic joints, including, optionally, shoulder, knee, ankle, hip, or other joints, by surgical intervention.

[0104] Also described and contemplated herein is the use of any of the devices, systems, or methods for the treatment of the skull by surgical intervention.

[0105] Also described and contemplated herein is the use of any of the devices, systems, or methods for the treatment of the jaw by surgical intervention.

[0106] Also described and contemplated herein is the use of any of the devices, systems, or methods for the diagnosis of spinal abnormalities.

[0107] Also described and contemplated herein is the use of any of the devices, systems, or methods for the diagnosis of spinal cord injury.

[0108] Also described and contemplated herein is the use of any of the devices, systems, or methods for the diagnosis of joint damage.

[0109] Also described and contemplated herein is the use of any of the devices, systems, or methods for the diagnosis of orthopedic injuries.

[0110] Also described and contemplated herein is the use of any of the devices, systems, or methods in non-medical applications such as gaming, driving, product design, shopping, manufacturing, athletics or fitness, navigation, remote collaboration, and / or education.

[0111] For purposes of summarizing the present disclosure, certain aspects, advantages, and novel features have been described herein. It should be understood that not all such aspects, advantages, or features may be embodied in any particular embodiment of the present disclosure, and that, upon reading this disclosure, one of ordinary skill in the art will recognize countless combinations of such aspects, advantages, or features. The embodiments disclosed herein may be embodied or implemented to achieve or optimize one advantage or advantages taught or suggested herein without necessarily achieving other advantages that may be taught or suggested herein. The systems and methods of the present disclosure each have several innovative aspects, no single one of which is solely responsible for the desirable attributes disclosed herein.

[0112] The present disclosure will be more fully understood from the following detailed description of the embodiments thereof, taken in conjunction with the drawings, claims and above description. A brief description of the drawings follows. [Brief explanation of the drawings]

[0113] Non-limiting features of some embodiments of the present invention are set forth with particularity in the following claims. The following drawings illustrate non-limiting embodiments for illustrative purposes only. In some embodiments, features from different drawings may be combined. It should be understood that the drawings are not necessarily drawn to scale. Distances, angles, etc. are merely illustrative and do not necessarily bear an exact relationship to the actual dimensions and layout of the illustrated devices.

[0114] [Figure 1] 1 is a schematic, pictorial illustration of an augmented reality-based image-guided surgery system with a head-mounted display (HMD).

[0115] [Figure 2A] 2 is a schematic pictorial diagram of an HMD for use in the system of FIG. 1; [Figure 2B] 2 is a schematic pictorial diagram of an HMD for use in the system of FIG. 1;

[0116] [Figure 3] 1 is a schematic pictorial diagram of an HMD comprising a waveguide-based optical engine (OE) having a liquid crystal display (LCD) and a closed-loop control assembly for controlling light intensity at the OE.

[0117] [Figure 4] 1 is a schematic, pictorial diagram of a headlight assembly (HA) for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG. [Figure 5] 1 is a schematic, pictorial diagram of a headlight assembly (HA) for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG. [Figure 6] 1 is a schematic, pictorial diagram of a headlight assembly (HA) for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG. [Figure 7] 1 is a schematic, pictorial diagram of a headlight assembly (HA) for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG. [Figure 8] 1 is a schematic, pictorial diagram of a headlight assembly (HA) for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG.

[0118] [Figure 9] FIG. 2 is a schematic, pictorial view of a self-adjustable temple arm tilt assembly for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32.

[0119] [Figure 10A] 29A is a schematic, pictorial diagram of an additional multi-degree-of-freedom (DOF) implementation implemented in one or more tilt assemblies with self-adjustable and ergonomic temple arms and rocker arms for use with any type of HMD shown in FIGS. 1, 2A, 2B, 3, and 29A. [Figure 10B] 29A is a schematic, pictorial diagram of an additional multi-degree-of-freedom (DOF) implementation implemented in one or more tilt assemblies with self-adjustable and ergonomic temple arms and rocker arms for use with any type of HMD shown in FIGS. 1, 2A, 2B, 3, and 29A.

[0120] [Figure 11] FIG. 29B is a schematic, pictorial diagram of another implementation of a multi-DOF implementation implemented in one or more respective tilt assemblies of a self-adjustable and ergonomic temple arm for use in any of the HMDs shown in FIGS. 1, 2A, 2B, 3, and 29A.

[0121] [Figure 12] 3 is a schematic, pictorial view of a nose pad for use with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG.

[0122] [Figure 13] FIG. 1 is a schematic pictorial diagram of a DOF implementation implemented in a pantoscopic tilting assembly (PTA) implemented with an optical engine for use in any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. [Figure 14]FIG. 1 is a schematic pictorial diagram of a DOF implementation implemented in a pantoscopic tilting assembly (PTA) implemented with an optical engine for use in any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32.

[0123] [Figure 15A] 15 is a schematic, pictorial illustration of a use case of the PTA of FIG. 14 used in several types of surgical procedures. [Figure 15B] 15 is a schematic, pictorial illustration of a use case of the PTA of FIG. 14 used in several types of surgical procedures. [Figure 15C] 15 is a schematic, pictorial illustration of a use case of the PTA of FIG. 14 used in several types of surgical procedures.

[0124] [Figure 16] 1 is a schematic, pictorial diagram of a structured light projector for use in any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32. FIG.

[0125] [Figure 17] FIG. 10 is a schematic diagram illustrating a method for generating synchronously exposed images by illuminating a rolling shutter image sensor with an illumination strobe within the integration time of the rolling shutter for use in any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32.

[0126] [Figure 18] FIG. 1 is a schematic diagram of a direct pairing system for directly pairing a workstation (WS) with any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32.

[0127] [Figure 19]17 is a flowchart illustrating a method for directly pairing the WS of FIG. 17 with multiple HMDs selected from any of the HMDs shown in FIGS. 1, 2A, 2B, 3, 21A to 29E, and 31A to 32.

[0128] [Figure 20] FIG. 2 is a block diagram illustrating a schematic implementation of the electronic architecture of the system of FIG. 1.

[0129] [Figure 21A] FIG. 2 illustrates another embodiment of a head-mounted display for use in the system of FIG. 1. [Figure 21B] FIG. 2 illustrates another embodiment of a head-mounted display for use in the system of FIG. 1. [Figure 21C] FIG. 2 illustrates another embodiment of a head-mounted display for use in the system of FIG. 1. [Figure 21D] FIG. 2 illustrates another embodiment of a head-mounted display for use in the system of FIG. 1.

[0130] [Figure 22A] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A. [Figure 22B] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A. [Figure 23A] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A. [Figure 23B] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A. [Figure 23C] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A. [Figure 23D] FIG. 21B is a diagram showing further details of the adjustment mechanism of the head-mounted display of FIG. 21A.

[0131] [Figure 24A] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein. [Figure 24B] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein. [Figure 25] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein. [Figure 26A] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein. [Figure 26B] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein. [Figure 26C] 10A-10C illustrate embodiments of additional pads and / or straps that may be incorporated into any of the head-mounted displays disclosed herein.

[0132] [Figure 27A] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 27B] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 27C] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 27D] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein.

[0133] [Figure 28A]FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 28B] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 28C] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 28D] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 28E] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein. [Figure 28F] FIG. 1 illustrates another embodiment of a headlight assembly for use with any of the head-mounted displays disclosed herein.

[0134] [Figure 29A] FIG. 10 illustrates another embodiment of a head-mounted display incorporating a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 29B] FIG. 10 illustrates another embodiment of a head-mounted display incorporating a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 29C] FIG. 10 illustrates another embodiment of a head-mounted display incorporating a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 29D] FIG. 10 illustrates another embodiment of a head-mounted display incorporating a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 29E] FIG. 10 illustrates another embodiment of a head-mounted display incorporating a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein.

[0135] [Figure 30] 29B is a schematic diagram illustrating different lenses and frames for use with the clip-on lens assembly of FIG. 29A.

[0136] [Figure 31A] FIG. 10 illustrates another embodiment of a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 31B] FIG. 10 illustrates another embodiment of a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 31C] FIG. 10 illustrates another embodiment of a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 31D] FIG. 10 illustrates another embodiment of a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein. [Figure 31E] FIG. 10 illustrates another embodiment of a clip-on lens assembly that can be used with any of the head-mounted displays disclosed herein.

[0137] [Figure 32] FIG. 1 illustrates an embodiment of an AR display having interchangeable lenses that can be used with any of the head-mounted displays disclosed herein. DETAILED DESCRIPTION OF THE INVENTION

[0138] Some image-guided surgery systems may apply augmented reality (AR) techniques to display one or more anatomical images of structures (e.g., bones, joints, soft tissues, organs, cartilage) within a patient's body that are to be operated on, superimposed on those structures. For example, the system may include a suitable head-mounted display (HMD) configured to display three-dimensional (3D) anatomical images, two-dimensional (2D) or 3D cross-sections, instrument trajectories, instrument depths, and additional information to assist the surgeon or other wearer in visualizing structures (e.g., vertebrae, joints, bones, soft tissues, organs) hidden from view by overlying tissue layers (e.g., during a minimally invasive interventional procedure or surgery that does not require open surgery to expose a target area of ​​the body).

[0139] Some embodiments of the present disclosure described herein provide assemblies and methods that can be implemented in conjunction with several types of HMDs to improve the quality of image-guided surgical or other interventional procedures, including spinal surgery and other types of orthopedic procedures (as well as other types or categories of procedures, e.g., dental procedures, cranial procedures, neurological procedures, joint surgery (e.g., shoulder, knee, hip, ankle, other joints), cardiac surgery, bariatric surgery, facial bone surgery, neurosurgery, etc.), including minimally invasive procedures that do not require open surgery but can be performed through small incisions (e.g., self-sealing incisions that do not require staples or sutures). It should be noted that each of the HMDs described below may comprise a basic configuration and additional optional components and assemblies that may be implemented in one or more of the HMDs in addition to the basic configuration.

[0140] The described systems, devices, and methods may be used in connection with other medical procedures (including therapeutic and diagnostic procedures), as well as other instruments and devices or other non-medical display environments. The methods described herein further include performing medical procedures (including, but not limited to, performing surgical interventions such as treating the spine, shoulder, hip, knee, ankle, other joints, jaw, skull, etc.). While medical applications are well suited for some embodiments, non-medical applications also benefit from many of the embodiments described herein. For example, non-medical applications may include consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, etc. The surgeon referred to herein may be a consumer or other wearer or user.

[0141] FIG. 1 is a schematic, pictorial illustration of an augmented reality (AR)-based image-guided system 11 used to perform a surgical or other interventional medical procedure, according to one embodiment of the present disclosure.

[0142] In one example, the surgical procedure (e.g., a minimally invasive surgical procedure or an open surgical procedure) includes one or more orthopedic surgical procedures performed on one or more vertebrae of the spine of a subject, herein referred to as patient 23, lying on operating table 12. One exemplary application is a lateral approach interbody fusion (LLIF) procedure to treat a disc problem in the lower back region (e.g., lumbar or lumbosacral region) of patient 23. In other embodiments, the techniques described below are applicable, mutatis mutandis, to other types of surgical procedures performed on other vertebrae (e.g., lumbar, thoracic, cervical, sacral) of patient 23, any suitable organ, bone(s), joint (e.g., sacroiliac, knee, shoulder, ankle, hip), or other tissue of patient 23. In non-limiting examples, system 11 may be used in cranial procedures, oral procedures, and other types of procedures performed on bony tissue of patient 23, such as maxillofacial surgery, knee surgery, hip surgery, and shoulder surgery. Additionally, system 11 may also be used, mutatis mutandis, in surgical or other interventional (e.g., therapeutic or diagnostic) procedures of soft tissue (e.g., neurological procedures, cranial procedures, joint repair or reconstructive procedures, scoping procedures, arthroscopic procedures, ablation procedures, etc.) System 11 may also be used in non-medical applications, including consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, etc., and the wearer may be someone other than a surgeon or medical professional (e.g., a consumer or other user).

[0143] In some embodiments, during a procedure, a medical professional, also referred to herein as a doctor or surgeon 26, uses suitable instruments to make an incision 24 in the patient's back. In some embodiments, the surgeon 26 inserts a fixation device, such as a spinous process clamp 30, into the incision 24 so that opposing jaws of the clamp 30 are positioned on opposite sides of a spinous process. The surgeon 26 then slides the clamp 30 over the lamina and adjusts the clamp 30 to grasp one or more spinous processes of the patient 23 selected by the surgeon 26. One optional implementation of the clamp 30 is described in more detail in WO 2022 / 079565, the disclosure of which is incorporated herein by reference.

[0144] In some embodiments, the clamp 30 serves as a support for patient markers 38 that are firmly attached to the clamp. During substantially all of the procedure, e.g., during the initial and subsequent stages, the patient markers (not shown) are used as a reference for the patient 23, and because they are rigidly connected to the patient 23, any movement of the patient 23 is reflected in a corresponding movement of the patient markers 38. Thus, during the initial stages of the procedure, the markers 38 are aligned with the anatomy of the patient 23.

[0145] In some embodiments, the fixation device may be a pin inserted into the patient's bone, for example the ilium. One optional implementation of such a pin is described in more detail in WO 2023 / 281395, the disclosure of which is incorporated herein by reference.

[0146] Embodiments relating to alignment tools, markers, marks, adapters, and methods are described in detail, for example, in U.S. Patent Application Publication No. 2022 / 0071712, U.S. Patent Application Publication No. 2022 / 0142730, U.S. Patent No. 10,939,977, and U.S. Patent Application Publication No. 2021 / 0161614, the disclosures of which are all incorporated herein by reference.

[0147] In some embodiments, system 11 comprises (i) a head-mounted display (HMD) 22, described in detail below, worn by surgeon 26, and (ii) one or more surgical and / or diagnostic instruments, such as, but not limited to, surgical instrument 190, and one or more reflectors, such as reflector 194, mounted on instrument 190. The reflectors may comprise markers for alignment and / or calibration purposes.

[0148] Reference is now made to inset 13, which shows one optional implementation of HMD 22 shown in front view.

[0149] In the context of this disclosure, the term "front view" refers to a view of the HMD 22 as seen by the eye of a person positioned in front of the surgeon 26 wearing the HMD 22. In the example of FIG. 1, the HMD 22 comprises a visor-based optical engine that includes a processor 33 and a communication device (shown and described below in connection with FIGS. 2B and 18) configured to exchange signals or transmissions with entities external to the HMD 22, such as a workstation or remote control tablet, as described below in connection with FIG. 18. Typically, the communication device is configured to send and receive signals or transmissions using a suitable wireless technology, such as, but not limited to, Wi-Fi (also referred to herein as Wi-Fi) or Bluetooth® wireless communication protocols or standards. The HMD 22 further comprises one or more power devices, such as, but not limited to, a battery (shown below in FIG. 20) or a supercapacitor or ultracapacitor, as described in more detail below.

[0150] In some embodiments, processor 33 is configured to receive information such as anatomical images and signals from one or more sensors (described below) and other entities of system 11 and display to surgeon 26 one or more images of an external portion of the patient's body overlaid on the surgeon's actual field of view. For example, during spinal surgery, processor 33 is configured to generate an augmented reality (AR) display that may show a 3D image of vertebrae overlaid on the patient's back as seen by the patient's eyes. Particular embodiments of images, signals, and AR displays are described in more detail below.

[0151] In some embodiments, the HMD 22 includes a visor 14 of a visor-based optical engine for each eye of the surgeon 26, which is not shown in FIG. 1 but an exemplary implementation of which is shown and described below in connection with FIG. 2A.

[0152] In some embodiments, the optical engine (OE) comprises (i) a projector configured to project an AR image generated by the processor, and (ii) an optical system configured to direct the projected AR image toward the visor, also referred to herein as AR display 15.

[0153] In various embodiments, the projector includes one or more light sources and / or image sources. As an example, the projector includes an organic light-emitting diode (OLED)-based image source and a display including a matrix of LEDs having a total size (e.g., diagonal size) of about 0.5 inches. Displays of other sizes may also be implemented.

[0154] In the context of this disclosure and the claims, the term "AR image" and grammatical variations thereof refer to a virtual reality (VR) image that includes at least a partially transparent portion and is overlaid on or integrated with a display having a scene in the background, and thus the combination of the VR image and the scene is referred to herein as an AR image.

[0155] In some embodiments, the AR display 15 is configured to display to the surgeon 26 an AR image generated by the processor 33 by reflecting the AR image into the pupil of the eye of the surgeon 26. The optical engine is shown and described below in connection with FIG. 2A.

[0156] In other embodiments, the OE of HMD 22 may have a different configuration and may be based on different technologies, such as, but not limited to, waveguide and liquid crystal based OEs, which are described in more detail below in connection with Figures 3 and 29A-29E.

[0157] In some embodiments, the HMD 22 comprises one or more light sources for tracking applications configured to direct a light beam onto the surface of an organ of interest or treatment area (e.g., the back) of the patient 23. In some embodiments, the light source comprises a pair of infrared (IR) LED projectors 17 configured to direct an IR light beam onto the surface of the treatment area. In other embodiments, the light source may include one or more light sources of any other suitable type configured to direct any suitable wavelength or wavelength band of light and mounted on the HMD 22 or elsewhere in the operating room.

[0158] In some embodiments, the HMD 22 includes a camera 16. In some embodiments, the camera 16 includes a red-green-blue (RGB) camera with an IR-pass filter, also referred to herein as an IR camera and also referred to herein as an IR tracker. In other embodiments, the camera 16 may include a monochrome camera configured to operate at IR wavelengths. The camera 16 is configured to capture images, including both the reflector 194 and the markers 38 and markers (not shown) attached to the patient 23. While the camera 16 in FIG. 1 is mounted on the HMD 22, these images may additionally or alternatively be captured by a suitable camera mounted on any other suitable location on the head or body of the surgeon 26, or in any suitable location within the operating room. The camera 16 is configured to generate signals indicative of the captured images and transmit those signals, also referred to herein as infrared (IR) images for embodiments incorporating an IR camera, to the processor 33. In such an embodiment, the processor 33 is configured to process the IR images acquired by the camera 16 to calculate the position and orientation of one or more surgical instruments (e.g., surgical instrument 190) and the HMD 22 tracking system reference point relative to the treatment area or location of the organ, bone, joint, or other target of interest.

[0159] In some embodiments, tracking applications based on images generated by camera 16 require monochromatic images. In some embodiments, camera 16 includes a color image sensor. The addition of color in the tracking images can, at least in some instances, degrade image quality due to de-Bayering interpolation applied to the color pixels to generate a continuous image based on the separated RGB pixels (two red, one green, and one blue) of the Bayer filter of the RGB image sensor.

[0160] In some embodiments, camera 16 includes a small sensor (such as a sensor designed for consumer products) with a color array filter (CAF), also known as a Bayer filter, that provides a unique response for each of the different color channels. By adding an external bandpass filter, the raw pixel data received from camera 16 can be treated as monochrome data.

[0161] In some embodiments, a band-pass filter (BPF) is applied in a selected portion of the infrared zone (e.g., from about 830 nm to 870 nm, or any other suitable range of wavelengths within the infrared spectrum).

[0162] In some embodiments, the processor 33 (or any other controller) is configured to apply a respective single gain value to each channel to counteract the effect of the Bayer filter on the passband of the filter.

[0163] In some embodiments, the basic configuration of the HMD 22 includes the aforementioned processor 33, a communication device (wireless and / or wired), a camera 16 (e.g., an IR camera), a projector 17 (e.g., an IR projector), a display 15, and an optical engine with a projector on the display 15 (shown in FIG. 2A below).

[0164] In some embodiments, the HMD 22 includes additional components in addition to the basic configuration. For example, the HMD 22 includes an inertial measurement unit (IMU) 18 configured to generate position signals indicative of the position and orientation of the HMD 22 at frequency levels between approximately 1 Hz and 10 kHz (e.g., 1 Hz to 50 Hz, 1 Hz to 200 Hz, 50 Hz to 250 Hz, 100 Hz to 200 Hz, 50 Hz to 1 kHz, 100 Hz to 10 kHz, overlapping ranges thereof, or any value within the recited ranges). Based on the position signals received from the IMU 18, the processor 33 can be configured to, for example, improve the response time of the system 11 to any relative motion between the HMD 22 and the organ of interest (or other target anatomical structure or region) of the patient 23.

[0165] In some embodiments, the IMU 18 is configured to operate in conjunction with the camera 16 (typically operating at a frequency of about 60 frames per second or any other suitable frequency corresponding to any other suitable number of frames per second) and provide the position and orientation of the HMD 22 to the processor 33 with reduced latency compared to images received from the camera 16. In such embodiments, the position and orientation of the HMD 22 can be calculated with the reduced latency achieved by using the IMU 18. Furthermore, if the optical path between the camera 16 and one or more of the markers is blocked, the processor 33 may rely on signals from the IMU 18 to calculate the position of the HMD 22 relative to the organ (or other anatomical structure or region of target therapy) of interest.

[0166] In some embodiments, the processor 33 is configured to perform alignment between the coordinate systems of the IMU 18 and the camera 16 and synchronization between the signals received from the IMU 18 and the camera 16.

[0167] In some embodiments, the HMD 22 includes one or more additional cameras 25 (e.g., a pair of red-green-blue (RGB) cameras). Each additional camera 25 (e.g., an RGB camera) is configured to generate a high-resolution (HR) image (e.g., an HR RGB image) of the organ (or other anatomical structure or region of target treatment) under surgery, and the processor 33 is configured to display the HR image on the display 15 of the HMD 22.

[0168] Additionally, the HMD 22 includes a pair of additional cameras 25 positioned at a known distance from each other so that the processor 33 is configured to generate a stereoscopic 3D image of the surgical site (e.g., an organ of interest or other anatomical structure or region of target treatment). Such technology is also referred to herein as a digital magnifier for augmented reality near-eye displays and is described in more detail, for example, in U.S. Provisional Patent Application No. 63 / 234,272 and International Publication No. WO 2023 / 021450, the disclosures of both of which are incorporated herein by reference.

[0169] In other embodiments, the HMD 22 may include any other suitable number of additional cameras 25 having similar characteristics to the RGB camera, or at least one of the additional cameras 25 may have different characteristics compared to the RGB camera.

[0170] In some embodiments, each additional camera 25 is further configured to acquire an image including a structured light pattern directed by the surgeon 26 toward the surgical site, and based on the image of the structured light pattern, the processor 33 is configured to improve the accuracy of the 3D imaging of the surgical site.

[0171] In some embodiments, the structured light projector is configured to direct a large number of beams (e.g., hundreds or thousands) onto the organ (or other anatomical structure or region of interest) to enable 3D imaging (and most importantly, depth imaging) of the organ (or other anatomical structure or region of target treatment). In some embodiments, the wavelength of the structured light should be suitable for generating spots on the skin and / or internal tissue during surgery. In some examples, the structured light may include a green laser beam, a blue laser beam, a red laser beam, an infrared laser beam, or a beam of any other suitable wavelength or wavelength range.

[0172] In some examples, the structured light includes visible wavelengths (e.g., green), such that camera 25 generates an image of a green spot on the skin and / or on the surface of the tissue being operated on. Based on the images received from camera 25, processor 33 is configured to generate a 3D image of the organ being operated on (or other anatomical structure or region of target treatment). Structured light is described in more detail below in connection with FIG. 16.

[0173] In other examples, the structured light includes a beam having a non-visible wavelength, such as an infrared wavelength. In some embodiments, processor 33 is configured to generate one or more images of an IR spot on the surface of the organ of interest (or other anatomical structure or region of target treatment). The images may be two-dimensional (2D) or, typically, 3D, using, for example, images (e.g., IR images) acquired by camera 16 and / or using an additional IR camera that may be mounted on HMD 22 or at a known position in the operating room. 2D and 3D images may be generated by processor 33 using, mutatis mutandis, the same techniques described above for generating images with visible spots of structured light.

[0174] In some embodiments, the HMD 22 includes a housing 29 configured to package all of the components described above. In some embodiments, the housing 29 includes a surface 20 configured to receive a headlight assembly mounted thereon. Several types of headlight assemblies and techniques for mounting them to the surface 20 are described in detail below in connection with Figures 4-8 and 27A-28F.

[0175] In some embodiments, the HMD 22 includes nose pads 28 that are adjustable and configured to support the HMD 22 over the nose of the surgeon 26. Some embodiments relating to the structure and functionality of the nose pads 28 are described in detail below in connection with FIG.

[0176] In some embodiments, the HMD 22 includes one or more light sensors 19, also referred to herein as ambient light sensors (ALS), configured to generate one or more signals indicative of light in an ambient area surrounding the HMD 22. Based on the signals received from the light sensors 19, the processor 33 is configured to adjust the brightness of the AR images presented on the display 15. Additionally or alternatively, based on the signals received from the light sensors 19, the processor 33 is configured to adjust the power applied to a light source mounted to the HMD 22 and / or an illumination assembly 27 described herein.

[0177] In some embodiments, the HMD 22 includes an IR-based proximity sensor (not shown) configured to generate signals for various applications, such as, but not limited to, hand gesture tracking.

[0178] Referring again to the overall view of Figure 1, in addition to or instead of the headlight assembly described above, system 11 may include a lighting assembly 27 that is mounted in any suitable location in the operating room and configured to direct white light, or light of any other suitable wavelength or range of wavelengths, toward an intraoperative site in the organ of interest (or other target treatment anatomical structure or region) of patient 23.

[0179] During a surgical or other interventional procedure, the surgeon 26 wears the HMD 22, which is configured to present the surgeon 26 with captured and stored images as well as additional calculated information based on a tracking system aligned with the organ being operated on (or other anatomical structure or region of target treatment).

[0180] In some embodiments, in serving as a reference, the marker 38 serves two functions: first, the marker is used to maintain alignment between the reference frame of the HMD 22 and the patient's anatomy, and second, the marker is used to confirm where the head and line of sight of the surgeon 26 is located relative to the organ of interest (or other target anatomy or region of treatment) of the patient 23.

[0181] During the initial stages of the procedure, an alignment marker (not shown) is placed on the patient's back or other anatomical location and used to perform alignment of the patient markers 38 with the anatomical structures of the patient 23. In contrast to the patient markers 38, the alignment markers may in some embodiments only be used during the initial stages of the procedure, e.g., to align the patient markers 38, and once alignment has been achieved, the alignment markers may be removed from the patient's back or other anatomical location for subsequent stages of the procedure.

[0182] In some embodiments, system 11 includes a processing system 31 communicatively coupled by cable and / or wirelessly to HMD 22. In some embodiments, processing system 31 includes a computer processor 32, a storage device 37 containing stored images 35, a screen 34, and input devices 36 such as a pointing device, a mouse, a touchscreen input such as a touchpad or touchscreen display, or a keyboard.

[0183] In some embodiments, the processing system 31 is configured to analyze images captured by one or more cameras of the HMD 22 and present the aforementioned AR images to the surgeon 26 on the display 15 .

[0184] As noted above, the HMD 22 includes a processor 33 for performing at least the functions described above, although in alternative embodiments, during operation the HMD 22 is connected to the processor 32 of the processing system 31 to remotely perform these processing and display functions, and the AR imagery is displayed to the surgeon 26 on the display(s) 15. Any function or task described herein as being implemented by the processor 32 may also be implemented by the processor 33, or vice versa.

[0185] The configuration of HMD 22 is provided as an example and is simplified for conceptual clarity. In some embodiments, processor 32 and processor 33 can share and / or allocate processing tasks between processors 32, 33. Each of processors 32, 33 can consist essentially of one processor or multiple processors.

[0186] 2A is a schematic rear view of an HMD 22, according to an embodiment of the present disclosure. In the context of the present disclosure, the term "rear view" refers to the view of the HMD 22 as seen by the surgeon 26 while wearing the HMD 22.

[0187] In some embodiments, the HMD 22 includes two optical engines (OEs) 40, one for each eye. In some embodiments, each OE 40 includes an AR projector 42 configured to direct AR images, as described above in FIG. 1 , to a respective AR display 15. In one example, each OE 40 includes an OLED-based image source and display (as described above with reference to FIG. 1 ) configured to project AR images generated by the processor 33, and optics (e.g., one or more lenses and mirrors) configured to direct the AR images to the AR display 15.

[0188] In some embodiments, the AR display 15 is part of the visor 14, which is coated with one or more suitable layers configured to reflect the projected VR image into the pupil of each eye, thereby allowing the surgeon 26 to see the VR image overlaid on the scene of interest (e.g., the organ being operated on (or other anatomical structure or region of target treatment)) in an augmented vision, virtual overlay on the real world manner.

[0189] In some embodiments, the visor 14 is fully or partially transparent so that when the surgeon 26 looks away from the AR display 15, he or she can see the surrounding scene without the AR image superimposed on it.

[0190] In some embodiments, the HMD 22 comprises two temple arms as described herein and nose pads 28 for mechanically supporting the fit of the HMD 22 on the head of the surgeon 26. In the context of this disclosure and the claims, the term "temple arms" and grammatical variations thereof refer to portions of the frame of the HMD 22 (or any other suitable type of HMD) that are coupled to the housing 29, typically (but not necessarily) worn over the ears of a user (e.g., the surgeon 26), and positioned in contact with at least a portion of each temple of the surgeon's 26's head.

[0191] In some embodiments, the left temple arm 43 comprises a processor 33 and, optionally, other devices such as a wireless communication device 45 configured to exchange signals between the HMD 22 and an external entity, and a storage device 46 configured to store images, signals, program instructions, and additional data of the HMD 22. Note that the processor 33, wireless communication device 45, and storage device 46 are shown in dashed lines because they are embedded within the internal volume of the left temple arm 43.

[0192] In some embodiments, the processor 33, the wireless communication device 45, and the storage device 46 may be disposed on one or more suitable substrates, such as, for example, one or more printed circuit boards (PCBs).

[0193] In some embodiments, all of the devices are located on a single rigid PCB. In some embodiments, at least one of the PCBs may be flexible. Additional embodiments regarding suitable types of flexible PCBs are described below in connection with FIG. 14.

[0194] In some embodiments, the HMD 22 includes a right temple arm 44 that includes an on / off or standby button 39 configured to power the HMD 22 on when in use and to power the HMD 22 off when not in use.

[0195] In some embodiments, the temple arms 43, 44 are configured to adjust to the shape of the left and right temples of the surgeon 26 or any other user and to be worn on the ears of the surgeon 26 to support the weight of the HMD 22 (in one example, together with the nose pads described below). The structure and functionality of the temple arms 43, 44 are described in more detail below in connection with Figures 9-11.

[0196] In some embodiments, nose pads 28 are configured to adjust to the shape of the nose of surgeon 26 or any other user. The structure and functionality of nose pads 28, as well as embodiments relating to the combination of nose pads 28 with temple arms 43, 44, are described in more detail below in connection with FIG.

[0197] In the context of this disclosure and in the claims, the terms "frame" and "head-mounted assembly" are used interchangeably and may refer to the combination of two or more elements of the housing 29, nose pads 28, and temple arms 43, 44, including the combination of the head strap 740 and knob 744 of FIG. 2B or FIG. 21A (which together form an adjustable strap assembly), or any other suitable assembly configured to hold the HUD 700 or HMD 2122 in a selected position on the surgeon's 26's head.

[0198] In some embodiments, a power cable (not shown) is threaded through a power cable strain relief 47 of the HMD 22. In this configuration, the power cable strain relief 47 is attached to the right temple arm 44, and the power cable is configured to provide an electrical connection between a power source (not shown) and several components of the HMD 22, such as, but not limited to, the on / off button 39.

[0199] In some embodiments, the power source comprises a suitable battery pack and one or more supercapacitors or ultracapacitors (not shown). In some embodiments, the battery pack includes lithium-based batteries, such as, but not limited to, batteries manufactured by RRC Power Solutions GmbH (Hamburg, Germany).

[0200] In some embodiments, a supercapacitor or ultracapacitor can be used to reduce long startup times when changing battery packs. Instead of powering down the HMD 22, the processor 33 can be configured to control components of the HMD 22 to enter a low-current standby mode. By powering down all components and peripherals, current can be reduced to a minimum to allow the supercapacitor or ultracapacitor to maintain the state of the HMD 22 for a sufficiently long time interval between battery pack changes without requiring an additional battery for the standby mode.

[0201] This particular configuration of the HMD 22 is shown as an example to illustrate the particular problem addressed by embodiments of the present disclosure and to demonstrate the application of these embodiments in improving the performance of such systems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary system, and the principles described herein may likewise be applied to other types of HMDs used in appropriate types of AR-based image-guided surgical systems. In addition, the HMD 22 may be used in non-medical applications, including consumer and commercial applications.

[0202] In some embodiments, system 11 is not limited to only augmented reality systems and / or systems with one or more HMDs. For example, patient and / or instrument tracking may be performed using a fixed tracking system (other than an HMD), and the display may also be on a fixed display, which may or may not be displayed as augmented reality and may or may not be mounted on the head of a user of system 11 (e.g., surgeon 26).

[0203] In some embodiments, HMD 22 may include various types of image sources, such as, but not limited to, OLEDs and liquid crystal on silicon (shown and described below in connection with FIG. 3). These image sources may be combined with various types of optics (or optical devices), such as, but not limited to, visor-based, waveguide-based, and bird's-eye-view-based optical engines. For example, HMD 22 may include a combination of OLED and visor, or a combination of liquid crystal on silicon and a waveguide source. However, it is noted that all possible permutations of the image sources and optics described above may be applicable to HMD 22.

[0204] In the example of FIG. 1, HMD 22 and its components and assemblies are implemented using a projection / visor-based optical engine. In some embodiments, HMD 22 may be implemented using other technologies and configurations, such as a waveguide-based optical engine. Additionally, instead of glasses, HMD 22 may comprise a helmet-shaped headset, also referred to herein as a head-up display (HUD), an exemplary configuration of which is described in more detail below in connection with FIG. 2B. In some embodiments, HMD 22 may include features of both the helmet-shaped headset of FIG. 2B and the version shown in FIG. 2A, as shown, for example, in FIGS. 21A-21D described below.

[0205] In some embodiments, the HMD 22, including all of the components described above in Figures 1 and 2A and some of the additional components (or variations thereof) described below in connection with Figures 5-14, weighs approximately 200 grams or less and has a suitable weight distribution to provide a high level of comfort for the surgeon 26 and to eliminate fatigue associated with the larger weight (e.g., approximately 0.5 kg to 1.5 kg) of other types of head-mounted devices.

[0206] 2B is a schematic, pictorial diagram of an exemplary head-up display (HUD) 700, according to one embodiment. HUD 700 also serves as and is referred to herein as an HMD, and may, for example, replace HMD 22 of FIG. 1 above.

[0207] In some embodiments, the HUD 700 comprises an optics housing 704 incorporating a camera 708. More specifically, the camera 708 may comprise an RGB camera configured as an IR camera using appropriate filters and software, or the camera 708 may comprise an infrared camera or an RGB-IR camera. In some embodiments, the housing 704 comprises an infrared-transparent window 712, and one or more infrared projectors 716 are mounted within the housing, e.g., behind the window.

[0208] In some embodiments, the HUD 700 comprises a pair of AR displays 720 mounted to the housing 704. In some embodiments, the displays 720 may comprise optical couplers, waveguides, or visors, for example, as described in connection with FIGS. 1 and 2A above.

[0209] In some embodiments, the AR displays 720 allow the surgeon 26 to view entities such as part or all of a selected field of view (not shown) via the AR displays 720, which are also configured to present the surgeon with images or any other information that may be received from the processing system 31.

[0210] In some embodiments, the HUD 700 includes a processor 724 that operates the elements of the HUD 700 and is mounted within a processor housing 726. The processor 724 typically communicates with the processing system 31 via an antenna 728. In some embodiments, the processor 724 may perform some of the functions performed by the processing system 31. In some embodiments, the processor 724 may replace the processing system 31 entirely.

[0211] In some embodiments, the HUD 700 includes a flashlight 732 mounted on the front of the HUD 700. The flashlight 732 is configured to direct a beam of visible spectrum light (e.g., wavelengths between about 350 nm and 800 nm or between about 300 nm and 900 nm) at a selected object so that the surgeon 26 or other wearer can clearly see the object through the display 720.

[0212] In some embodiments, the HUD 700 includes a power source (e.g., a battery (not shown)) configured to provide power to some elements of the HUD 700 via a battery cable input 736. The power source may additionally or alternatively include one or more capacitors, supercapacitors, or ultracapacitors.

[0213] In some embodiments, the HUD 700 is held and gripped in place on the head of the surgeon 26 using a head strap 740, which includes a knob 744 that the surgeon 26 can use to adjust the head strap of the HUD 700. The head strap 740 and knob 744 together may be referred to as an adjustable strap assembly.

[0214] In some embodiments, the HUD 700 may include additional components, such as, but not limited to, the components described above in Figures 1 and 2A. For example, the HUD 700 may include an IMU 18 configured to generate position signals indicative of the location and orientation of a tracking system reference point / origin of the HUD 700. Additionally, the HUD 700 may include an IR LED projector, such as projector 17 of the HMD 22.

[0215] Additionally or alternatively, the flashlight 732 of the HUD 700 may be coupled to the housing 704 using a suitable detachable lighting fixture assembly (DLFA) configured to be attached to and detached from the housing 704 and / or an upper bridge (not shown) of the HUD 700, or any other suitable location on the HUD 700. Removing the flashlight 732 reduces weight from the HUD 700, which may also be done if the lighting assembly 27 of FIG. 1 above directs sufficient light to perform a surgical procedure. Non-limiting examples of detachable lighting fixture assemblies are described in more detail below in connection with FIGS. 4-8 and 27A-28F.

[0216] This particular configuration of HUD 700 is shown as an example to illustrate the particular problems addressed by certain embodiments and to demonstrate the application of these embodiments in improving the performance of such systems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary HMD, and the principles described herein may likewise be applied to other types of head-mounted displays, head-up displays, used in appropriate types of AR-based image-guided surgical systems. For example, additional features of head-mounted displays or head-up displays are described in detail in, for example, U.S. Patent Application Publication No. 2017 / 0178375, which is incorporated herein by reference.

[0217] 3 is a schematic, pictorial diagram of an HMD 50 comprising a liquid crystal-based optical engine (OE) 55 and a closed-loop control assembly 59 configured to control light intensity within the OE 55, according to one embodiment of the present disclosure. HMD 50 may, for example, replace HMD 22 of FIG. 1 above.

[0218] In some embodiments, the HMD 50 includes a waveguide-based OE 55 that includes a backlight source 54. In some embodiments, the backlight source 54 includes one or more LEDs configured to provide visible light through a waveguide 52 that is coated with an opaque covering to prevent, or at least reduce, the amount of stray light, which is light leakage of backlight photons. More specifically, the backlight source 54 may include red, green, and blue (RGB) LEDs and may be configured to emit white light generated by combining the light of the RGB LEDs.

[0219] In some embodiments, the backlight passes through a light encoding device (e.g., a liquid crystal on silicon (LCOS) device 51) configured to modulate the backlight with information encoded by the processor 33. For example, in response to receiving a signal indicating an encoded slice of a computerized tomography (CT) image, the LCOS device 51 is configured to modulate the backlight to generate an image of the CT slice that is presented on the display 49a of the HMD 50.

[0220] In some embodiments, OE 55 includes a light sensor 53 configured to measure stray light between backlight source 54 and LCOS 51 without interfering with the operation of OE 55. In other words, in some embodiments, light sensor 53 is not positioned along the optical axis of the backlight to be modulated and uses the stray light to measure the intensity of the backlight emitted from backlight source 54 into waveguide 52.

[0221] Reference is now made to inset 57, which illustrates a block diagram of closed-loop control assembly 59. In some embodiments, in response to sensing the intensity of backlight emitted from backlight source 54, light sensor 53 is configured to generate a signal indicative of the measured intensity of the backlight.

[0222] In some embodiments, based on the signal received from the light sensor 53, the processor 33 is configured to control the backlight driver 58 to adjust the current applied to the RGB LEDs of the backlight source 54.

[0223] While it is possible in principle to control the current supplied to a backlight source, due to the non-uniform response of the LEDs of any light source (such as backlight source 54) (even from the same batch of LEDs), the intensity of the backlight may be non-uniform and vary. More specifically, (i) the backlight may vary over time within the same backlight source 54, for example, as the LEDs age and / or in response to changes in temperature of the components surrounding the LEDs, (ii) the backlight may vary between different backlight sources 54 of different respective OEs 55 (e.g., between the left and right OEs 55 of an HMD 50), (iii) the backlight may vary between OEs of different HMDs 50, or (iv) any combination thereof.

[0224] In other words, the processor 33, or any suitable dedicated circuitry, controls the drivers to adjust the current supplied to each of the LEDs of the backlight source 54 so as to keep the light level constant. A sequential strobe scheme is used so that, in one embodiment, a single light sensor 53 may be sufficient to control the light emitted in all three color channels (RGB).

[0225] Thus, in one embodiment, controlling the backlight based on direct off-axis measurement of stray light of OE 55 improves the brightness uniformity of the AR image presented on AR display 49a.

[0226] Referring again to the overall view of Figure 3, in some embodiments, the HMD 50 includes an additional AR display 49b configured to present an additional AR image to the left eye of the surgeon 26 by applying the same technique to a similar optical engine (not shown) mounted to a section 56 (e.g., a bracket) of the left temple arm (not shown) of the HMD 50. This implementation may also be incorporated into other HMDS and / or HUDs described herein.

[0227] In some embodiments, processor 33 is configured to present different AR images on AR display 49a and AR display 49b to display to surgeon 26 an image, such as a stereoscopic image (e.g., a 3D CT image) of the organ (or other anatomical structure or region of target treatment) being operated on.

[0228] In some embodiments, the HMD 50 includes an adapter 48 formed within the frame 41 of the HMD 50 and adapted to fit onto the HMD 50, and nose pads of a suitable type, such as nose pads 28 shown in FIG. 2A above and described in more detail below in connection with FIG. 12. Additionally, the temple arms of the HMD 50 extend from section 56 and may have a configuration similar to that of FIG. 2A above. Embodiments of some configurations of the temple arms of the HMDs and HUDs of the present disclosure are described in detail below in connection with FIGS. 9-11.

[0229] This particular configuration of HMD 50 is shown as an example to illustrate the particular problem addressed by certain embodiments and to demonstrate the application of these embodiments in improving the performance of such systems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary HMD configuration, and the principles described herein may likewise be applied to other types of HMDs and HUDs used in any suitable type of near-eye display AR-based image-guided surgical system. In addition, HMD 50 may be used in other medical or non-medical systems, including consumer or commercial applications.

[0230] Headlight Assembly Example FIG. 4 is a schematic, pictorial diagram of a headlight assembly (HA) 60 for use with any of the HMDs and HUDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32, according to various embodiments.

[0231] In some embodiments, HA 60 includes a flashlight 61 that may have similar features to flashlight 732 of FIG. 2B above, or any other flashlight disclosed herein.

[0232] In some embodiments, the HA 60 comprises a removable light fixture assembly (DLFA) 66 adapted to attach the flashlight 61 to the surface 20 of the housing 29 and to remove the flashlight 61 from the surface 20 .

[0233] Reference is now made to insets 62, 63, and 64. Inset 62 shows DLFA 66 without flashlight 61. In some embodiments (as shown in inset 63), DLFA 66 includes one or more clips 65 (e.g., one clip, two clips, three clips, or four or more clips) configured to (i) attach DLFA 66 to a base 67 located on surface 20 when DLFA 66 (and flashlight 61) is moved toward surface 20, and (ii) detach DLFA 66 from base 67 when clip(s) 65 are pushed toward the interior volume of DLFA 66.

[0234] In some embodiments (as shown in inset 64), base 67 includes electrical connections 68 (e.g., two or more vertical pogo pins, three pogo pins, or four or more pogo pins) configured to transmit power and / or signals or data between housing 29 and flashlight 61.

[0235] FIG. 5 is a schematic, pictorial diagram of a headlight assembly (HA) 70 for use with any of the HMDs and HUDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32, according to some embodiments.

[0236] In some embodiments, HA70 includes a flashlight (not shown) that may have similar features of flashlight 61 of FIG. 4 above and / or flashlight 732 of FIG. 2B above, and / or any other flashlight disclosed herein.

[0237] Reference is now made to inset 72. In some embodiments, HA 70 comprises a DLFA 71 adapted to attach a flashlight to surface 20 of housing 29 and to detach the flashlight from surface 20.

[0238] In some embodiments, the DLFA 71 includes one or more clips 73 (e.g., one clip, two clips, three clips, or four or more clips) configured to (i) attach the DLFA 71 to a base 74 located on the surface 20 when the DLFA 71 is moved in direction 76 (e.g., away from the forehead of the surgeon 26), and (ii) detach the DLFA 71 from the base 74 when the (one or more) clips 73 are pushed toward the base 74 while the DLFA 71 is simultaneously moved in direction 75 (e.g., toward the forehead of the surgeon 26).

[0239] In some embodiments, the base 74 includes electrical connections 68 (e.g., two or more horizontal pogo pins, three pogo pins, or four or more pogo pins) configured to transmit power and / or signals or data between the housing 29 and the flashlight described above.

[0240] FIG. 6 is a schematic, pictorial diagram of a headlight assembly (HA) 80 for use with any of the HMDs and HUDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32, according to various embodiments.

[0241] In some embodiments, HA80 includes a flashlight (not shown) that may have similar features to flashlight 61 of FIG. 4 above and flashlight 732 of FIG. 2B above, and / or any other flashlight disclosed herein.

[0242] Reference is now made to inset 81. In some embodiments, HA 80 comprises a DLFA 82 adapted to attach a flashlight to surface 20 of housing 29 and to detach the flashlight from surface 20.

[0243] In some embodiments, a base 86 formed on surface 20 includes a trench 85 configured to receive a DLFA 82 as described herein.

[0244] In some embodiments, the DLFA 82 has two leads 84 (one on each side of the DLFA 82), which are configured to (i) slide through the trench 85 along direction 75 to attach the DLFA 82 to the base 86, and (ii) slide through the trench 85 along direction 76 to remove the DLFA 82 from the base 86.

[0245] In some embodiments, the DLFA 82 includes a handle 83 for moving the DLFA 82 in directions 75 and 76, and one or more clips configured to attach and detach the DLFA 82 in coordination with movement in directions 75 and 76.

[0246] In some embodiments, the housing 29 includes electrical connections (e.g., one or more vertical or horizontal pogo pins (not shown)) configured to carry power and / or communication signals or data between the housing 29 and the above-described flashlight connected to the DLFA 82.

[0247] FIG. 7 is a schematic, pictorial diagram of a headlight assembly (HA) 90 for use with any of the HMDs and HUDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32, according to various embodiments.

[0248] In some embodiments, HA90 includes a flashlight (not shown) that may have similar features to flashlight 61 of FIG. 4 above, flashlight 732 of FIG. 2B above, and / or any other flashlight disclosed herein.

[0249] Reference is now made to inset 91. In some embodiments, HA 90 comprises a DLFA 92 adapted to mount and remove a flashlight from a base 93 coupled to surface 20 of housing 29.

[0250] In some embodiments, inset 91 shows how DLFA 92 is attached to and detached from base 93 when moved in directions 75 and 76, respectively.

[0251] Referring again to the overall view of Figure 7, in some embodiments, the base 93 comprises leads 94 configured to slide along trenches (not shown) formed in the underside of the DLFA 92 facing the base 93. The base 93 further comprises two fences 95 on the sides of the base.

[0252] In some embodiments, the base 93 includes electrical connections (e.g., two or more vertical and / or horizontal pogo pins (not shown)) configured to conduct power and / or communication signals or data between the housing 29 and the above-described flashlight connected to the DLFA 92.

[0253] Referring again to inset 91, in some embodiments, DLFA 92 comprises two pairs of flexible fins 96 (one pair on each side of DLFA 92) that partially surround an opening 97 configured to contain a fence 95 for attaching DLFA 92 to base 93.

[0254] In some embodiments, the configuration of at least some of the DLFA 92 and base 93 is similar to the design of GoPro® style mounts for cameras and other electronic accessories manufactured by GoPro Corporate (3025 Clearview Way, San Mateo, CA).

[0255] FIG. 8 is a schematic pictorial diagram of a headlight assembly (HA) 100 for use with any of the HMDs and HUDs shown in FIGS. 1, 2A, 2B, 3, 21A-29E, and 31A-32, according to various embodiments.

[0256] In some embodiments, the HA 100 comprises a flashlight 61 and a removable light fixture assembly (DLFA) 101 adapted to attach the flashlight 61 to the surface 20 of the housing 29 and to remove the flashlight 61 from the surface 20.

[0257] Reference is now made to insets 102 and 103. In some embodiments, DLFA 101 comprises a hole 104 adapted to contain an axis (not shown) for controlling the elevation angle 105 of flashlight 61, for example, with respect to an imaginary plane parallel to surface 20 or any other suitable reference plane.

[0258] Referring to inset 103, in some embodiments, DLFA 101 comprises a base 106 configured to couple to surface 20 and provide a connection between DLFA 101 and surface 20 of housing 29.

[0259] Referring now to inset 107, in some embodiments, base 106 includes electrical connections 68 (e.g., two or more vertical or horizontal pogo pins) configured to carry power and / or signals or data between housing 29 and flashlight 61.

[0260] In some embodiments, base 106 includes a pair of magnets 109a, 109b, and DLFA 101 includes a pair of magnets 108a, 108b. When DLFA 101 is placed on base 106, magnets 108a and 109a attract each other, and similarly, magnets 108b and 109b attract each other.

[0261] In such an embodiment, attachment and detachment of the magnetic base between the DLFA 101 and the base 106 is quick and easy because it does not require mechanical release of clips or any other type of capturing and / or locking device. Thus, the surgeon 26 or medical staff member in the operating room can use one hand to attach and detach the DLFA 101 and flashlight 61 and subsequently adjust the angle 105 to direct the light beam to the area of ​​interest, e.g., the organ being operated on (or other anatomical structure or area of ​​target treatment).

[0262] In some embodiments, the DLFA 101 and base 106 may include three, four, five or more pairs of magnets 108, 109 to improve the stability of the magnetic-based coupling and prevent undesired rotation of the DLFA 101 relative to the base 106. According to some embodiments, the size, shape, and magnetic level of the magnets 108, 109, as well as the distance between each pair of magnets 108, 109, may define the stability of the magnetic-based coupling.

[0263] In some embodiments, a single pair of magnets 108, 109 may be sufficient to allow stability of the magnetic base coupling and prevent unwanted rotation of the DLFA 101 relative to the base 106.

[0264] These particular configurations of the HA and DLFA in Figures 4-8 described above (and Figures 27A-27D and 28A-28F described below) are shown as examples to illustrate the particular problems addressed by various embodiments and to demonstrate the application of these embodiments in improving the lighting performance of such HMDs and / or HUDs. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary configuration, and the principles described herein may likewise be applied to other types of headlight assemblies integrated with any suitable type of HMD and / or HUD used in near-eye display AR-based image-guided surgical procedures.

[0265] In some embodiments, the headlight assembly shown in Figures 4-8 above (and in Figures 27A-27D and 28A-28F described below) may be removed from the HMD and / or HUD configuration if the lighting assembly 27 of Figure 1 above provides sufficient light to perform a surgical procedure.

[0266] Inclined Assembly Example The embodiments relating to tilt assemblies described in detail below in connection with Figures 9, 10A, 10B, 11, 12, 13, 14, 15A, 15B, 15C, and 23B may be implemented, mutatis mutandis, in any of the HMDs and HUDs shown in Figures 1, 2A, 2B, 3, 21A-29E, and 31A-32, and the techniques described in connection with Figures 9-15C and 23B are also applicable to any other suitable configurations of AR-based glasses and / or helmets, as well as other types of AR-based near-eye head-mounted display assemblies, whether for medical or non-medical applications.

[0267] FIG. 9 is a schematic, pictorial illustration of a tilt assembly 111 (eg, a temple tilt assembly) according to one embodiment of the present disclosure.

[0268] In some embodiments, the tilt assembly 111 is configured to tilt the temple arms 44 relative to the housing 29. In other words, the tilt assembly 111 provides the surgeon 26 with a horizontal degree of freedom (DOF) to adjust the HMD 22 to the head shape of the surgeon 26 or the head shape of any other user of the HMD 22. The surgeon 26 can be replaced by a consumer for non-medical applications.

[0269] In some embodiments, the tilt assembly 111 may be implemented using a hinge (not shown), also referred to herein as an axis. In some embodiments, the tilt assembly 111 is implemented in a virtual hinge, also referred to herein as a virtual axis. The terms "virtual hinge" and "virtual axis" and grammatical variations thereof refer to tilting one object relative to another object without the use of a real, i.e., physical hinge or real, i.e., physical axis.

[0270] Reference is now made to inset 110, which is a top view of tilt assembly 111 incorporated into HMD 22.

[0271] 9, the virtual axis of tilt assembly 111 comprises three sections of a strap made of stainless steel or any other suitable material. More specifically, tilt assembly 111 comprises section 112 coupled to housing 29, section 113 coupled to temple arm 44, and section 114 configured to flex in response to movement of temple arm 44 in direction 117 or direction 118. For example, if a first surgeon has a smaller head than a second surgeon, tilt assembly 111 will allow movement of temple arm 44 and section 113 in (i) direction 117 when the first surgeon is wearing HMD 22 and (ii) direction 118 when the second surgeon is wearing HMD 22.

[0272] In an exemplary implementation of the virtual axis of tilt assembly 111, sections 112 and 113 are coupled to housing 29 and temple arm 44, respectively, using screws 115 and holes 116. Sections 112 and housing 29 move together as a rigid entity, and section 113 is moved in directions 117 and 118 by bending section 114.

[0273] In some embodiments, instead of using screws and holes, at least one of the tilt assemblies 111 of sections 112 and 113 may be connected to the respective portions of the HMD 22 using any other connection technique, such as, but not limited to, fastening with devices other than screws, adhesives, welding, and / or soldering.

[0274] Additionally or alternatively, at least one of section 112 and section 113 of tilt assembly 111 may be formed as an integral part of HMD 22. For example, section 112 may be formed in a single casting together with at least a portion of housing 29.

[0275] In some embodiments, tilt assembly 111 may include different materials used in at least two of the sections. For example, sections 112 and 113 may be made of stainless steel, while section 114 may be made of a softer, more flexible material, such as, but not limited to, a nickel-titanium alloy also known as Nitinol, or any other suitable material with suitable properties, flexibility within the elastic range over the required range of movement.

[0276] Reference is now made to inset 119, which is a side view of tilt assembly 111 incorporated into HMD 22. The example inset 119 shows the connection of sections 112 and 113 to their respective portions of HMD 22 using screws 115 and holes 116 (in some embodiments, it is contemplated that additional screws 115 may be inserted through holes 116).

[0277] In some embodiments, the techniques described for the tilt assembly 111 are also implemented between the temple arm 43 and the housing 29, mutatis mutandis.

[0278] In some embodiments, any other suitable type of tilt assembly may be implemented between the temple arm 43 and the housing 29.

[0279] FIG. 10A is a schematic, pictorial illustration of a tilt assembly 77, according to one embodiment.

[0280] In some embodiments, the tilt assembly 77 is implemented in the HMD 22, but may also be implemented in other types of HMDs.

[0281] In some embodiments, temple arm 44 comprises a section 121 configured to conform to the shape of the right side of the head of surgeon 26 or any other user of HMD 22. Temple arm 44 further comprises a section 122 configured to conform to the shape of the right side of the temporal and / or posterior portion of the human head, referred to herein as the nuchal region of surgeon 26 or any other user of HMD 22.

[0282] Reference is now made to inset 120, which is a side view of section 122 and tilt assembly 77 mounted to temple arm 44 to obtain additional DOF of adjustment between HMD 22 and the head of surgeon 26 or any other user of HMD 22.

[0283] In some embodiments, the tilt assembly 77 comprises a rotatable rocker arm 123, a hinge 124 connecting between the section 121 and the rocker arm 123, and a cushion 125 formed on the surface of the rocker arm 123.

[0284] Reference is now made to inset 131, which is a pictorial representation of the internal structure of tilt assembly 77.

[0285] In some embodiments, rocker arm 123 (which may be made from a polymer, including but not limited to, polycarbonate and / or polyoxymethylene (POM) and / or any other suitable material) has a proximal section 127 and a distal section 129 configured to rotate about hinge 124, and a spring 130 (e.g., a torsion spring).

[0286] In some embodiments, the rocker arm 123 is configured to rotate about a hinge 124 relative to the longitudinal axis 191 of the section 121 .

[0287] For example, when HMD 22 is attached to the head of the first surgeon mentioned above, section 129 moves from its resting state all the way in direction 128a to follow the ergonomic structure of the user's head, typically following slightly in direction 126a, and section 127 moves in direction 126b to follow the shape of the first surgeon's neck.

[0288] Similarly, when HMD 22 is attached to the head of a second surgeon (having a different head shape compared to the head shape of the first surgeon), section 129 moves in direction 126a and section 127 moves in direction 126b to conform to the shape of the back of the head, also referred to herein as the nuchal region of the second surgeon. Torsion spring 130 may be configured to reverse the direction of movement. In the illustrated example, torsion spring 130 moves section 127 in direction 128b to improve clamping between rocker arm 123 and the head of a user (e.g., the second surgeon). Furthermore, torsion spring 130 is configured to move section 127 to allow smooth insertion of HMD 22 onto the respective surgeon's head.

[0289] In some embodiments, section 121 has an opening or partial opening to accommodate section 127 when rotated in direction 126b.

[0290] Reference is now made to Figure 10B, which illustrates an alternative configuration of rocker arms shown in inset 120 and inset 131. In the example of Figure 10B, HMD 22 may include an array 198 of rocker arms 193 (e.g., two rocker arms 193a, 193b). At least one of rocker arms 193 may be similar to rocker arm 123 (and may be adapted to mate with array 198) or may include any other suitable design of rocker arm adapted to be incorporated into array 198.

[0291] 10B, the assembly of array 198 includes bar 196, also referred to herein as a common bar, configured to rotate about hinge 197 connecting section 121 and bar 196. Additionally, both rocker arms 193a and 193b are connected to hinges 193a and 193b, respectively, which are connected to bar 196. In one embodiment, the configuration of hinges 197, 193a, and 193b can be similar to the configuration of hinge 124 shown in insets 120 and 131 of FIG. 10A. In another embodiment, at least one of hinges 197, 193a, and 193b can have a configuration other than that of hinge 124.

[0292] Reference is now made to inset 132 of FIG. 10A, which shows cushion 125 of HMD 22.

[0293] In some embodiments, the cushion 125 is positioned over the entire surface of the rocker arm 123 that faces the surgeon's head.

[0294] In some embodiments, the cushion 125 may be molded onto the rocker arm 123 or may comprise a separate piece attached to the rocker arm 123. The cushion 125 may comprise a wide variety of solid or spongy materials (e.g., silicone, neoprene, polyurethane, and / or other suitable materials).

[0295] In some embodiments, the sponge may comprise closed cells that do not absorb fluids (eg, sweat), or open cells that are adapted to absorb fluids.

[0296] In some embodiments, cushion 125 comprises a viscoelastic foam, also known as "memory foam," to provide a soft cushion.

[0297] In some embodiments, when a heavy object is placed on the viscoelastic foam, the foam gradually conforms to the shape of the object, and after the weight (i.e., force) is removed, the foam slowly returns to its original shape.

[0298] In some embodiments, when cushion 125 uses a viscoelastic material, a human body temperature of approximately 36°C to 37°C accelerates the memory foam properties described above.

[0299] According to some embodiments, the soft cushion and viscoelastic foam are adapted to create an even distribution of pressure on the head of a surgeon using the HMD 22, thereby enabling effective clamping between the rocker arms 123 and the head while also maintaining comfort for the user (e.g., surgeon 26). Furthermore, the clamping effect may be beneficial for safety reasons, to prevent the HMD 22 from falling off the head during surgery.

[0300] In some embodiments, HMD 22 comprises two or more DOF achieved by using tilt assembly 111 and tilt assembly 77, the two-part shape of rocker arm 123, and the selected material of cushion 125. According to some embodiments, increasing the number of DOF improves grip and alignment between the frame of HMD 22 (e.g., temple arms 43, 44, and housing 29) and the contour and / or curvature of the surgeon's head performing the surgical procedure.

[0301] In some embodiments, the outer surface of cushion 125 has a texture suitable for improving grip against the nape of the head of surgeon 26. For example, the outer surface of cushion 125 (intended to be placed in contact with the nape) may include grooves, as shown for example in inset 132.

[0302] FIG. 11 is a schematic, pictorial illustration of a tilt assembly 88, according to one embodiment.

[0303] In some embodiments, tilt assembly 88 is implemented in HMD 22, but may also be implemented in other types of HMDs, such as other HMDs described herein. Additionally, tilt assembly 88 may be used in place of tilt assembly 77 in Figure 10A above, with each section having number 137 instead of number 122 of the corresponding section shown in Figure 10A above.

[0304] Reference is now made to inset 135, which shows a side view of tilt assembly 88 in the XYZ coordinate system.

[0305] In some embodiments, tilt assembly 88 includes a skeleton 136 that may be made from a suitable metal (e.g., aluminum 5052 H32 (Beachwood, Ohio) supplied by Aleris International Inc. or other aluminum or metal or metal alloy material or polymer or elastomeric material) adapted to be shaped along at least the XY plane of an XYZ coordinate system. In some embodiments, skeleton 136 may also be shaped to some extent along the Z axis (although in some implementations this shaping is not necessary).

[0306] In some embodiments, the tilt assembly 88 comprises a liner 138 that may be placed over and coupled to the backbone 136. In some embodiments, the liner 138 comprises an overmolded cushion having similar properties and materials (e.g., viscoelastic foam) as the cushion 125 of FIG. 10A above. In some implementations, the section 139 of the temple arm 43 comprises a structure similar to the backbone and liner of the section 137, mutatis mutandis.

[0307] In some embodiments, the metal of the scaffold 136 is adapted to convert from elastic deformation (the scaffold returns to its original shape in response to an applied force and small deformation) to plastic deformation (the scaffold undergoes larger deformation and retains the resulting shape in response to an applied force).

[0308] For example, referring to the overall view of Figure 11, if the minimum distance 140 between the edges of sections 137 and 139 of temple arms 44 and 43, respectively, is smaller than the size of the nape of surgeon 26, the anatomy of one or both of sections 137 and 139 can be deformed and shaped so that the modified (adjusted) distance 140 matches the corresponding size of the nape of surgeon 26. Furthermore, in this configuration, linings 138 of sections 137 and 139 fit snugly against the respective sections of the nape of surgeon 26.

[0309] Reference is now made to inset 135. In some embodiments, lining 138 of sections 137 and 139 is provided with a suitable texture to improve grip on the nape of the head of surgeon 26. For example, the outer surface of lining 138 (intended to be placed in contact with the nape) may be provided with grooves having a suitable orientation (e.g., parallel to the XY plane and / or parallel to the XZ plane) or parallel to the longitudinal axes of sections 137 and 139 of temple arms 44 and 43, respectively. In some embodiments, lining 138 may have any other suitable texture, other than grooves, adapted to improve grip between the nape section of surgeon 26 and sections 137 and 139 of HMD 22.

[0310] FIG. 12 is a schematic, pictorial illustration of nose pads 28 according to one embodiment.

[0311] In some embodiments, nose pads 28 are implemented in HMD 22, however, in some embodiments, nose pads 28 may also be implemented in other types of HMDs, including any of the HMDs disclosed herein, mutatis mutandis.

[0312] Reference is now made to the head of the surgeon 26 to show the position of the nose pad 28 on the nose 141 of the surgeon 26 .

[0313] Reference is now made to inset 148, which shows a higher magnification of nose pad 28 positioned over a portion of nose 141. In the example of inset 148, nose pad 28 is presented with a polymer lining 151 that is partially transparent for purposes of presentation (e.g., a viscoelastic material such as, but not limited to, the viscoelastic material of cushion 125 of FIG. 10A above, or any other suitable type of elastomer (e.g., silicone-based polymer), neoprene, polyurethane, or any other suitable material).

[0314] In some embodiments, the nose pads comprise a metal-based skeleton 150 surrounded by a lining 151 and which may comprise similar materials and functionality as the skeleton 136 shown in FIG. 11 above.

[0315] Nose 141 has an anterior portion 146, the cartilage and nerves of which may be sensitive to constant contact with nose pads 28. However, skin 147 on both sides of nose 141 is not in close proximity to cartilage and is therefore less sensitive to contact with nose pads 28.

[0316] Reference is now made to inset 149, which shows a front view of nose pads 28. Note that in the overall view of HMD 22, nose pads 28 and HMD 22 are shown from the rear, opposite to the front view of inset 149.

[0317] In some embodiments, nose pad 28 comprises a left section 142, a right section 143, and a middle section 144 connecting the left and right sections. In some embodiments, middle section 144 has an opening, while in some embodiments, middle section 144 can have a solid structure without an opening.

[0318] In some embodiments, the nose pads include a section 145 adapted to couple between the nose pads 28 and the frame of the HMD 22. In one embodiment, the section 145 may have a DOF that is adjusted relative to the frame of the HMD 22. The DOF may be implemented using any suitable type of tilt assembly, which may be hinged or based on a vertical axis, for example, as described in detail in any of FIGS. 9, 10A, 10B, and 11 above.

[0319] The DOF implemented in section 145 may operate synchronously or asynchronously with an additional DOF (also referred to herein as a vertical DOF or pantoscopic tilt assembly, described in more detail below in connection with FIG. 14) between the frame and the AR display of HMD22.

[0320] In some embodiments, section 145 may be rigidly coupled to the frame of HMD 22 without any degree of freedom to tilt relative to the frame.

[0321] In one implementation, nose pad 28 includes skeleton 150 arranged in sections 142 , 143 , and 145 to adjust the shape of nose pad 28 to the shape of nose 141 .

[0322] Referring again to inset 148, in some embodiments, nose pad 28 is adapted to be shaped so that section 144 of nose pad 28 does not come into direct contact with front portion 146 of nose 141, and in some embodiments, gap 153 provides a buffer between section 144 and front portion 146.

[0323] In some embodiments, the surgeon may place section 144 directly above the anterior portion 146 of the nose.

[0324] In some embodiments, section 145 may include an assembly configured to adjust the height (e.g., along the Z axis) of nose pads 28 relative to the frame of HMD 22. The adjustment may be performed in steps over a predetermined range of movement, or may be performed continuously using one or more suitable assemblies mounted on or between nose pads 28 or a portion of the frame of HMD 22 (e.g., within housing 29).

[0325] In some embodiments, the liner 151 may have a suitable texture, such as, but not limited to, the textures described above for the cushion 125 and liner 138 of Figures 10A and 11, respectively.

[0326] In some embodiments, nose pad 28 may include two ball joints for sections 142 and 143, respectively, to provide the user with improved adjustment of the shape of nose pad 28 relative to the width of nose 141.

[0327] 13 and 14 are schematic, pictorial illustrations of pantoscopic tilt of optical engine (OE) 165 and pantoscopic tilt assembly (PTA) 155, according to various embodiments. Optical engine 165 may replace, for example, OE 42 of FIG. 2A above and / or OE 55 of FIG. 3 above.

[0328] In some embodiments, PTA 155 may be implemented in HMD 22 and HMD 50, HUD 700, and any other type of HMD assembly, mutatis mutandis, using the embodiments described below in connection with Figures 13 and 14 and / or in connection with Figures 21A-29E and 31A-32. Furthermore, the implementation of PTA 155 may be affected by several operational modes of a surgical procedure, for example, as shown and described below in connection with Figures 15A-15C. PTA 155 may be used, for example, to rotate the see-through display assembly relative to the frontal plane of a user, such as surgeon 26.

[0329] Reference is now made to FIG. 13, which is primarily used as an introduction to pantoscopic tilting and its optional implementation.

[0330] 13, the eye 158 of the surgeon 26 has an optical axis (OA) 160 (also referred to herein as the first optical axis) that extends from the center of the eye through the pupil 161 of the surgeon 26. An angle 168 (e.g., of about 25°) is defined between the horizontal line 159 and the OA 160.

[0331] In the example of FIG. 13, an optical axis 162 (also referred to herein as a second optical axis) of OE 165 is defined from an OE eye relief point 163 within OE 165.

[0332] Reference is now made to frame 164, which illustrates fixed pantoscopic tilt implemented with respective pantoscopic tilt angles of approximately 0° and approximately 12° applied to the upper and lower glasses of frame 164. In the example of frame 164, the pantoscopic tilt angle is defined between frame 169 (e.g., corresponding to AR display 15 of HMD 22) and glasses 167.

[0333] In some embodiments, in a near-eye display AR-based system, such as HMD 22 of system 11, the pantoscopic tilt is set to align the second optical axis with the first optical axis. In the example of Figure 13, the pantoscopic tilt is set to convergently align optical axis 162 with optical axis 160.

[0334] Reference is now made to FIG. 14, which illustrates the DOF in the vertical axis implemented using an example vertical tilt assembly, referred to herein as a pantoscopic tilt assembly (PTA) 155, 156.

[0335] In some embodiments, a pantoscopic tilt angle 178 of approximately 35° is applied to HMD 22 using PTA 155 and PTA 156. However, other pantoscopic tilt angles may be applied (e.g., an angle between 25 and 45 degrees, an angle between 30 and 40 degrees, an angle between 33 and 37 degrees, overlapping ranges thereof, or any value within the recited ranges). Reference is now made to inset 170, which illustrates the vertical DOF implemented in PTA 155.

[0336] In some embodiments, the PTA 155 comprises a bar 172 rigidly coupled to the optical engine 165 and a hinge 171 configured to rotate the bar 172 relative to the temple arms 44 of the frame of the HMD 22. According to some embodiments, the movement is relative movement between the optical engine and the frame about a vertical axis.

[0337] In one non-limiting example, the frame of the HMD 22, more specifically the housing 29, temple arms 43, 44, and nose pads 28, are not moved when the PTA 155 tilts the OE 165 to the desired pantoscopic tilt angle at angle 178, and similarly to the frame when the PTA 156 (described herein) tilts the OE 165 to the desired pantoscopic tilt angle at angle 178.

[0338] In some embodiments, OE165 may comprise an optical assembly comprising one or more cameras, one or more light sources, and other components, all moved according to the DOF implemented using PTA155 of PTA156.

[0339] Reference is now made to inset 175, which illustrates a vertical DOF implemented in PTA 156. In some embodiments, PTA 156 comprises a bar 172 coupled to (e.g., molded with) a rotatable section 179 of a disk having a slit 174. PTA 156 further comprises an assembly 177, whereby when OE 165 is tilted relative to temple arms 44 of the frame, section 179 is moved in a selected tangential direction 176 relative to assembly 177 and locked into place within slit 174 (e.g., using locking element 173) to secure OE 165 at a desired pantoscopic angle relative to temple arms 44 and other components of the frame of HMD 22. Implementations of a vertical DOF using PTA 156 are also referred to herein as virtual axes, since components are moved (e.g., rotated about an axis) without the use of a physical hinge, such as hinge 171 of PTA 155.

[0340] In some embodiments, PTA 156 may comprise any other suitable type of virtual axis. For example, referring to inset 110 of FIG. 9 above, PTA 156 may comprise three sections, such as section 112, section 113, and section 114, referred to herein as the first section, second section, and third section, respectively, shaped to fit the frame and OE 165. In such embodiments, the first section is coupled to OE 165, the second section is coupled to the frame (e.g., to a respective temple arm on each side or to housing 29), and the third section is adapted to flex in response to relative movement between the first and second sections when surgeon 26 or any other user adjusts the pantoscopic tilt angle.

[0341] In some embodiments, the PTA 156 may comprise a rigid bar coupled to a frame and flexible arms having properties similar to the skeleton 136 shown and described above in inset 135 of FIG. 11. In such embodiments, in response to the surgeon 26 adjusting the pantoscopic tilt angle, the flexible arms are adapted to convert from elastic to plastic deformation and retain the shape obtained in response to the force applied to adjust the pantoscopic tilt angle.

[0342] These particular configurations of PTAs 155, 156 are shown by way of example to illustrate the particular problems addressed in the exemplary implementation of the vertical DOF for controlling the pantoscopic angle of the optical engine relative to the frame of the HMD 22. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary configuration and implementation, and the principles described herein may similarly be implemented in other types of vertical DOF used to control the pantoscopic tilt angle in near-eye display AR-based image-guided surgery systems (including, but not limited to, the configurations described below with reference to FIG. 23B ). Embodiments may also be used in non-medical applications, such as commercial and / or consumer applications, including athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, etc.

[0343] In some embodiments, optical engine 165 typically comprises an electronic device configured to exchange electrical signals with processor 33 (shown in FIG. 1) and other components of HMD 22 and system 11.

[0344] In some embodiments, the HMD 22 includes hardware configured to exchange signals while implementing the vertical DOF using the PTAs 155, 156.

[0345] Reference is now made to inset 180 and inset 186, which illustrate two exemplary implementations of hardware configured to exchange signals while applying pantoscopic tilt to OE 165. In some embodiments, a rigid PCB 181 with mounted electronic devices is located on a frame of HMD 22 (e.g., in housing 29 of FIG. 1 ), a rigid PCB 182 with mounted other electronic devices is located on OE 165, and a flexible PCB 183 is configured to flex in at least two axes to enable smooth exchange of electrical signals between housing 29 and OE 165.

[0346] In the configuration of inset 186, the flexible PCB 184 has an opening 185 shaped as a slit along an axis of the flexible PCB 184 to allow bending of the flexible PCB 184 along two or more axes, thereby allowing exchange of electrical signals between the housing 29 and the OE 165.

[0347] The rigid PCB and flexible PCB configurations of inset 180 and inset 186 are shown by way of example, and in some embodiments, any other suitable configuration may be used to enable the exchange of electrical signals between housing 29 and OE 165 while performing pantoscopic tilt, not just when HMD 22 is operating at a preset pantoscopic tilt angle.

[0348] In some embodiments, the glasses of the display (e.g., displays 49a, 49b in FIG. 3 above) may have a separate frame (e.g., display assembly frame 41) separate from the optical frame comprising the OE, camera, sensors, and other devices. In such embodiments, frame 41 remains stationary relative to the head (ears and nose), and the optical frame has a vertical DOF relative frame 41 (based on a hinge or virtual axis) around the center of the surgeon's eyeball.

[0349] 15A, 15B, and 15C are schematic, pictorial illustrations of use cases for PTA 155 and PTA 156 used in several types of surgical procedures (e.g., in both seated and standing positions), according to some embodiments. PTA 2355, described below with reference to FIG. 23B, may also be used in such procedures. These illustrations serve to illustrate, among other things, the particular advantages or benefits of including adjustable pantoscopic tilt, such as allowing PTAs 155, 156, 2355, etc., to be set to different pantoscopic tilt angles for different types of surgical procedures, each of which may utilize a different line of sight (LOS) relative to an area of ​​interest (e.g., a surgical site on a patient's body), according to some embodiments.

[0350] Reference is now made to Figures 15A and 15B, which illustrate spinal surgery based on a lateral approach.

[0351] A patient's or surgeon's line of sight (LOS) to a region of interest during a medical intervention (e.g., a surgical site on a patient's body) is typically, for example, vertical or diagonally downward (e.g., because the patient or surgical site is located below the surgeon's eyes). However, in some cases, the surgeon may be able to see the patient or surgical site, or the surgeon's LOS during a medical intervention may be different from downward, for example, horizontal or substantially horizontal or straight ahead (e.g., when the surgical site or region of interest is located in front of or substantially in front of the surgeon's eyes). In such cases, the HMD should enable a horizontal or substantially horizontal view and / or an augmented reality view of the region of interest.

[0352] For example, a lateral approach may be required in a lateral pathway interbody fusion (LLIF) procedure. In such a procedure, the patient 23 may be positioned on their side as shown in FIG. 15A, and the surgeon (shown in FIG. 15B) takes a lateral approach and makes an incision 187 centrally over the flank of the patient 23. The surgeon shown in FIG. 15B is in a seated position, thus providing a substantially horizontal view of the surgical site.

[0353] Using a lateral approach, the surgeon, in one embodiment, inserts surgical instruments from a lateral trajectory, thus being able to reach the vertebrae and disc without displacing nerves or opening back muscles.

[0354] In some embodiments, the surgeon in FIG. 15B uses an HMD 188 with a PTA 155 or PTA 156 described in FIG. 14 above (or a PTA 2355 described below with reference to FIG. 23B). In some embodiments, the PTA of the HMD 188 is configured to set any appropriate pantoscopic tilt angle. In the lateral approach example of FIG. 15B, the surgeon 99a may select a first pantoscopic tilt angle (e.g., an angle of about 15° or another angle, e.g., 5°-30°, 10°-25°, 10°-20°, overlapping ranges thereof, or any value within the recited ranges), and the surgeon 99b may select a different pantoscopic tilt angle (e.g., an angle of about 30° or another angle, e.g., 20°-40°, 25°-35°, 30°-40°, 25°-40°, 30°-50°, overlapping ranges thereof, or any value within the recited ranges).

[0355] Reference is now made to Figure 15C, which illustrates a surgical approach in which the patient 23 lies prone with his or her back to the surgeon 26, who is in an upright position, as also shown in the surgical procedure described in Figure 1 above.

[0356] In some embodiments, the surgeon 99c of FIG. 15C uses an HMD 189 equipped with an appropriate PTA, such as PTA 155 or PTA 156 (or PTA 2355, described below with reference to FIG. 23B). For conceptual clarity, a surgical approach in which the surgeon 99c is standing and inserting surgical instrument(s) from an upper track or while looking down is referred to herein as the surgeon's "standing position." In the example of the standing position of FIG. 15C, the surgeon 99c may select a pantoscopic tilt angle of approximately 35° or any other appropriate pantoscopic tilt angle and may adjust the pantoscopic tilt angle during surgery using the vertical DOF enabled by the PTAs 155, 156 (and 2355).

[0357] As described above with reference to FIG. 13 , the pantoscopic tilt angle may be defined as the angle between the axis of the frame and the axis of the display (e.g., between the horizontal axis of the frame and the optical axis of the display). In some embodiments, the PTAs of both HMD 188 and HMD 189 are configured to set any suitable pantoscopic tilt angle between about 5° and about 40°, and the adjustment of the pantoscopic tilt angle may be continuous or in predetermined steps (e.g., about 1° or 5°). In some implementations, the PTAs are configured to move (e.g., pivot, rotate, or slide) the display about an axis aligned or substantially aligned with the center of the user's eyeball of the display. In some implementations, such movement can have several advantages, including, for example, alignment of the user's optical axis with the optical axis of the display, the ability to focus on both near and far objects simultaneously using the same head-mounted display device (e.g., glasses), and improved image quality for both forward and downward viewing. As used herein, pantoscopic tilt angle may refer to the amount of tilt toward the wearer's cheek (e.g., higher values ​​refer to movement of the display closer to the wearer's cheek).

[0358] In some embodiments, HMD188 and HMD189 may replace, for example, any of HMD22 (of Figures 1 and 2A above), HUD700 (of Figure 2B above), HMD50 of Figure 3 above, HMD2122 of Figure 21A, and / or HMD2922 of Figure 29A.

[0359] In some embodiments, the vertical DOF (e.g., as implemented in PTA155, PTA156, PTA2355) may be used, mutatis mutandis, in any other surgical or other interventional procedure. In such a procedure, the surgeon or other professional may select any appropriate posture for themselves and their patient. Furthermore, according to some embodiments, even if a particular procedure is typically performed while the surgeon and / or patient are in a particular posture, the surgeon or other professional may decide to change their posture and / or the patient's posture relative to the patient's posture during the procedure, and thus an adjustable pantoscopic tilt angle, e.g., as implemented in PTA155, PTA156, and PTA2355, is important to the quality of the procedure.

[0360] Example of structured light projector implementation FIG. 16 is a schematic, pictorial diagram of a structured light projector (SLP) 200 implemented in an HMD 22, according to one embodiment. The SLP 200 may also be implemented, mutatis mutandis, in the HUD 700 of FIG. 2B , the HMD 50 of FIG. 3 , the HMDs 188 and 189 of FIGS. 15B and 15C , respectively, the HMD 2122 of FIG. 21A , the HMD 2922 of FIG. 29A , and any other suitable type of HMD used in surgical or other medical therapy and / or diagnostic procedures for any suitable type of near-eye display. While medical applications are well suited for some embodiments, non-medical applications also benefit from many of the embodiments described herein. For example, non-medical applications may include consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, and the like.

[0361] In some embodiments, a surgeon (e.g., surgeon 26) makes an incision 202 (e.g., similar to incision 24) in the skin 208 and other tissue on the back of patient 23 to expose one or more vertebrae 206 of patient 23 to be operated on. Some of the areas to be operated on may not be exposed or may not be fully exposed by incision 202, depending on the medical application.

[0362] In some embodiments, the SLP 200 includes a laser dot pattern projector configured to project structured light containing a large number (e.g., hundreds to hundreds of thousands) of dots 210 arranged in a suitable pattern onto an area 204 on an organ or body region of interest (e.g., the back) of the patient 23. This pattern serves as an artificial texture to identify locations on large anatomical structures that lack fine detail of their own (e.g., the surfaces of the skin 208 and vertebrae 206 without their edges).

[0363] In some embodiments, a pseudo-random pattern of dots 210 is used to uniquely identify clusters and can be used for disparity measurements, which in some embodiments are used to calculate depth and to improve the accuracy of 3D imaging of region 204.

[0364] In some embodiments, the wavelength of the dots 210 may be visible to the human eye (e.g., blue, green, or red) or invisible (e.g., infrared). According to some embodiments, the blue dots may advantageously retain their original shape (e.g., circular) and be clearly visible on the skin 208. In some embodiments, the SLP 200 is configured to direct a blue or green laser dot (depending on the quality of the laser source and optics and other parameters) at the region 204.

[0365] In some embodiments, camera 25 (e.g., an RGB camera) may be used to generate a 3D image of region 204, and based on the image received from camera 25, processor 33 is configured to generate a 3D image of region 204.

[0366] In some embodiments, additional depth-sensing technology may be implemented in the HMD 22. This technology relies on a single camera with a precisely calibrated offset relative to the SLP 200. In such embodiments, based on the calibrated offset, the processor 33 is configured to generate depth information without the need for a stereo camera. The depth information is obtained by identifying the relative displacement of dot clusters.

[0367] Additionally or alternatively, system 11 may include a structured light projector mounted on a wall or arm in the operating room. In such an embodiment, a calibration process between the structured light projector and one or more cameras (e.g., camera 25 on HMD 22, or one or more suitable cameras mounted in any suitable location in the operating room) may be required to obtain a 3D image based on dots 210 projected onto area 204.

[0368] In some embodiments, the SLP 200 may project infrared or any other beam having an invisible wavelength (or wavelength range), and one or more cameras, such as the camera 16 described in FIG. 1 above, may be used to obtain the 3D image.

[0369] The location of SLP 200 within HMD 22 is chosen as an example, and in some embodiments, SLP 200 may be mounted in HMD 22 (or any other of the HMDs and HUDs described above) in any other suitable location.

[0370] Rolling shutter example FIG. 17 is a diagram that schematically illustrates a method for generating an image by strobe- ing an infrared (IR) light source onto a rolling shutter image sensor within the integration time of the rolling shutter, according to one embodiment.

[0371] In some embodiments, the image sensor of camera 16 (which may include an IR camera or an RGB camera configured to function as an IR camera) has any suitable number of pixels, such as a 2 Mb sensor containing approximately two million pixels, or a sensor between 0.5 Mb and 10 Mb, between 1 Mb and 5 Mb, between 2 Mb and 10 Mb, overlapping ranges thereof, or any value within the recited ranges.

[0372] In some embodiments, each pixel has an integration time, which is the time interval that the pixel is open for exposure. Figure 17 shows the integration times for three pixels along the time scale "t".

[0373] In some embodiments, numbers 215, 216, and 217 refer to the integration times of the first pixel, millionth pixel, and two millionth pixel of the image sensor, respectively. The first pixel is opened at t0 and closed at t2, defining pixel integration time 222.

[0374] In some embodiments, the duration of the integration time 222 may be determined using the integration capacitance of each pixel of the image sensor. In some embodiments, at t0, the capacitor of the first pixel is released to charge with a voltage generated by the first pixel in response to sensing a photon on the surface of the first pixel. At t2, the capacitor of the first pixel is stopped from charging and is ready to be read out to generate the IR image described above in FIG. 1.

[0375] In some embodiments, the integration time of the first pixel (e.g., t0 to t2) is between 5 ms and 15 ms (e.g., between 5 ms and 10 ms, between 6 ms and 10 ms, between 8 ms and 15 ms, an overlapping range thereof, about 8 ms, or any value within the enumerated range). Similarly, as shown in the column labeled 217, the integration time of the two millionth pixel begins at t1 (about 7 ms after t0) and continues for between 5 ms and 15 ms (e.g., between 5 ms and 10 ms, between 6 ms and 10 ms, between 8 ms and 15 ms, an overlapping range thereof, about 8 ms, or any value within the enumerated range). Note that within the 5 ms and 15 ms (e.g., between 5 ms and 10 ms, between 6 ms and 10 ms, between 8 ms and 15 ms, an overlapping range thereof, about 8 ms, or any value within the enumerated range (e.g., between t0 and t2), all pixels of the image sensor are open. Furthermore, during the time interval 220 (e.g., about 1 ms) from t1 to t2, all pixels of the image sensor are open simultaneously.

[0376] In some embodiments, during time interval 220, processor 33 (or any other processor or controller in system 11) controls IR projector 17 (e.g., via a driver) to direct a strobing of an IR beam, referred to herein as an IR strobe, toward a surgical target area on the body of patient 23. Additionally, during the same time interval 220, processor 33 (or any other processor or controller in system 11) controls camera 16 to acquire IR images from the surgical target area.

[0377] According to some embodiments, camera 16 includes a rolling shutter that sequentially reads out each pixel. In some embodiments, the rolling shutter of camera 16 operates in a global shutter mode by implementing a sufficiently long pixel integration time 222 and directing an IR strobe during a time interval 220 during which all pixels (e.g., 2 million pixels) of the image sensor are open.

[0378] In such an embodiment, artifacts associated with a rolling shutter, such as, but not limited to, shifting of objects in an image due to sequential pixel readout times, are reduced or eliminated.

[0379] Pairing Subsystem Example FIG. 18 is a schematic, pictorial diagram of a direct pairing subsystem 230 for directly pairing a workstation (WS) 232 with HMD 22 and HMD 50, according to one embodiment.

[0380] In some embodiments, subsystem 230 is the communications subsystem of system 11 shown above in Figure 1. According to some embodiments, subsystem 230 is configured to connect between WS 232 and one or more HMDs and HUDs of any type (including any of the HMDs or HUDs disclosed herein) in addition to or instead of HMD 22 and HMD 50.

[0381] In some embodiments, WS 232 serves as a Wi-Fi hotspot device, and devices of system 11 (e.g., HMDs) typically connect to WS 232 of subsystem 230. The pairing process between devices is referred to herein as two modes of operation: "hopping" and pairing. Hopping and pairing are described below and in conjunction with FIG. 19.

[0382] Reference is now made to inset 236, which illustrates another configuration of the communications subsystem. In the example of inset 236, the subsystem comprises (i) WS 232, (ii) HUD 700 (and / or other suitable HMDs described herein), and (iii) wireless router 238. Router 238 can be hacked, and for cybersecurity reasons, there may be motivation to exclude a router (such as router 238) from the configuration of the communications subsystem.

[0383] Referring again to the overall view of Figure 18, in some embodiments, the subsystem 230 comprises (in addition to the HMD) a remote control station (e.g., a tablet) 234 intended for use by medical staff during preparation for and during the surgical or other interventional procedure.

[0384] In some embodiments, WS 232, HMD 22 and HMD 50, and tablet 234 are wirelessly connected using a service set identifier (SSID), which is a series of characters that uniquely names a wireless local area network (WLAN) comprising WS 232, HMD 22 and HMD 50, tablet 234, and optionally additional devices. The SSID may be configured to allow stations of subsystem 230 to connect to a desired network when multiple independent networks operate in the same physical area. Additionally, WS 232 may be configured to generate a password, such as password 240, which may be transmitted to HMD 22 and HMD 50 to enable a secure connection using the key exchange process described herein. This additional layer of security may be used to improve the cybersecurity of the network of subsystem 230.

[0385] In some embodiments, the WS 232 acts as the client and the HMD(s) to which it is connected acts as the server in the example configuration of Figure 18. Embodiments of the pairing process are described below in connection with Figure 19.

[0386] In some embodiments, the communication technology may comprise Wi-Fi, a network protocol suite based on the IEEE 802.11 suite of standards that may be used in wireless local area network (LAN) applications.

[0387] In some embodiments, WS232 and HMD22 and HMD50 are equipped with Bluetooth (registered trademark) (BT) adapters, and the key exchange process is performed using BT technology, a short-range wireless technology standard used to exchange data between fixed devices (e.g., WS232 implemented in a desktop or laptop computer) and mobile devices (such as, but not limited to, HMD22 and HMD50) over short distances using ultra-high frequency (UHF) radio waves in the ISM (industrial, scientific, and medical) band (e.g., 2.402 GHz to 2.48 GHz).

[0388] In some embodiments, WS 232 (and optionally other WSs located in the same medical center or facility) includes an additional Wi-Fi adapter, also referred to herein as a second Wi-Fi adapter (not shown). In such embodiments, the key exchange process occurs using a peer-to-peer (P2P) connection. In such embodiments, WS 232 uses two Wi-Fi connections: a first Wi-Fi connection for the hotspot connection and a second Wi-Fi connection for the key exchange process using the second Wi-Fi adapter.

[0389] In some embodiments, WS232 is configured to encode the hotspot key (e.g., password 240) into an optical code or other machine-readable code, such as a barcode or quick response (QR) code, that is generated using appropriate software or online tools and displayed on the display of WS232 or any other suitable display. In such embodiments, an HMD intended to pair with WS232 is configured to scan the optical code or machine-readable code (e.g., barcode or QR code) and decrypt the key to complete the key exchange process and pairing.

[0390] In some embodiments, optical code or machine-readable code (e.g., barcode or QR code) scanning may be performed using one or more additional cameras 25 (e.g., RGB cameras) or using a camera 16 configured to capture monochrome images instead of IR images (e.g., an RGB camera that can also function as an IR camera).

[0391] This particular configuration of subsystem 230 is presented as an example to illustrate the particular issues related to connectivity and cybersecurity addressed by embodiments of the present disclosure and to demonstrate the application of these embodiments in improving the performance of such communications subsystems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary communications subsystem, and the principles described herein may likewise be applied to other types of communications subsystems used in any suitable type of AR-based image-guided surgical system.

[0392] FIG. 19 is a flowchart that schematically illustrates a method for directly pairing a WS 232 with an HMD 22, according to various embodiments.

[0393] The flowchart may also be applicable, mutatis mutandis, to connecting WS232 to one or more of HMD50, HMD188, and HMD189 of Figures 3, 15B, and 15C, respectively, as well as to connecting WS232 to HUD700 of Figure 2B, HMD2122 of Figure 21A, and / or HMD2922 of Figure 29A.

[0394] Furthermore, the method is applicable for connecting between any suitable device and any workstation configured to display information such as, but not limited to, images and markers on the organ being operated on using augmented reality technology or any suitable technology other than augmented reality.

[0395] The method begins with a scanning step 300 in which the processor 33 introduces the communication device of the HMD 22 into the medical center.

[0396] In a first decision step 302, the method distinguishes between a first use case in which the HMD 22 is known to the selected network and a second use case in which the HMD 22 is new to the selected network.

[0397] In the first use case, the method proceeds to a parameter application step 304, where the processor 33 uses a set of parameters, typically parameters used in a previous connection (also referred to herein as previous parameters), known based on a previous connection between the HMD 22 and the WS 232. Note that in this use case, the pairing process has a predetermined time limit and is performed automatically, so that the method continues by checking whether the pairing was successful (shown in step 312 below).

[0398] In the second use case, the method proceeds to a pairing initiation step 306, where a workstation (e.g., WS 232) initiates a pairing process with the HMD 22 for a given time interval. Note that the time interval (also referred to herein as a time limit) for steps 304 and 306 is determined by a user or administrator of the system. Thus, the time intervals for steps 304 and 306 may be similar to each other or may differ from each other.

[0399] In some embodiments, the pairing process described below has a predetermined time interval (e.g., about 15 seconds, about 20 seconds, about 30 seconds), also referred to herein as a time limit, the meaning of which is explained in more detail below.

[0400] In key exchange process step 310, a key exchange process is performed for pairing between the HMD 22 and the WS 232. The key exchange process may be based on Bluetooth, P2P, QR Code, or any other communication technology and protocol, as described in detail in connection with FIG. 18 above. According to some embodiments, the password 400 and the network SSID are stored in advance (e.g., typically before step 300) in the HMD 22 (or in any other HMDs installed in the medical center as described in step 300 above).

[0401] In a second decision step 312, the processor 33 of the HMD 22 (and optionally the processor of the WS 232) checks whether the pairing was successful. Note that, according to some embodiments, the pairing process must succeed within the time limits described above in steps 304 and 306 for each of the two use cases.

[0402] As mentioned above, in the first use case (e.g., the HMD 22 is already paired with the selected network), the method proceeds from step 304 directly to step 312 to check whether the pairing was successful.

[0403] If pairing fails and / or is not successfully completed within a predetermined time limit, the method proceeds to an alternative WS pairing initiation step 314 in which a processor in another workstation (e.g., other than WS 232) associated with the selected network initiates the pairing process. The method then loops back to step 302 described above.

[0404] If the pairing is successful within the time limit, the method proceeds to a third decision step 316, where a user (e.g., surgeon 26) of subsystem 230 and / or processor 33 checks whether to move the HMD 22 to another network. The decision to move the HMD to another network may be made based on operational considerations, clinical considerations, technical (e.g., communications) considerations, or any other suitable considerations.

[0405] If the HMD 22 does not need to be moved to another network, the method proceeds to a data exchange step 318, as described above, where the HMD 22 and WS 232 exchange data during or between surgical procedures.

[0406] If the HMD 22 is moved to another network, the method proceeds to an unpairing step 320, where the HMD 22 is unpaired from the WS 232 and the connection parameters for connecting with the WS 232 are deleted from the storage device of the HMD 22. After unpairing between the HMD 22 and the WS 232, the method loops back to step 314 and then to step 306 as described above.

[0407] The method of Figure 19 is provided as an example to illustrate an embodiment relating to secure communication occurring directly between a workstation and an HMD, without a router or any other type of intermediate communication device. As described in connection with Figure 18 above, according to some embodiments, the elimination of a router is particularly advantageous for complying with cybersecurity requirements when communicating between an HMD 22 and a WS 232, or between any other two or more suitable medical and communication devices.

[0408] The method has been simplified for conceptual clarity and relies on the hardware description above in Figure 18. In some embodiments, the method may be implemented mutatis mutandis on any other suitable hardware comprising any suitable type of medical and / or communication device.

[0409] Examples of Electronic Subsystems FIG. 20 is a block diagram that schematically illustrates an implementation of the architecture of the electronic subsystem (ES) 444 of the system 11, according to one embodiment of the present disclosure.

[0410] In some embodiments, ES444 may be implemented mutatis mutandis in any of the HMDs and HUDs described above or below.

[0411] In some embodiments, the ES 444 comprises a carrier board (CB) 400 made from a suitable PCB or any other suitable substrate having traces for exchanging signals between the components described herein.

[0412] In some embodiments, ES 444 comprises a battery pack, referred to herein as battery 420, or any other suitable power source configured to power ES 444. As mentioned above, ES 444 may comprise a supercapacitor or ultracapacitor (not shown) connected to CB 400 in parallel with battery 420 and configured to be used to eliminate long startup times when replacing battery 420 of HMD 22 (or any other HMD or HUD described above or below).

[0413] In some embodiments, the ES 444 comprises a processor 33, a wireless communication device 45 (e.g., a Wi-Fi 6 transceiver connected to a Wi-Fi 6 antenna 406), and a storage device 46 mounted on the CB 400.

[0414] In some embodiments, the ES 444 comprises a system-on-chip (SOC) or system-on-module (SOM) device that includes the processor 33, the wireless communication device 45, the storage device 46, a graphics processing unit (GPU) (not shown), an artificial intelligence (AI) accelerator (not shown), an image signal processor (ISP), and / or other components. For example, without limitation, the SOC device may comprise any suitable SOC device selected from the Snapdragon family manufactured by Qualcomm (San Diego, California).

[0415] In some embodiments, the ES 444 includes a controller 417 configured to control and drive the IR LED projector 17 described above in FIG. 1 , an IMU controller 418 configured to drive the IMU 18 described above in FIG. 1 , and an ambient light sensor (ALS) controller 419 for the light sensor 19 described above in FIG. 1 . The controllers 417 and 418 are mounted on the CB 400. In some embodiments, the IMU 18 may include both a sensing device and a drive circuit. Similarly, the ALS controller 419 may be integrated with the sensor 19, and the controller 417 may be integrated with the LED projector 17.

[0416] In some embodiments, ES 444 comprises a backlight driver 58 configured to drive and regulate the current supply to display 15, as described with reference to FIG. 3 above.

[0417] In some embodiments, the ES 444 includes a microphone assembly (MA) 78 including a microphone and its electronics, and a speaker assembly (SA) 79 including a speaker and its electronics. The MA 78 and SA 79 are mounted on the CB 400.

[0418] In some embodiments, the ES444 comprises a bus 98 configured to carry power and data signals between the aforementioned devices mounted on the CB400 and to carry power and data signals between the CB400 and external entities described herein.

[0419] In some embodiments, the ES 444 includes an interface 402 configured to exchange power and data signals between the CB 400 and (i) the camera 16, (ii) the HA 60 (or any other headlight assembly described above), (iii) the SLP 200, and (iv) an additional camera 25 (e.g., an RGB camera).

[0420] In some embodiments, interface 402 comprises a Camera Serial Interface (CSI), which is a Mobile Industry Processor Interface (MIPI) Alliance specification, referred to herein as MIPI CSI, configured to convey signals between (i) camera 16 and camera 25 and (ii) CB 400 for generating IR and RGB images, respectively.

[0421] In some embodiments, the ES 444 comprises an interface 404 configured to output signals indicative of AR images (described above) between the CB 400 and a display of the HMD. In some embodiments, the display comprises a display 15 associated with the OE 42 (shown and described in FIG. 2A above). In some embodiments, the display 15 may comprise displays 49a and 49b of the HMD 50 shown and described in FIG. 3 above.

[0422] In some embodiments, interface 404 comprises a MIPI Display Serial Interface (MPI DSI), a high-speed interface used in various types of consumer devices.

[0423] In some embodiments, the ES 444 is configured to exchange video signals with an external entity, hi some embodiments, the ES 444 is configured to transmit video of the scene captured by the RGB camera 25 along with the rendered augmented reality image.

[0424] In some embodiments, the transmission may be performed using Wi-Fi-6 (an IEEE standard for wireless local area networks) to obtain low latency, high speed transmission of signals.

[0425] The transmission technology is not limited to Wi-Fi-6, but may also be carried out over a (5th generation) 5G cellular network or other communication network.

[0426] In some embodiments, these communication technologies may be used to allow operating room staff to see exactly what the surgeon (e.g., surgeon 26) sees, and for training and / or remote surgery applications.

[0427] In some embodiments, surgeon 26 performs surgery or other medical intervention in the operating room shown in FIG. 1 and receives feedback / comments / instructions on how to proceed with the surgical or other interventional procedure from a remote surgeon or other specialist located at a remote station 412 having a suitable communications assembly 414 (e.g., Wi-Fi-6) configured to receive signals on video output 415.

[0428] In some embodiments, the signal on the video output 415 may be recorded to document the surgical or other interventional procedure in the medical record.

[0429] In some embodiments, the ES 444 is configured to receive and display an input signal of video, more specifically, high definition (HR) video images, also referred to herein as video input 411, received from an external source 408 having a suitable communications assembly 410 (e.g., Wi-Fi-6) configured to receive a video input 411 signal.

[0430] In some embodiments, the ES 444 is configured to receive input video 411 from a digital surgical microscope and display the received video signal using, for example, the aforementioned digital magnifying glass technology described in U.S. Provisional Patent Application No. 63 / 234,272 and International Publication No. WO 2023 / 021450, the disclosures of both of which are incorporated herein by reference. Note that in this use case, displaying such HR images received from the digital surgical microscope may provide the surgeon 26 with an augmented reality rendered HR image.

[0431] In some embodiments, ES 444 is configured to receive and display input video 411 input signals, including endoscopic video received from an endoscope system. This use case allows surgeon 26 to perform various types of endoscopic or laparoscopic surgical procedures without having to look away from a remote monitor.

[0432] In some embodiments, external source 408 may comprise any other suitable video source(s) configured to generate any suitable type of HR video images. In such embodiments, processor 33 is configured to receive video input 411 including these HR video images and display the images on display 15 (or any other suitable type of display) using the AR techniques described above or any other presentation technique. Additionally, processor 33 is configured to present any other type of patient information received from external source 408.

[0433] In some embodiments, the GPU and AI accelerator of ES444 may be used in conjunction with processor 33 for controlling system 11 (of FIG. 1 above) using voice commands. For example, the GPU and AI accelerator of ES444 may be used in conjunction with processor 33 for voice and / or command recognition using any suitable type of artificial intelligence technique.

[0434] This particular configuration of ES 444, external source 408, and remote station 412 is shown as an example to illustrate the particular problem addressed by embodiments of the present disclosure and to demonstrate the application of these embodiments in improving the performance of system 11 and similar types of image-guided surgical systems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary electronic architecture, and the principles described herein may be similarly applied to other types of HMDs and hardware used in appropriate types of AR-based image-guided surgical systems and other types of image-guided surgical systems. While medical applications are well suited to some embodiments, non-medical applications also benefit from many of the embodiments described herein. For example, non-medical applications may include consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, and the like.

[0435] Adding a head-mounted display example 21A-21D illustrate another exemplary embodiment of a head-mounted display 2122. The head-mounted display 2122 has many similarities to other head-mounted displays disclosed herein, including, for example, head-mounted display 22 of FIG. 2A , head-mounted display 50 of FIG. 3 , and head-mounted display 700 of FIG. 2B described above, and the same or similar reference numbers are used to refer to the same or similar components. Accordingly, this description of the head-mounted display 2122 will focus on the differences from head-mounted displays 22, 50, and 700. Any element or feature described herein with reference to other head-mounted displays, such as head-mounted displays 22, 50, and 700, may be incorporated into the head-mounted display 2122, and similarly, any element or feature described herein with reference to head-mounted display 2122 may be incorporated into other head-mounted displays, such as head-mounted displays 22, 50, and 700.

[0436] Some benefits of the head-mounted display 2122 design relate to ergonomics, comfort, and / or the ability to enable a user, such as surgeon 26, to utilize the system for relatively long periods of time, such as four hours or more, without unnecessary fatigue and / or other adverse effects. For example, in some embodiments, a head-mounted display such as head-mounted display 22 of FIG. 2A may have a weight of approximately 260 g, and such a design may transmit roughly 200 g of that weight to the user's nose via nose pads 28. While this may be acceptable in some situations, in other circumstances, such as during longer surgeries, it may be desirable to reduce the load on the user's nose, for example, to 90 g or less. A design such as the head-mounted display 2122 shown in FIG. 21A can achieve such benefits. For example, the head-mounted display 2122 can distribute its weight around the wearer's head, including the wearer's forehead and the back of the wearer's head, to reduce at least a portion of the weight on the wearer's nose. Such a configuration can also reduce pressure on the temples of the wearer, which may be another relatively weight-sensitive area in addition to the nose. Stated another way, such a head-mounted display may distribute pressure more widely over larger and / or less sensitive areas such as the forehead or back of the head.

[0437] Head mounted display 2122 combines certain features of other head mounted displays disclosed herein, including head mounted display 22, head mounted display 50, and head mounted display 700. For example, as described in more detail below, head mounted display 2122 includes left and right temple housings that bear some similarity to the left and right temple arms of head mounted display 22, but that do not contact and / or are not supported by the temples or ears of the wearer. As another example, as described in more detail below, head mounted display 2122 includes a back pad and adjustable strap mechanism that may be similar to those used in head mounted display 700. Additionally, some embodiments may include an optional top or upper strap that can further distribute weight over the top of the wearer's head.

[0438] Another advantage of a design that distributes weight to less sensitive areas of the wearer's head, such as that of head mounted display 2122, is that additional weight can be added to the head mounted display without significantly increasing pressure on the wearer's head in any particular spot or sensitive area. For example, a flashlight assembly can be attached to the head mounted display without significantly increasing pressure on the wearer's head in any particular spot or sensitive area.

[0439] 21A is a front perspective view of head mounted display 2122, FIG. 21B is a rear perspective view of head mounted display 2122, FIG. 21C is a left side view of head mounted display 2122, and FIG. 21D is a top view of head mounted display 2122. Some similarities of head mounted display 2122 to other head mounted displays disclosed herein include, for example, a head strap 740 and a knob 744 for adjusting head strap 740, similar to head mounted display 700 of FIG. 2B. However, head strap 740 may have some additional features to accommodate certain differences in head mounted display 2122, which are described in more detail below.

[0440] Other similarities to other head-mounted displays 2122 disclosed herein include, for example, an optics housing 704 with one or more cameras 708 and an infrared projector 716 (see FIG. 21A ), one or more processors 33, a wireless communication device 45, and / or a storage device 46 (see FIG. 21B ), an optical engine 55 which may include components the same as or similar to the optical engine 55 described above with reference to FIG. 3 , multiple AR displays 49 a, 49 b similar to those described above with reference to FIG. 3 , and adjustable nose pads 28 similar to those described above with reference to FIG. 12 . In this embodiment, the optical engine is housed within an optical engine housing or frame 2104. In some embodiments, the optics housing 704 and the optical engine housing 2104 are part of the same frame or housing, and in some embodiments, the optics housing 704 and the optical engine housing 2104 are separate components. In this embodiment, the optics housing 704 and the optical engine housing 2104 are configured to tilt together (see FIG. 23B ), although some embodiments may allow the optical engine housing 2104 to tilt or otherwise move relative to the optics housing 704. The optics housing 704, the optical engine housing 2104, and the components contained within or attached to those housings (such as, but not limited to, displays 49 a, 49 b, and optical engine 55) may collectively be referred to as a see-through display assembly 2349. Furthermore, in any embodiment disclosed herein that includes one or more displays configured to tilt using a pantoscopic tilt assembly (PTA), the displays and any other components, frames, and / or housings that tilt with the displays may collectively be referred to as a see-through display assembly. In some embodiments, the optical engine housing or frame 2104 includes some or all of the features of the display assembly frame 41 of FIG. 3 .

[0441] One difference between the head mounted display 2122 and the head mounted display 22 of FIG. 2A is that the head mounted display 2122 includes left temple housings 2143 and right temple housings 2144 instead of left temple arms 43 and right temple arms 44. The left and right temple housings 2143, 2144 may incorporate some or all of the same features as the left and right temple arms 43, 44 of the head mounted display 22 of FIG. 2A , such as, for example, including an on / off button 39, housing one or more processors 33, wireless communication devices 45, and / or storage devices 46, and including strain reliefs 47 for cables. However, the head mounted display 2122 is configured to be attached to the head of a user (e.g., surgeon 26) differently, and therefore the left and right temple housings 2143, 2144 do not include rocker arms like the head mounted display 22. Specifically, the head mounted display 2122 is configured to be attached to the user's head using a head strap 740. Head strap 740 has similarities to head strap 740 of head mounted display 700 of FIG. 2B, but also has some differences that are described below.

[0442] Referring to FIG. 21B, the head strap 740 of the head-mounted display 2122 comprises, or consists essentially of, a first or left side strap 2152 and a right or second side strap 2155. The left side strap 2152 comprises a first end 2153 attached to the first or left end of the frame 2150, and the right side strap 2155 comprises a first end 2156 attached to the second or right end of the frame 2150 (see FIG. 21A). The first or left end 2150 is configured to be positioned adjacent to the first or left side of the user's (e.g., surgeon 26) head, and the second or right end 2150 is configured to be positioned adjacent to the second or right side of the user's (e.g., surgeon 26) head. The left side strap 2152 further comprises a second end 2154, and the right side strap 2155 further comprises a second end 2157. As described in more detail below, the adjustment mechanism 2158 can be used to adjust the relative positions of the second ends 2154, 2157 to adjust the circumferential size of the head strap 740. The adjustment mechanism 2158 and the head strap 740 can be together referred to as an adjustable strap assembly.

[0443] 21B, the head strap 740 further includes a forehead support 2160 comprising a strap extending from a first end 2161 pivotally connected to the left side strap 2152 to a second end 2162 pivotally connected to the right side strap 2155. The forehead support 2160 further includes a central support 2163 extending downward from the main strap and secured to an intermediate or central location on the frame 2150. Although not shown in these figures, cushions or padding may be added to any portion of the head strap 740, including the forehead support 2160 and elsewhere. Examples of such cushions or padding are described below with reference to FIGS. 25 and 26A-26C.

[0444] 21B , in this embodiment, the left and right side straps 2152, 2155 each include a front portion 2165 pivotally connected to a rear portion 2166 by a connector 2164. Further, in this embodiment, the front portion 2165 is pivotally connected at the same pivot connection as the first and second ends 2161, 2162 of the straps of the forehead support 2160. However, such a configuration is not required, and various other embodiments may include more, fewer, or no pivot connections. However, the pivot configuration shown in FIG. 21B has been found to be desirable to accommodate a variety of users (e.g., surgeons 26) while maintaining the head-mounted display 2122 comfortably and securely on the user's head.

[0445] The head strap 740 also desirably includes a pad or cushion 2170 attached to an adjustment mechanism 2158 that can engage the back of the user's head. Additionally, the head strap 740 in this embodiment also includes front and rear slots 2171, 2172 that can be used to attach optional top or upper side straps, as described in more detail below with reference to Figures 24A-24B.

[0446] 21C, this shows that the optical engine housing 2104 and the optics housing 704 can be desirably pivoted or tilted relative to the frame 2150 to adjust the pantoscopic tilt angle 2168. Further details of the mechanism used to adjust the pantoscopic tilt angle 2168 in the head mounted display 2122 are provided below with reference to FIG. 23B. Additionally or alternatively, the same or similar tilt mechanisms may be used as described above with reference to FIGS. 13 and 14, etc.

[0447] 21D , this top view of the head mounted display 2122 helps to illustrate the particular adjustability created by the adjustment mechanism 2158. Specifically, the adjustment mechanism 2158 desirably adjusts the circumferential size, which is essentially defined by the left side strap 2152, the right side strap 2155, and the frame 2150. For example, when the adjustment mechanism 2158 moves the ends 2154, 2157 closer together, the circumferential size increases, and when the adjustment mechanism 2158 moves the ends 2154, 2157 apart, the circumferential size decreases. Arrow 2180 represents this increase or decrease in circumferential size.

[0448] 21D , this figure also shows that the left temple housing 2143 and the right temple housing 2144 can desirably move or pivot inward and outward relative to the ends 2181, 2182 of the frame 2150. This movement or pivoting is indicated by arrow 2183. The movement 2183 of the temple housings 2143, 2144 can be the result of a combination of adjusting the circumferential size of the head strap 740 and bending the head strap 740 as the user places the head mounted display 2122 on or removes the head mounted display 2122 from their head. The movement or pivoting can be facilitated by the same or similar tilt assembly 111 as described above with reference to FIG. 9 .

[0449] To facilitate adjustability of the head mounted display 2122 and its head strap 740, left and right temple housings 2143, 2144 may also desirably be movably coupled to a portion of the head strap 740. For example, in this embodiment, the left temple housing 2143 is slidably coupled to the left side strap 2152, and the right temple housing 2144 is slidably coupled to the right side strap 2155. More specifically, each of the temple housings 2143, 2144 desirably comprises or consists essentially of a follower 2185 that is slidable back and forth and attached to the left and right side straps 2152, 2155 by one or more fasteners, such as fastener 2186. Further details of this structure are described below with reference to FIGS. 23A-23D .

[0450] 21A and 21D also show that the head mounted display 2122 includes a cover 2190 that is removably coupled to the frame 2150. Desirably, this cover 2190 can be removed and replaced with a removable flashlight assembly, as described in more detail below with reference to FIGS. 27A-27D and 28A-28F.

[0451] 21D, this figure also illustrates that each of the temple housings 2143, 2144 can include a plurality of fins 2199 (e.g., heat dissipation fins) that can, for example, aid in heat dissipation. Various other heat dissipation mechanisms may also or alternatively be used, including vents, heat sinks, fins, protrusions, active and / or passive cooling, etc. Such mechanisms may be desirable, for example, to dissipate heat generated by one or more processors 33 located within one or both of the temple housings 2143, 2144.

[0452] This particular configuration of the HMD 2122 is shown as an example to illustrate the particular problem addressed by certain embodiments and to demonstrate the application of these embodiments in improving the performance of such systems. However, embodiments of the present disclosure are by no means limited to this particular type of exemplary HMD configuration, and the principles described herein may similarly be applied to other types of HMDs and HUDs used in any suitable type of near-eye display AR-based image-guided surgery system. The HMD 2122 may be used for non-medical applications. For example, non-medical applications may include consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, etc.

[0453] Example of adjustment mechanism details 22A and 22B show further details of the adjustment mechanism 2158 that can be used to adjust the size of the head strap 740 of the head mounted display 2122. Additionally, FIGS. 23A-23D show further details of the follower mechanism that helps control the inward and outward pivoting 2183 of the left and right temple housings 2143, 2144 (see FIG. 21D). These figures also show further details of the pantoscopic tilt mechanism.

[0454] As described above, the head-mounted display 2122 may desirably be configured for use by users (e.g., surgeons 26) having a variety of head sizes and / or shapes. Accordingly, the head strap 740 may desirably be adjustable, such as by adjusting the circumferential size of the head strap 740 to accommodate users' heads having various shapes and sizes. In addition to adjusting the circumferential size of the head strap 740 to make the head-mounted display 2122 more comfortable for the user and to better accommodate users with heads of different sizes and / or shapes, the left and right temple housings 2143, 2144 may desirably be movably or pivotally coupled to the frame 2150 (as described above with reference to FIG. 21D ). Thus, as the head strap 740 is adjusted to be smaller (e.g., by reducing the circumferential size of the head strap 740 using the knob 744), the left and right temple housings 2143, 2144 may desirably pivot inward (e.g., toward the user's head) relative to the frame 2150. Similarly, when the head strap 740 is adjusted to be larger (e.g., by increasing the circumferential size of the head strap 740 using the knob 744), the left and right temple housings 2143, 2144 can desirably pivot outward (e.g., away from the user's head) relative to the frame 2150. Such pivoting can also occur when the user puts on or takes off the head mounted display 2122.

[0455] 22A and 22B show further details of how the knob 744 of the adjustment mechanism 2158 changes the circumferential size of the head strap 740 (see, for example, adjustment 2180 in FIG. 21D ). Specifically, in this embodiment, each of the side straps 2152, 2155 includes a rack 2210 that engages a pinion or pinion gear 2212. In this embodiment, there is a single pinion gear 2212 that rotates with the knob 744 and engages both of the racks 2210. However, in some embodiments, there may be separate pinion gears, each engaging one of the racks 2210. Rotation of the knob 744, and thus the pinion gear 2212, desirably moves the strap ends 2157, 2154 closer together or further apart, thereby changing the circumferential size of the head strap.

[0456] To maintain the circumferential size of the head strap in a particular configuration, the adjustment mechanism 2158 further includes a tensioning mechanism (e.g., a stop mechanism) 2215. The tensioning mechanism 2215 is configured to maintain the knob 744 in a particular position until, for example, a user overcomes a threshold force of the tensioning mechanism 2215 to rotate the knob 744. For example, the tensioning mechanism 2215 shown in FIGS. 22A and 22B includes a gear 2218 on the inner surface of the knob 744, which engages two other gears 2217. Additionally, a tensioning member 2219 is disposed between the gears 2217 and applies friction to the gears 2217 to prevent or limit their rotation until a threshold force is overcome. It should be noted that the tensioning mechanism 2215 shown in FIGS. 22A and 22B is merely an example, and that various other techniques may be used to maintain the adjustment mechanism 2158 in a particular configuration until a user desires to adjust the configuration. For example, other methods of generating friction may be used, a separate locking mechanism may be used, or a gear assembly including mechanical gain configured to self-lock may be used.

[0457] It should be noted that this is just one example of how the adjustment mechanism can work, and that various other methods may be used to adjust the size of the head strap 740. For example, the left and right side straps 2152, 2155 may be adjusted and held in place relative to one another using different types of gear trains or mechanisms, friction, etc.

[0458] 23A-23D, these figures show further details of the follower mechanism and tilt mechanism. Specifically, referring to FIGS. 23B-23D, these figures show further details of the follower 2185 and fastener 2186 described above with reference to FIG. 21D. These figures show the right temple housing 2144; the same or similar features may be used for the left temple housing 2143. As seen in FIG. 23B, the follower 2185 desirably includes or consists essentially of an elongated protrusion slidably coupled to an elongated slot 2387 in the sidewall of the temple housing 2144. As described above, as the head strap 740 is adjusted and / or flexed, the follower 2185 can desirably slide relative to the temple housing 2144, thereby allowing the temple housing 2144 to move inward or outward with the head strap 740 while remaining substantially vertically supported by the head strap 740.

[0459] 23C and 23D, these figures show further details of how the follower 2185 is connected to the side strap 2155 and the temple housing 2144. Specifically, a fastener 2186 and a cap or washer 2388 attach the follower 2185 to the side strap 2155. Additionally, the wall of the temple housing 2144 desirably includes a portion 2389 that is positioned between the elongated protrusion of the follower 2185 and the strap 2155, thereby holding the follower 2185 in place relative to the temple housing 2144.

[0460] It should be noted that this is just one example of how the temple housings can be movably or slidably attached to the side straps, and other techniques may be used. Desirably, the temple housings are supported generally vertically by the side straps while being free to move generally horizontally relative to the side straps. Such a configuration can provide adequate vertical support of the temple housings by the head straps 740 while allowing adjustment to provide a comfortable, snug fit to the user's head.

[0461] Returning to FIG. 23B, this figure also shows further details of the pantoscopic tilt assembly (PTA) 2355, which allows tilting (e.g., rotating, pivoting, translating, or sliding) of the optical engine housing 2104 and the optical housing 704 relative to the frame 2150. The PTA 2355 performs a similar function to the PTA 155 and PTA 156 described above with reference to FIG. 14, and the same or similar reference numbers are used to refer to the same or similar components. Additionally, any of the other PTAs disclosed herein, including but not limited to PTA 156 and PTA 155, may be incorporated into the head mounted display 2122, and the PTA 2355 of the head mounted display 2122 may be incorporated into any of the other head mounted displays disclosed herein.

[0462] The PTA 2355 comprises or essentially consists of a virtual hinge or axis (represented by point 2361) created by an arcuate slot 2374 as a radial cam and a rotatable section or guide member 2379 as a follower around which the optical engine housing 2104 and housing 704 can pivot (e.g., tilt, rotate, move or slide) relative to the frame 2150, thereby causing adjustment of the pantoscopic tilt angle 2168. The center of virtual hinge or axis 2361 is desirably located at the center of a human eyeball (e.g., eyeball 158 in FIG. 13 ) so that movement of portable or movable parts (e.g., of optical engine housing 2104 and housing 704) around or relative to stationary parts (e.g., frame 2150 and portion 2372 of frame 2150 defining slot 2374) maintains an undistorted view on the optical engine display (e.g., display 49a in FIG. 21A ). To maintain optical engine housing 2104 and housing 704 at a particular tilt angle or position relative to frame 2150, PTA 2355 further comprises locking element 2373. In this embodiment, locking element 2373 comprises a spring-loaded ball that can engage any one of a plurality of detents 2375 fixed relative to frame 2150 to maintain optical engine housing 2104 and housing 704 (and other portions of see-through display assembly 2349 that tilt with housing 2104 and housing 704) in any one of a plurality of predetermined positions corresponding to detents 2375. Specifically, in this embodiment, ball 2373 is coupled to arm 2376 that is configured to pivot about axis 2380 (axis 2380 is parallel to the plane of the drawing, such that pivoting movement of arm 2376 is out of or into the page). Furthermore, arm 2376, and thus ball 2373, is spring-biased about axis 2380 using, for example, spring 2371 (located below the lower end of arm 2376 in this cross-sectional view).When a user (such as surgeon 26) wishes to rotate the optical engine housing 2104 and optical housing 704 relative to the frame 2150, the user may apply a force to the optical engine housing 2104 and / or housing 704, causing the rotatable section 2379 to transmit the force via the arm 2376 to the locking element 2373. When sufficient force is applied to overcome the threshold force of the spring-loaded ball, the locking element 2373 may disengage one of the detents 2375 of the frame 2150, allowing the optical engine housing 2104 to pivot about the virtual axis 2361 relative to the frame 2150. Once the optical engine housing 2104 is at the desired pantoscopic tilt angle 2168, the locking element 2373 may maintain the optical engine housing 2104 at that angle by re-engaging one of the detents 2375. The optical engine housing 2104 is then desirably maintained at that angle relative to the frame 2150 until the threshold force of the spring-loaded locking element 2373 is again exceeded. Although FIG. 23B shows the PTA 2355 on only the right side of the head mounted display 2122, a similar or similar structure may be used on the left side of the head mounted display 2122.

[0463] Various modifications to the pantoscopic tilt assembly 2355 may be made. For example, a spring-loaded pin may be used instead of a spring-loaded ball. As another example, the spring-loaded pin or ball may be spring-loaded by a spring directly engaged with and / or aligned with the ball, instead of engaging a lever arm (e.g., arm 2376) that further engages the ball. As another example, instead of or in addition to a detent mechanism, friction may be used to maintain the optical engine housing 2104 and optical housing 704 at a particular angle relative to the frame 2150. Furthermore, the described mechanism (e.g., PTA 2355) is merely one example; in this embodiment, a virtual axis 2361 with five detents 2375 is implemented, providing a 20° range of adjustability (e.g., 15° to 35° horizontal tilt), although various embodiments may include other ranges of adjustability, as discussed above with reference to FIGS. 15A-15C. Any other virtual axis mechanism can be implemented, such as using friction instead of detents and / or using a fully positive locking mechanism instead of a “semi-positive” locking mechanism (such as a spring-loaded ball and detents) that unlocks when a threshold force is exceeded. Additionally, some PTAs may utilize real, i.e., physical, axes for pivoting (e.g., axes generated by mechanical hinges) instead of virtual axes. However, using a virtual axis may be desirable, such as to reduce or eliminate lateral field of view limitations and / or obstructions. For example, with reference to FIG. 23B , if the frame 2150 and housing 2104 included mechanical components that extended to and formed a physical hinge at point 2361 (instead of the current virtual axis or virtual hinge at point 2361), those mechanical components may be visible in the surgeon's peripheral vision and thus unnecessarily obstruct some or all of the surgeon's peripheral vision. However, the disclosure herein is not limited to PTAs that utilize virtual axes; some embodiments may utilize physical hinges to form axes about which the PTA can rotate.

[0464] Example of additional head strap mechanism As described above with reference to the head mounted display 2122 shown in FIGS. 21A-21D , the head strap 740 may optionally include a removable top or upper strap. FIGS. 24A and 24B show a head mounted display 2122 to which such a removable top or upper strap 2410 is coupled. Specifically, the top strap 2410 includes a first end 2411 that is threaded through a slot 2171 and folded back onto itself. The first end 2411 may include, for example, a hook-and-loop fastener, a button, or the like, that maintains the position of the first end 2411 relative to the slot 2171. Similarly, the top strap 2410 includes a second end 2412 that is threaded through a slot 2172 and folded back onto itself. The second end 2412 may include, for example, a hook-and-loop fastener, a button, or the like, that maintains the position of the second end 2412 relative to the slot 2172.

[0465] Because some users may prefer to use the head mounted display 2122 without a top strap and some users may prefer to use the head mounted display 2122 with a top strap, it may be desirable to have an optional removable top strap, such as top strap 2410. That said, some embodiments may include a permanently attached top strap, and some embodiments may not have the option to attach a top strap. It may also be desirable to make the top strap 2410 adjustable, such as by adjusting its length. One way to achieve this is to allow for varying amounts of strap at the first and / or second ends 2411, 2412 to be threaded through corresponding slots and folded back onto itself. A further or alternative way to achieve this is to include an adjuster on the strap, such as using a buckle, hook-and-loop fastener (e.g., see the discussion below in connection with FIGS. 26A-26C ), or the like. The top strap 2410 may be formed from a variety of materials, such as a flexible polymer, fabric, etc. In some embodiments, the top strap 2410 comprises a soft fabric, which can aid in comfort, for example.

[0466] 21A-21D show the head strap 740 of the head mounted display 2122 without any additional padding or cushioning other than the back pad 2170. Referring to FIG. 25, this figure shows an example of a head mounted display 2122 with additional padding 2520 in front of the head mounted display 2122 to provide some cushioning between the head mounted display 2122 and the forehead of a user (e.g., surgeon 26). Specifically, the padding 2520 in this embodiment includes a first portion 2521 positioned between the main strap of the forehead support 2160 and the surgeon 26, a second portion 2522 positioned between the central support 2163 and the surgeon 26, and a third portion 2523 positioned between the frame 2150 and the surgeon 26.

[0467] FIG. 25 shows only one example of cushions or pads added to the head mounted display 2122; more or less cushions may be provided, and cushions may be provided in additional locations, such as along the side straps of the head mounted display 2122.

[0468] Referring to FIGS. 26A-26C, these figures show the same head mounted display 2122 of FIG. 25 with the addition of a forehead pad 2520, but also with the addition of an upper strap 2610. The upper strap 2610 is similar to the upper strap 2410 of FIGS. 24A and 24B, except that additional features are shown in FIGS. 26A-26C. Specifically, FIG. 26C shows that each end of the upper strap 2610 utilizes a button 2670 to maintain the upper strap 2610 in place relative to the remainder of the head mounted display 2122. Additionally, FIGS. 26A and 26C show that the upper strap 2610 is provided in two sections: a rear section 2671 and a front section 2672. The rear section 2671 is layered on top of the front section 2672. A hook-and-loop fastener 2673 desirably maintains the rear section 2671 in place relative to the front section 2672. Such a design may desirably allow the length of the top strap 2610 to be adjusted, such as by changing the relative position of the rear portion 2671 with respect to the front portion 2672. This embodiment desirably uses a hook-and-loop fastener between the front portion 2672 and the rear portion 2671, although other embodiments may use other techniques such as buttons, buckle mechanisms, elastic, etc.

[0469] Example of adding a removable lighting system As discussed above, various embodiments of the head-mounted display may include a permanently attached or detachable headlight or lighting system, including a flashlight, such as for illuminating an area where a surgical procedure is being performed. Various examples of such lighting systems are described above with reference to the embodiments of, for example, FIGS. 2B, 4, 5, 6, 7, and 8. The head-mounted display 2122 of FIG. 21A described above may be modified for use with any of such disclosed lighting systems. Additionally, FIGS. 27A-27D show another exemplary embodiment of a detachable lighting system 2701 for use with the head-mounted display 2122. The detachable lighting system 2701 may also be incorporated into any of the other head-mounted displays disclosed herein.

[0470] 27A and 27B show a removable lighting system 2701 attached to a mounting socket 2710 of a housing 704 of a head-mounted display 2122. For example, the removable cover 2190 shown in FIG. 21A has been removed and replaced with a mounting base 2705 for the lighting system 2701. As seen in FIGS. 27A and 27B, the lighting system 2701 comprises, or essentially consists of, a flashlight 2703 pivotally coupled to the mounting base 2705 by an arm 2707. More specifically, the arm 2707 is pivotally coupled at a first end to the flashlight 2703 and at a second end to the mounting base 2705. These pivotal connections desirably allow the flashlight 2703 to be repositioned as needed. For example, in the head mounted display 2122, the optics housing 704 incorporating the flashlight mounting socket 2710 tilts with the optical engine housing 2104 when the pantoscopic tilt angle of the optical engine housing 2104 is adjusted relative to the frame 2150. Thus, as the pantoscopic tilt angle is adjusted, the orientation of the flashlight 2703 may also change relative to the frame 2150. In such cases, it may be desirable to reposition the flashlight 2703 so that it can be aimed at the same area of ​​interest regardless of the current pantoscopic tilt angle of the optical engine housing 2104. The arm 2707 may be intended to allow the light beam to be aligned with the wearer's line of sight for optimal visibility.

[0471] Figure 27C shows a side view of the lighting system 2701 detached from the rest of the head mounted display. This view shows further details of the mounting base 2705, which will be described in more detail below with respect to the embodiment of Figures 28A-28F. Figure 27D shows a perspective view of the head mounted display 2122 with the lighting system 2701 detached. This view shows further details of the flashlight mounting socket 2710, which will be described in more detail below with respect to the embodiment of Figures 28A-28F.

[0472] 28A-28F, these figures illustrate another embodiment of a lighting system 2801 that can be used with head-mounted display 2122 (or any other head-mounted display disclosed herein). The same or similar reference numbers as lighting system 2701 of FIGS. 27A-27D are used to refer to the same or similar features. The primary substantive difference between lighting system 2801 and lighting system 2701 is that lighting system 2801 includes or essentially consists of two pivotally connected arms 2707 instead of a single arm 2707. Specifically, with reference to FIG. 28A, the two arms 2707 are pivotally connected together, with one of the arms 2707 pivotally connected to mounting base 2705 and the other arm 2707 pivotally connected to flashlight 2703. This design can allow for a greater range of adjustability of the position and / or angle of flashlight 2703, for example, as compared to lighting system 2701 of FIG. 27A. However, the single arm design of the illumination system 2701 of FIG. 27A may also be beneficial, especially if the use of a single arm 2707 allows for a sufficient range of adjustment while reducing the weight and / or complexity of the system.

[0473] 28E, this figure shows further details of the flashlight mounting socket 2710. Specifically, the flashlight mounting socket 2710 comprises, or consists essentially of, a first rod 2820 extending generally horizontally and a second rod 2822 extending generally parallel to the first rod 2820. The flashlight mounting socket 2710 further comprises two electrical contacts 2834 at the base of the socket. The electrical contacts 2834 may be used, for example, to provide power for use by the flashlight 2703. More or fewer electrical contacts may be used (including zero electrical contacts). In some embodiments, one or more electrical contacts may be used for control signals to turn the flashlight 2703 on or off, adjust the brightness of the flashlight 2703, etc.

[0474] 28B-28D, these figures show an illumination system 2801 and a close-up view of the mounting base 2705 of the illumination system 2801 removed from the mounting socket 2710 of the head mounted display 2122. The mounting base 2705 in this embodiment includes a first recess 2821 and a second recess 2823. The first recess 2821 is sized and positioned to engage with the first rod 2820 of the mounting socket 2710 (see FIG. 28E), and the second recess 2823 is sized and positioned to engage with the second rod 2822 of the mounting socket 2710 (see FIG. 28E). For example, to attach the mounting base 2705 to the mounting socket 2710, a user may position the first recess 2821 to engage with the first rod 2820, and then the user may rotate the lighting system 2801 downward (e.g., pivot about the first rod 2820) until the second recess 2823 engages with the second rod 2822. As can be seen in FIG. 28C , the first recess 2821 opens generally in a rearward direction, and the second recess 2823 opens generally in a downward direction. By opening the two recesses in different directions, this can be such that when both recesses 2821, 2823 are engaged with the rods 2820, 2822, respectively, the engagement of the second recess 2823 with the second rod 2822 resists the recess 2821 from disengaging from the first rod 2820.

[0475] 28C and 28D , the mounting base 2705 further comprises a movable latch 2824 that can pivot about a pivot axis 2828 to enable or disable the second rod 2822 from entering or exiting the second recess 2823. Thus, when the second recess 2823 is fully engaged with the second rod 2822 and the latch 2824 is in a closed position (e.g., the position shown in FIGS. 28C and 28D ), the latch 2824 resists the second rod 2822 from disengaging from the second recess 2823, thereby completely (or at least sufficiently) constraining the position of the lighting system 2801 relative to the rest of the head mounted display 2122.

[0476] 28D , the mounting base 2705 further comprises a user interface mechanism, such as a button 2826, that a user can operate to move the latch 2824 to an open position, thereby allowing removal of the mounting base 2705 from the mounting socket 2710. Desirably, the latch 2824 is biased to the closed position by a spring, such as, for example, spring 2830 seen in FIG. 28D . As also seen in FIG. 28D , the latch 2824 desirably comprises a sloped or tapered outer surface that engages the second rod 2822 during installation, forcing the latch 2824 to pivot and allowing the second rod 2822 to engage the second recess 2823 without necessarily requiring the user to operate the button 2826 during installation. In other words, the latch 2824 and button 2826 may be configured so that the mounting base 2705 can be attached to the mounting socket 2710 without the user having to operate the button 2826, but may be configured so that the button 2826 must be operated to remove the mounting base 2705 from the mounting socket 2710.

[0477] FIG. 28D also shows electrical contacts 2832 extending downwardly from the bottom of the mounting base 2705. Desirably, the electrical contacts 2832 are spring-loaded and / or resiliently bendable so that a sufficiently low resistance electrical connection can be made between the electrical contacts 2832 and corresponding electrical contacts 2834 of the mounting socket 2710 (see FIG. 28E). Although the side view of FIG. 28D shows only one electrical contact 2832, the mounting base 2705 desirably comprises or consists essentially of two electrical contacts 2832 positioned corresponding to the positions of the two electrical contacts 2834 of the mounting socket 2710 shown in FIG. 28E. However, only one is visible in FIG. 28D because the second is directly behind the first in this side view orientation. Additionally, similar to that described above with respect to the electrical contacts 2834 of the mounting socket 2710, the mounting base 2705 may include more or fewer electrical contacts 2832.

[0478] 28F, which shows lighting system 2801 attached to the rest of head mounted display 2122, this figure also shows strain relief member 2830 extending from the front of mounting base 2705. Although not shown, strain relief member 2830 may have one or more wires passing through it that, for example, transmit power and / or control signals (received from electrical contacts 2834) from electrical contacts 2832 to flashlight 2703.

[0479] Optical System Considerations and Clip-On Lens Assembly Examples Various users of the head-mounted displays disclosed herein (e.g., surgeons 26) may require prescription lenses for vision correction. For example, at least some of such users may typically wear prescription eyeglasses to correct their vision. Some of the head-mounted displays disclosed herein may be able to be worn over such prescription eyeglasses, although such a configuration may not be ideal. As an alternative to requiring the surgeon to wear both prescription eyeglasses and a head-mounted display disclosed herein, various embodiments of the head-mounted displays disclosed herein may be configured to have prescription lenses coupled to them to allow the surgeon to use the head-mounted display with clear vision without having to wear a separate corrective device, such as glasses, contact lenses, or the like.

[0480] 29A-29E illustrate one embodiment of a head-mounted display 2922 that includes clip-on, snap-on, removable, and / or interchangeable prescription lenses to enable the head-mounted display 2922 to be used by a variety of users (e.g., surgeons 26) with a variety of corrective lens requirements. The head-mounted display 2922 is similar to the head-mounted display 50 described above with reference to FIG. 3, and the same or similar reference numbers are used to refer to the same or similar components. The head-mounted display 2922 shown in FIG. 29A includes two major differences from the head-mounted display 50 shown in FIG. 3. First, left and right temple arms 43, 44 (which may be included in the head-mounted display 50 but are not shown in FIG. 3) are shown attached to the display assembly frame 41. The temple arms 43, 44 may be the same as or similar to the left and right temple arms 43, 44 of the head-mounted display 22 of FIG. 2A, for example. Second, displays 49a, 49b are modified to utilize a rear lens 2912 that is part of a removable lens assembly 2910 (eg, a clip-on lens assembly, a snap-on lens assembly, etc.) (see FIG. 29B).

[0481] FIG. 29C shows a cross section of right display 49a, while left display 49b is of similar construction. Display 49a includes a front lens 2902 and a rear lens 2912. Between front lens 2902 and rear lens 2912 is a reflective waveguide lens 2904, which can be used to display images transmitted from waveguide 52 (see FIG. 3). For example, waveguide lens 2904 may be a lens incorporating multiple partially reflective mirrors that allow a user to see through waveguide lens 2904 while also allowing the user to see images transmitted by waveguide 52 to waveguide lens 24. In some embodiments, reflective waveguide lens 2904 comprises a reflective waveguide lens available from Lumus Ltd. (Ness Ziona, Israel). It should be noted that while the embodiment shown in FIG. 29C includes a removable lens assembly, the various lenses shown in FIG. 29C may also be used in a similar configuration in embodiments in which none of the lenses are intended to be removable. For example, some embodiments may include a front lens 2902, a rear lens 2912, and a reflective waveguide lens 2904 arranged in substantially the same relative positions as shown in FIG. 29C, but instead of making the rear lens 2912 removable, all permanently or semi-permanently installed.

[0482] In some embodiments, the rear lens 2912 is shaped by default (e.g., before considering compensation for a personal prescription) to provide a specific diopter compensation to achieve focus of the AR image at a specific working distance. For example, in some embodiments, the rear lens 2912 is shaped by default to provide -2D diopter compensation to achieve focus at a working distance of 0.5 m (e.g., 50 cm ± 20 cm). The front lens 2902 may be shaped to compensate for the above-mentioned effects of the rear lens 2912 (e.g., to reduce or eliminate the effect of the rear lens 2912 on the real view via the display 49 a). For example, with a default -2D rear lens 2912, the front lens 2902 may be shaped to provide, for example, +2D diopter compensation. Focal length considerations in the disclosed systems may differ from those of regular glasses or other augmented reality systems (e.g., consumer augmented reality systems). For example, regular glasses or other augmented reality systems (e.g., consumer augmented reality systems) may be configured to achieve focus at distances of approximately 3-4 m or greater. However, when using the systems disclosed herein in conjunction with surgical or other medical procedures, the desired working distance or focal length may be significantly shorter, such as approximately 50 cm, or within a range of approximately 30 cm to 70 cm. For example, a wearer may view a treatment or diagnostic site from a relatively close range, such as from a standing or sitting position adjacent to the patient. However, the systems disclosed herein may also be used in other applications (e.g., athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, etc.) that may each have a longer or shorter desired focal length, and the diopter compensation in the front and rear lenses may be adjusted accordingly to account for such different focal lengths.

[0483] 29C , for a user (e.g., surgeon 26) who does not require vision correction, the default posterior lens 2912 may be acceptable. However, for a user who does require vision correction, it may be desirable to replace the posterior lens 2912 with an alternative lens that alters the compensation to correct that particular user's vision. Accordingly, the posterior lens 2912 of the head mounted display 2922 is part of a detachable, removable, and / or interchangeable clip-on lens assembly 2910 shown in FIG. 29B . Referring to FIG. 29B , the clip-on lens assembly 2910 comprises or essentially consists of a frame 2911 (e.g., a posterior lens frame) that holds the left and right posterior lenses 2912 and includes a bridge portion 2916 that fixes the distance between the two posterior lenses 2912. Note that alternative embodiments may also use individual clip-on lenses, for example, without the bridge 2916 connecting the posterior lenses together to form a single clip-on lens assembly.

[0484] The frame 2911 of the clip-on lens assembly 2910 further comprises, or consists essentially of, two protrusions 2918 protruding from the top of the frame 2911. The protrusions 2918 are shaped to fit into corresponding recesses 2919 in the frame 41 of the head mounted display 2922 (see FIG. 29A ). In addition, the frame 2911 includes two clips 2914 protruding from a bottom portion of the frame 2911 and shaped to form a snap fit into slots 2915 in the frame 41 (see FIG. 29C ). To mount the clip-on lens assembly 2910 onto the frame 41, a user may, for example, insert the protrusions 2918 into the recesses 2919 and then pivot the clip-on lens assembly 2910 to the final position shown in FIGS. 29A and 29C with the clips 2914 snapping into the corresponding slots 2915. Removal can be achieved by the reverse process, i.e., pulling the clip 2914 from the slot 2915, pivoting the bottom of the frame 2911 away from the frame 41, and then removing the protrusion 2918 from the recess 2919 in the frame 41.

[0485] Various other mechanical methods of removably attaching the posterior lens assembly to the frame 41 may also or alternatively be used. For example, more clips 2914 and / or protrusions 2918 may be used, and the clips 2914 and / or protrusions 2918 may be replaced and / or supplemented by magnets, components that form a friction fit, adhesives, screws, other fasteners, etc.

[0486] 29C 。 Referring now to FIG. 29D, this figure shows further details of the cross-sectional view of FIG. 29C. Specifically, FIG. 29D shows how lens 2902 and lens 2912 can be sealed to reflective waveguide lens 2904. For example, a first seal 2920 can be placed around front lens 2902 to seal front lens 2902 to the front surface of reflective waveguide lens 2904. FIG. 29E is a perspective view of one of front lenses 2902 showing one example of seal 2920 placed around front lens 2902. In this embodiment, seal 2920 can comprise a double-sided tape, such as a double-sided foam tape, having adhesive on both sides, thereby permanently or semi-permanently adhering front lens 2902 to the front surface of reflective waveguide lens 2904.

[0487] 29D , a second seal 2922 is shown between the frame 2911 of the clip-on lens assembly 2910 and the back surface of the reflective waveguide lens 2904. To make the clip-on lens assembly 2910 more easily removable and / or replaceable, the seal 2922, in some embodiments, may comprise a single-sided tape, such as a foam tape, having adhesive on one side, such as the side connected to the frame 2911. Thus, with such a configuration, the frame 2911 may be sealed to the reflective waveguide lens 2904, but may be easily removed and replaced without leaving any residual adhesive on the reflective waveguide lens 2904. In some embodiments, one or both of the seals 2920, 2922 may also function to act as spacers between the front and rear lenses and the reflective waveguide lens, which may be beneficial in some cases.

[0488] In this embodiment, the posterior lens 2912 desirably attaches to the frame 2911 of the clip-on lens assembly 2910. For example, as seen in FIG. 29C , the posterior lens 2912 may include a hump or protrusion 2930 that fits into a corresponding groove or recess 2932 in the frame 2911. The fit between the frame 2911 and the posterior lens 2912 may desirably be an interference fit that securely holds and seals the posterior lens 2912 to the frame 2911. Additionally or alternatively, the periphery of the posterior lens 2912 may be adhered to the frame 2911 with an adhesive.

[0489] 29A-29E have several beneficial features, including, for example, relatively easy, tool-less installation of a custom prescription rear lens, maintaining a seal between both the front and rear lenses and the reflective waveguide lens, and the potential for relatively rapid manufacturing of customized head-mounted displays. Maintaining a seal between the front and rear lenses and the reflective waveguide lens can be particularly beneficial, for example, in maintaining ideal transparency of the reflective waveguide lens.

[0490] The frame 2911 of the clip-on lens assembly 2910 may be manufactured from a variety of materials. In some embodiments, it may be desirable to manufacture the frame 2911 from PEEK (polyetheretherketone), for example, which may have a relatively desirable strength-to-weight ratio and may be suitable for use with various cleaning procedures. In some embodiments, the weight of the frame 2911 may be approximately or less than 4 g. In some embodiments, other materials may be used for the frame 2911, such as polycarbonate, which may be a more use-efficient material in some cases.

[0491] Referring to FIG. 30 , this figure schematically illustrates how the cross-sectional profile of the frame 2911 of a clip-on lens assembly 2910 can vary with a posterior lens 2912 having different diopter compensation. Specifically, FIG. 30 illustrates cross-sectional views of three different variations of the clip-on lens assembly 2910, namely, clip-on lens assemblies 2910a, 2910b, and 2910c. Each of these variations exhibits a width W that represents the overall width of the clip-on lens assemblies 2910a, 2910b, and 2910c. Clip-on lens assembly 2910b exhibits zero diopter compensation and is the narrowest assembly, with a width W of approximately 4.7 mm. Clip-on lens assembly 2910a illustrates an example with a posterior lens 2912 having +6D diopter compensation, resulting in an assembly width W of approximately 7.7 mm. Finally, clip-on lens assembly 2910c is shown with a rear lens 2912 having -6D diopter compensation, resulting in an assembly width of approximately 10.5mm.

[0492] 31A-31E, these figures show additional views and / or portions of the head mounted display 2122 of FIG. 21A to illustrate a removable lens assembly 3110 (e.g., a clip-on lens assembly, a snap-on lens assembly, etc.). The clip-on lens assembly 3110 of the head mounted display 2122 is similar to the clip-on lens assembly 2910 of the head mounted display 2922 described above with reference to FIGS. 29A-29E, and the same or similar reference numbers are used to refer to the same or similar features.

[0493] One difference in the clip-on lens assembly 3110 of the head-mounted display 2122 is that the outer shape of the rear lens 2912 and the frame 2911 (e.g., rear lens frame) in which the lens fits is different. Specifically, the outer shape has a stepped shape to provide clearance for a portion 3150 of the housing 2104 of the optical engine 55, which is shaped somewhat differently from the corresponding portion of the head-mounted display 2922. Another difference is that the configuration of the protrusion 2918 and the recess 2919 is different. Specifically, the protrusion 2918 is part of the housing 2104, and the recess 2919 is part of the frame 2911, which is opposite to the configuration described above with reference to FIGS. 29A-29E. The clip-on lens assembly 3110 still includes a clip 2914 similar to that of the clip-on lens assembly 2910. Furthermore, the overall cross-sectional structure of the displays 49a, 49b of the head-mounted display 2122 can be functionally similar to the structure described above with reference to FIGS. 29C and 29D.

[0494] 29C and 29D, but shows an alternative screw-on design instead of a clip-on or snap-on design. Specifically, the structure shown in FIG. 32 includes a display assembly frame 41 having a reflective waveguide lens 2904 coupled thereto, a front lens 2902 bonded to the front surface of the waveguide lens 2904 using a seal 2920, and a rear lens 2912 attached to the rear surface of the waveguide lens 2904, with a seal 2922 disposed between the rear lens 2912 and the waveguide lens 2904. However, instead of the frame 2911 being clipped or snapped onto the frame 41, the frame 2911 is screwed to the frame 41 with one or more screws 3202.

[0495] Another difference in the embodiment of FIG. 32 is that the anterior surface of the posterior lens 2912 is shown as flat or substantially flat. Such a design may be useful, for example, to save space within a display assembly. On the other hand, having a flat anterior surface for such optically corrective lenses may not be standard in the prescription lens industry and may therefore be more difficult and / or expensive to manufacture. Thus, in some embodiments, the anterior surface of the posterior lens 2912 may be curved, similar to the posterior lens 2912 of FIG. 29C.

[0496] Although specific examples of the use of the disclosed embodiments are given with respect to body parts including spinal vertebrae, the principles of the disclosure may also be used with bones and / or body parts other than the spine, including the hip bone, pelvic bone, leg bones, arm bones, talus, foot bones, shoulder bones, skull, oral and maxillofacial bones, sacroiliac joints, etc.

[0497] The disclosed embodiments are generally presented in the context of image-guided surgery systems or methods, and therefore the disclosed systems and devices should not be considered limited to only surgical or medical applications, but also contemplate non-medical applications. For example, the disclosed embodiments are applicable to consumer or commercial applications such as athletics and fitness, gaming, driving, product design, navigation, manufacturing, logistics, shopping and commerce, education and training, remote collaboration, and the like.

[0498] The terms "top," "bottom," "first," "second," "upper," "lower," "height," "width," "length," "end," "side," "horizontal," "vertical," and similar terms may be used herein, with the understanding that these terms refer only to the structures illustrated in the figures and are utilized solely to facilitate the description of embodiments of the present disclosure. Various embodiments of the present disclosure are presented in range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the present disclosure. The ranges disclosed herein encompass all overlaps, subranges, and combinations thereof, as well as individual numerical values ​​within the ranges. For example, description of a range such as about 25 to about 45 degrees should be considered to specifically disclose subranges such as 25 to 35 degrees, 30 to 40 degrees, 35 to 45 degrees, etc., as well as individual numbers within that range (e.g., 25, 30, 35, 40, 45, 32, 30.5, and all whole and partial increments therebetween). Language such as "up to," "at least," "greater than," "less than," "between," and the like, includes the recited numbers. Numbers preceded by terms such as "about" or "approximately" include the recited numbers. For example, "approximately 260 g" includes "260 g." As used herein, the terms "approximately," "about," and "substantially" refer to an amount near the recited amount that still performs the desired function or achieves the desired result.

[0499] Some embodiments include various features presented as a single feature (rather than multiple features). For example, in one embodiment, the system includes a single HMD, a single camera, a single processor, a single display, a single flashlight, a single PTA, a single PTA detent mechanism, a single head strap adjustment knob, etc. In alternative embodiments, multiple features or components are provided.

[0500] In some embodiments, the systems disclosed herein comprise one or more of tilting means (e.g., hinges, virtual hinges, arcuate slots, detents, straps configured to flex), adjustment means (e.g., knobs, rack and pinions), imaging means (e.g., cameras, fluoroscopes, MRI machines, CT machines), calibration means (e.g., calibration jigs), alignment means (e.g., adapters, markers, objects, cameras), biasing means (e.g., springs), fastening means (e.g., anchors, adhesives, clamps, pins), segmentation means (e.g., one or more neural networks), etc.

[0501] The processors described herein may include one or more central processing units (CPUs) or processors or microprocessors. The processor may be communicatively coupled to one or more memory units, such as random access memory (RAM) for temporary storage of information, one or more read-only memories (ROM) for persistent storage of information, and one or more mass storage devices, such as hard drives, diskettes, solid-state drives, or optical media storage devices. The processor (or memory units communicatively coupled to the processor) may include modules containing program instructions or algorithm steps configured to be executed by the processor to perform any of the processes or algorithms described herein. The processor may be communicatively coupled to external devices (e.g., display devices, data storage devices, databases, servers, etc.) over a network via a network communication interface.

[0502] In general, the algorithms or processes described herein may be implemented by logic embodied in hardware or firmware, or by a collection of software instructions, possibly having entry and exit points, written in a programming language such as, for example, Python, Java, Lua, C, C#, or C++. Software modules or products may be compiled and linked into executable programs installed in dynamic link libraries, or may be written in an interpreted programming language such as, for example, BASIC, Perl, or Python. It will be understood that software modules may be callable from other modules or from themselves, and / or may be invoked in response to detected events or interrupts. Software modules configured to run on a computing device may be provided on a computer-readable medium, such as a compact disc, digital video disc, flash drive, or any other tangible medium. Such software code may be partially or completely stored in a memory device of a running computing device, such as processing system 31, for execution by the computing device. The software instructions may also be embedded in firmware, such as an EPROM. It will be further understood that a hardware module may be composed of connected logic units such as gates and flip-flops, and / or may be composed of programmable units such as programmable gate arrays or processors. The modules described herein are preferably implemented as software modules, but may also be represented in hardware or firmware. In general, any module or program or flowchart described herein may refer to a logical module that may be combined with other modules or divided into sub-modules, regardless of physical organization or storage.

[0503] The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Additionally, in some implementations, certain method or process blocks or steps may be omitted. The methods and processes described herein are also not limited to any particular order, and the associated blocks, steps, or states may be performed in other orders as appropriate. For example, the described blocks, steps, or states may be performed in an order other than the order specifically disclosed, or multiple blocks or states may be combined into a single block or state. The example blocks, steps, or states may be performed sequentially, in parallel, or in some other manner. Blocks, steps, or states may be added to or deleted from the example embodiments of the disclosure. The example systems and components described herein may be configured differently from that described. For example, elements may be added to, removed from, or rearranged in the example embodiments of the disclosure.

[0504] Any process description, element, or block in the flow diagrams described herein and / or shown in the accompanying drawings should be understood as potentially representing a module, segment, or portion of code that includes one or more executable instructions for implementing a particular logical function or step in the process.

[0505] It will be understood that the systems and methods of the present disclosure each have several innovative aspects, no single one of which is solely responsible for or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of one another or may be combined in various ways. The section headings used herein are provided merely to enhance readability and are not intended to limit the scope of the embodiments disclosed in a particular section to the features or elements disclosed in that section.

[0506] Certain features that are described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as acting in particular combinations and may initially be claimed as such, one or more features from a claimed combination may, in some cases, be deleted from that combination, and a claimed combination may also be directed to a subcombination or a variation of a subcombination. No single feature or group of features is necessary or essential to every embodiment.

[0507] Conditional language, particularly "can," "could," "might," or "may," and the like, unless otherwise specified or understood otherwise within the context of use, is generally intended to convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Thus, such conditional language is not generally intended to imply that features, elements, and / or steps are somehow required by one or more embodiments, nor that one or more embodiments necessarily include logic for determining whether those features, elements, and / or steps are to be included or performed in any particular embodiment, with or without user input or prompting.

[0508] Terms such as "comprising," "including," and "having" are synonymous and are used inclusively and without limitation and do not exclude additional elements, features, acts, operations, etc. Additionally, the term "or" is used in an inclusive sense (rather than an exclusive sense); for example, when used to connect a list of elements, the term "or" may refer to one, some, or all of the elements in the list. Additionally, the articles "a," "an," and "the," as used in this application and the appended claims, should be construed to mean "one or more" or "at least one," unless otherwise specified.

Claims

1. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of a wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; 1. An adjustable strap assembly comprising: a first side strap having a first end connected to the first end of the frame; a second side strap having a first end connected to the second end of the frame; an adjustment mechanism configured to adjust a position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; an adjustable strap assembly comprising: a see-through display assembly pivotally coupled to the frame so that a pantoscopic tilt angle can be adjusted; a first temple housing pivotally coupled to the first end of the frame and slidably coupled to the first side strap of the adjustable strap assembly; a second temple housing pivotally connected to the second end of the frame and slidably connected to the second side strap of the adjustable strap assembly; A head-mounted display device comprising:

2. 2. The head-mounted display device of claim 1, wherein the see-through display assembly is pivotally connected to the frame using a pantoscopic tilt assembly comprising an arc-shaped slot that pivotally connects a portion of the see-through display assembly to a portion of the frame.

3. 3. The head-mounted display device of claim 2, wherein the pantoscopic tilt assembly further comprises a detent mechanism comprising a spring-loaded pin or ball and a plurality of detents, the detent mechanism configured to selectively hold the see-through display assembly in one of a plurality of predetermined positions relative to the frame.

4. 4. The head mounted display device of claim 3, wherein the detent mechanism further comprises a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

5. 10. The head-mounted display device of claim 1, wherein the see-through display assembly comprises a removable lens assembly that can be removed and replaced with a second removable lens assembly to change the lens prescription of the see-through display assembly.

6. The see-through display assembly includes: a display assembly frame; a waveguide lens coupled to the display assembly frame; a front lens secured to the display assembly frame at or in front of the waveguide lens; a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, or a clip; a rear lens fixed to the rear lens frame; The head mounted display device of claim 1 , comprising:

7. The head mounted display device of claim 1 further comprising a flashlight assembly removably coupled to the frame.

8. a first follower slidably connecting the first temple housing to the first side strap; a second follower slidably connecting the second temple housing to the second side strap; The head mounted display device of claim 1 further comprising:

9. The head mounted display device of claim 1 , wherein the frame further comprises nose pads configured to engage the wearer's nose.

10. a first end pivotally connected to the first side strap; a second end pivotally connected to the second side strap; a central support connected to the frame; A forehead support The head mounted display device of claim 1 further comprising:

11. 11. The head-mounted display device of claim 10, wherein the first side strap comprises a connector that pivotally connects a front portion of the first side strap to a rear portion of the first side strap and pivotally connects the first end of the forehead support to the first side strap, and the second side strap comprises a connector that pivotally connects a front portion of the second side strap to a rear portion of the second side strap and pivotally connects the second end of the forehead support to the second side strap.

12. a first follower slidably coupled to the first temple housing and coupled to the first side strap at a location between the connector of the first side strap and the first end of the first side strap; a second follower slidably coupled to the second temple housing and coupled to the second side strap at a location between the connector of the second side strap and the first end of the second side strap; The head mounted display device of claim 11 further comprising:

13. 13. A head mounted display device as described in any one of claims 10 to 12, further comprising an upper strap removably coupled at a first end to the forehead support and at a second end to the adjustment mechanism of the adjustable strap assembly.

14. each of the first side strap and the second strap includes a rack, and the adjustment mechanism of the adjustable strap assembly includes: a pinion engaged with the rack of the first side strap and the rack of the second side strap; a knob configured to rotate the pinion to adjust the circumferential size defined by the first side strap, the second side strap, and the frame; and A head mounted display device according to any one of claims 1 to 12, comprising:

15. The head mounted display device of claim 14 , wherein the adjustment mechanism of the adjustable strap assembly further comprises a tensioning mechanism that resists rotation of the knob until a threshold force is overcome.

16. The head mounted display device of any one of claims 1 to 12, wherein the adjustment mechanism of the adjustable strap assembly further comprises a pad configured to engage the back of the wearer's head.

17. The see-through display assembly is configured to display to the wearer an augmented reality (AR) image including a virtual reality (VR) image presented overlaid on a portion of a patient's body, and the head-mounted display device: one or more processors configured to receive one or more anatomical images of the patient and signals indicative of at least a position of the see-through display assembly relative to the scene, and to render the AR image on the see-through display assembly; The head mounted display device of claim 1 further comprising:

18. The head mounted display device of claim 17 , wherein at least one of the one or more processors is located within the first temple housing or the second temple housing.

19. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of the wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; a frame, further comprising nose pads configured to engage the wearer's nose; 1. An adjustable strap assembly comprising: a first side strap having a first end connected to the first end of the frame; a second side strap having a first end connected to the second end of the frame; an adjustment mechanism configured to adjust a position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; an adjustable strap assembly comprising: a first end pivotally connected to the first side strap; a second end pivotally connected to the second side strap; a central support connected to the frame; a forehead support comprising: a see-through display assembly pivotally coupled to the frame so as to allow adjustment of a pantoscopic tilt angle, a see-through display assembly comprising a removable lens assembly that can be removed and replaced with a second removable lens assembly to change the lens prescription of the see-through display assembly; a first temple housing pivotally coupled to the first end of the frame and slidably coupled to the first side strap of the adjustable strap assembly; a second temple housing pivotally coupled to the second end of the frame and slidably coupled to the second side strap of the adjustable strap assembly; a flashlight assembly removably connected to the frame; A head-mounted display device comprising:

20. 20. The head mounted display device of claim 19, wherein the first side strap comprises a connector that pivotally connects a front portion of the first side strap to a rear portion of the first side strap and pivotally connects the first end of the forehead support to the first side strap, and the second side strap comprises a connector that pivotally connects a front portion of the second side strap to a rear portion of the second side strap and pivotally connects the second end of the forehead support to the second side strap.

21. a first follower slidably coupled to the first temple housing and coupled to the first side strap at a location between the connector of the first side strap and the first end of the first side strap; a second follower slidably coupled to the second temple housing and coupled to the second side strap at a location between the connector of the second side strap and the first end of the second side strap; 21. The head mounted display device of claim 20, further comprising:

22. each of the first side strap and the second strap includes a rack, and the adjustment mechanism of the adjustable strap assembly includes: a pinion engaged with the rack of the first side strap and the rack of the second side strap; a knob configured to rotate the pinion to adjust the circumferential size defined by the first side strap, the second side strap, and the frame; and 20. The head mounted display device of claim 19, comprising:

23. 23. The head mounted display device of claim 22, wherein the adjustment mechanism of the adjustable strap assembly further comprises a tensioning mechanism that resists rotation of the knob until a threshold force is overcome.

24. 24. A head mounted display device as described in any one of claims 19 to 23, wherein the adjustment mechanism of the adjustable strap assembly further comprises a pad configured to engage the back of the wearer's head.

25. 24. A head mounted display device as described in any one of claims 19 to 23, further comprising an upper strap removably coupled at a first end to the forehead support and at a second end to the adjustment mechanism of the adjustable strap assembly.

26. The see-through display assembly is configured to display to the wearer an augmented reality (AR) image including a virtual reality (VR) image presented overlaid on a scene on the patient's body, and the head-mounted display device: one or more processors configured to receive one or more anatomical images of the patient and signals indicative of at least a position of the see-through display assembly relative to the scene, and to render the AR image on the see-through display assembly; The head-mounted display device of any one of claims 19 to 23, further comprising:

27. 27. The head mounted display device of claim 26, wherein at least one of the one or more processors is disposed within the first temple housing or the second temple housing.

28. A head-mounted display device according to any one of claims 19 to 23, wherein each of the first temple housing and the second temple housing comprises a plurality of heat dissipation fins.

29. The see-through display assembly includes: an arcuate slot pivotally connecting a portion of the see-through display assembly to a portion of the frame; a detent mechanism including a spring-loaded pin or ball and a plurality of detents configured to selectively hold the see-through display assembly in one of a plurality of predetermined positions relative to the frame; and 24. A head mounted display device according to any one of claims 19 to 23, pivotally connected to the frame using a pantoscopic tilt assembly comprising:

30. 30. The head mounted display device of claim 29, wherein the detent mechanism further comprises a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

31. The frame is a first rod defining a pivot axis; a second rod disposed parallel to the first rod; A flashlight mounting socket having The flashlight assembly includes: a first recess shaped to engage the first rod of the flashlight mounting socket and pivot about the first rod; a second recess shaped to engage the second rod of the flashlight mounting socket, the second recess oriented such that when the second recess engages the second rod, the first recess cannot disengage the first rod; and a movable latch configured to selectively hold the second recess in engagement with the second rod; The head-mounted display device according to any one of claims 19 to 30, comprising:

32. 32. The head mounted display device of claim 31, further comprising a spring biasing the movable latch toward a position that holds the second recess in engagement with the second rod.

33. The flashlight assembly includes: A flashlight and a mounting base including the first recess, the second recess, and the movable latch; one or more arms pivotally connecting the flashlight to the mounting base; 33. A head-mounted display device according to any one of claims 31 to 32, comprising:

34. The see-through display assembly includes: a display assembly frame; a waveguide lens coupled to the display assembly frame; a front lens secured to the display assembly frame at or in front of the waveguide lens; a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, or a clip; a rear lens fixed to the rear lens frame; A head mounted display device according to any one of claims 19 to 33, comprising:

35. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of the wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; a head-mounted assembly configured to hold the frame in position on the wearer's head; See-through display and a pantoscopic tilt assembly pivotally connecting the see-through display to the frame so that a pantoscopic tilt angle can be adjusted, an arcuate slot pivotally connecting a portion of the see-through display assembly to a portion of the frame; a detent mechanism including a spring-loaded pin or ball and a plurality of detents configured to selectively hold the see-through display assembly in one of a plurality of predetermined positions relative to the frame; and a pantoscopic tilt assembly comprising: A head-mounted display device comprising:

36. 36. The head mounted display device of claim 35, wherein the detent mechanism further comprises a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

37. 36. The head mounted display device of claim 35, wherein the arcuate slot defines a virtual hinge with an axis of rotation configured to be aligned with a center of the wearer's eyeball.

38. The head-mounted assembly is an adjustable strap assembly, a first side strap having a first end connected to the first end of the frame; a second side strap having a first end connected to the second end of the frame; an adjustment mechanism configured to adjust a position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; 38. A head mounted display device according to any one of claims 35 to 37, comprising an adjustable strap assembly comprising:

39. a first end pivotally connected to the first side strap; a second end pivotally connected to the second side strap; a central support connected to the frame; A forehead support 39. The head mounted display device of claim 38, further comprising:

40. a first temple housing pivotally coupled to the first end of the frame and slidably coupled to the head-mounted assembly; a second temple housing pivotally coupled to the second end of the frame and slidably coupled to the head-mounted assembly; one or more processors configured to render images for display by the see-through display, at least one of the one or more processors being located within the first temple housing or the second temple housing; The head-mounted display device of any one of claims 35 to 37, further comprising:

41. 38. A head-mounted display device as described in any one of claims 35 to 37, wherein the head-mounted assembly comprises a first temple arm coupled to the frame and configured to be placed over a first ear of the wearer, and a second temple arm coupled to the frame and configured to be placed over a second ear of the wearer.

42. A head-mounted display device according to any one of claims 35 to 37, wherein the frame further comprises nose pads configured to engage the wearer's nose.

43. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of the wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; 1. An adjustable strap assembly comprising: a first side strap having a first end connected to the first end of the frame; a second side strap having a first end connected to the second end of the frame; an adjustment mechanism configured to adjust a position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; an adjustable strap assembly comprising: a first temple housing pivotally coupled to the first end of the frame and slidably coupled to the first side strap of the adjustable strap assembly; a second temple housing pivotally coupled to the second end of the frame and slidably coupled to the second side strap of the adjustable strap assembly; a see-through display connected to the frame; A head-mounted display device comprising:

44. a first follower slidably connecting the first temple housing to the first side strap; a second follower slidably connecting the second temple housing to the second side strap; 44. The head mounted display device of claim 43, further comprising:

45. the first follower includes an elongated protrusion that is slidable back and forth within an elongated slot of the first temple housing in response to pivoting of the first temple housing relative to the frame; 45. A head mounted display device as described in claim 44, wherein the second follower comprises an elongated protrusion that can slide back and forth within an elongated slot in the second temple housing in response to pivoting of the second temple housing relative to the frame.

46. each of the first side strap and the second strap includes a rack, and the adjustment mechanism of the adjustable strap assembly includes: a pinion engaged with the rack of the first side strap and the rack of the second side strap; a knob configured to rotate the pinion to adjust the circumferential size defined by the first side strap, the second side strap, and the frame; and 46. ​​A head-mounted display device according to any one of claims 43 to 45, comprising:

47. 47. The head mounted display device of claim 46, wherein the adjustment mechanism of the adjustable strap assembly further comprises a tensioning mechanism that resists rotation of the knob until a threshold force is overcome.

48. A head-mounted display device as described in any one of claims 43 to 47, wherein the adjustment mechanism of the adjustable strap assembly further comprises a pad configured to engage the back of the wearer's head.

49. a first end pivotally connected to the first side strap; a second end pivotally connected to the second side strap; a central support connected to the frame; A forehead support 49. The head-mounted display device of any one of claims 43 to 48, further comprising:

50. 50. The head mounted display device of claim 49, further comprising an upper strap removably coupled at a first end to the forehead support and at a second end to the adjustment mechanism of the adjustable strap assembly.

51. A head-mounted display device as described in any one of claims 43 to 50, wherein the frame further comprises nose pads configured to engage the wearer's nose.

52. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of a wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; a head-mounted assembly configured to hold the frame in position on the wearer's head; See-through display and a flashlight assembly removably connected to the frame; Equipped with The frame is a first rod defining a pivot axis; a second rod disposed parallel to the first rod; A flashlight mounting socket having The flashlight assembly includes: a first recess shaped to engage the first rod of the flashlight mounting socket and pivot about the first rod; a second recess shaped to engage the second rod of the flashlight mounting socket, the second recess oriented such that when the second recess engages the second rod, the first recess cannot disengage the first rod; and a movable latch configured to selectively hold the second recess in engagement with the second rod; A head-mounted display device comprising:

53. 53. A head mounted display device as described in claim 52, further comprising a spring biasing the movable latch toward a position that holds the second recess in engagement with the second rod.

54. The flashlight assembly includes: A flashlight and a mounting base including the first recess, the second recess, and the movable latch; one or more arms pivotally connecting the flashlight to the mounting base; 53. The head mounted display device of claim 52, comprising:

55. A head-mounted display device as described in any one of claims 52 to 54, wherein the flashlight mounting socket further comprises one or more electrical contacts configured to electrically couple to corresponding one or more electrical contacts of the flashlight assembly.

56. A head-mounted display device as described in any one of claims 52 to 55, wherein the frame further comprises nose pads configured to engage the wearer's nose.

57. A head-mounted display device, a frame extending from a first end to a second end, the first end configured to be positioned adjacent a first side of a wearer's head and the second end configured to be positioned adjacent a second side of the wearer's head; a head-mounted assembly configured to hold the frame in position on the wearer's head; a see-through display assembly coupled to the frame, a display assembly frame; a waveguide lens coupled to the display assembly frame; a front lens secured to the display assembly frame at or in front of the waveguide lens; a rear lens frame removably coupled to the display assembly frame using at least one of a snap fit, a friction fit, or a clip; a rear lens fixed to the rear lens frame; a see-through display assembly comprising: A head-mounted display device comprising:

58. a first seal between the front lens and the waveguide lens; a second seal between the rear lens frame and the waveguide lens; 58. The head mounted display device of claim 57, further comprising:

59. the rear lens frame includes a first protrusion on an upper portion of the rear lens frame that fits into a first corresponding recess of the display assembly frame; A head-mounted display device as described in any one of claims 57 to 58, wherein the rear lens frame has a second protrusion at the bottom of the rear lens frame that forms a snap fit with a second corresponding recess in the display assembly frame.

60. A head-mounted display device as described in any one of claims 57 to 59, wherein the see-through display assembly is pivotally connected to the frame so that a pantoscopic tilt angle can be adjusted.

61. The see-through display assembly includes: an arcuate slot pivotally connecting a portion of the see-through display assembly to a portion of the frame; a detent mechanism including a spring-loaded pin or ball and a plurality of detents configured to selectively hold the see-through display assembly in one of a plurality of predetermined positions relative to the frame; and 61. A head mounted display device as described in claim 60, wherein the head mounted display device is pivotally coupled to the frame using a pantoscopic tilt assembly comprising:

62. 62. The head mounted display device of claim 61 , wherein the detent mechanism further comprises a guide member slidably engaged with the arcuate slot and configured to apply a force to the spring-loaded pin or ball to move the spring-loaded pin or ball from one of the plurality of detents to another of the plurality of detents.

63. A head-mounted display device as described in any one of claims 57 to 62, wherein the frame further comprises nose pads configured to engage the wearer's nose.

64. a frame configured to be worn on a user's head; a display (i) connected to the frame and configured to rotate relative to the frontal plane of the user to set a pantoscopic tilt angle of the display; and (ii) at least partially transparent and configured to display to the user an augmented reality (AR) image including a virtual reality (VR) image presented overlaid on a scene on the patient's body. A head-mounted display (HMD) comprising: at least one processor configured to: (i) receive one or more anatomical images of the patient and a signal indicative of at least the position of the display relative to the scene; and (ii) render the AR image on the display; A system comprising:

65. The frame is an adjustable head-mounted assembly, an adjustable temple arm configured to rest on the user's ear and be adjusted to conform to at least a portion of the user's temple; an adjustable nose pad configured to be placed over the user's nose and adjusted to conform to the shape of at least a portion of the nose; a housing connected to the temple arms and nose pads; 65. The system of claim 64, comprising an adjustable head-mounted assembly comprising:

66. 66. The system of claim 65, wherein at least one of the temple arms comprises a first section and a second section and a first tilt assembly and a second tilt assembly, the first tilt assembly configured to tilt the first section relative to the frame, and the second tilt assembly configured to rotate the second section relative to the first section.

67. 67. The system of claim 66, wherein the second tilt assembly comprises a rocker arm configured to rotate about a hinge relative to a longitudinal axis of the first section.

68. 68. The system of claim 67, wherein the first section has an opening configured to accommodate at least a portion of the rocker arm when the rocker arm is rotating.

69. 68. The system of claim 67, wherein the rocker arm comprises a cushion made from viscoelastic foam shaped to conform to the user's nape and improve clamping between the frame of the HMD and the user's head.

70. 70. The system of claim 69, wherein the cushion comprises a material selected from the list of materials consisting essentially of at least one of: (i) silicone, (ii) neoprene, and (iii) polyurethane.

71. 70. The system of claim 69, wherein the cushion comprises a sponge.

72. 67. The system of claim 66, wherein the second tilt assembly comprises an alloy coated with a viscoelastic foam and molded to conform to the user's neck.

73. 67. The system of claim 66, wherein (i) the first section and the second section, (ii) the first tilt assembly and the second tilt assembly, and (iii) the nose pads are adapted to provide the user with two or more degrees of freedom (DOF) for adjusting the frame to fit the contours of the user's head.

74. 67. The system of claim 66, wherein the second tilt assembly comprises an array of rocker arms, each of the rocker arms configured to rotate about a respective hinge.

75. 75. The system of claim 74, wherein the array of rocker arms is mounted on a common bar and includes an additional hinge, the common bar being configured to rotate about the additional hinge.

76. The frame is an adjustable head-mounted assembly, a first side strap having a first end connected to a first end of the frame; a second side strap having a first end connected to a second end of the frame; an adjustment mechanism configured to adjust a position of the second end of the first side strap relative to the second end of the second side strap to adjust a circumferential size defined by the first side strap, the second side strap, and the frame; 65. The system of claim 64, comprising a head-mounted assembly comprising:

77. a first temple housing pivotally connected to the first end of the frame and slidably connected to a first side strap; a second temple housing pivotally connected to the second end of the frame and slidably connected to the second side strap; 77. The system of claim 76, further comprising:

78. 78. A system as described in any one of claims 64 to 77, wherein at least one eye has a first optical axis and the display optical engine has a second optical axis, and the pantoscopic tilt angle is set to align the second optical axis with the first optical axis.

79. 79. The system of claim 78, wherein the HMD comprises a pantoscopic tilt assembly (PTA) connected to the frame and the display and configured to rotate the second optical axis relative to the first optical axis to adjust the pantoscopic tilt angle.

80. 80. The system of claim 79, wherein the PTA comprises a hinge connecting between the first optical axis and the second optical axis.

81. 81. The system of claim 80, wherein the PTA comprises a bar coupled to the optical engine comprising the display, an edge of the bar coupled to the hinge, the hinge adapted to rotate the bar relative to the first axis of the frame.

82. 80. The system of claim 79, wherein the PTA comprises at least two portions other than hinges and comprises a virtual axis adapted to rotate the second optical axis relative to the first optical axis.

83. 83. The system of claim 82, wherein the virtual axis comprises: (i) a bar coupled to an optical engine comprising the display and to a rotatable section of a disk having a slit; and (ii) an element configured to be inserted into the slit and moved along the slit in a direction tangential to the slit.

84. 83. The system of claim 82, wherein the virtual axis comprises an arm comprising a first section, a second section, and a third section, the first section coupled to an optical engine comprising the display, the second section coupled to the frame, and the third section coupled between the first section and the second section and configured to bend in response to a force applied to the PTA to adjust the pantoscopic tilt angle.

85. 83. The system of claim 82, wherein the virtual axis comprises (i) a rigid arm connected to the frame and (ii) a flexible arm connected to an optical engine comprising the display, the flexible arm adapted to convert from elastic to plastic deformation and retain the resulting shape in response to a force applied to the PTA to adjust the pantoscopic tilt angle.

86. The PTA is a detent mechanism comprising a spring-loaded pin or ball and a plurality of detents configured to selectively hold the display in one of a plurality of predetermined positions relative to the frame; 81. The system of claim 80, further comprising:

87. a frame extending from a first end to a second end; a head-mounted assembly that is adjustable and configured to hold the frame in position on the user's head; a display that is at least partially transparent and configured to display to the user an augmented reality (AR) image including a virtual reality (VR) image presented overlaid on a scene on the patient's body; a pantoscopic tilt assembly pivotally connecting the display to the frame so that a pantoscopic tilt angle can be adjusted; a first temple housing pivotally connected to the first end of the frame by a first tilt assembly; a second temple housing pivotally connected to the second end of the frame by a second tilt assembly; A head-mounted display comprising: A head-mounted display, wherein at least one of the first temple housing or the second temple housing comprises at least one processor configured to receive one or more anatomical images of the patient and a signal indicative of at least a position of the display relative to the scene, and to render the AR image on the display.

88. 88. The head mounted display of claim 87, wherein the head worn assembly comprises an adjustable head strap configured to engage at least the back of the user's head and the user's forehead.

89. 88. A head mounted display as described in claim 87, wherein the first temple housing and the second temple housing are each part of the head mounted assembly.

90. A head-mounted display as described in any one of claims 88 to 89, wherein the head-mounted assembly further comprises an adjustable nose pad configured to conform to the shape of at least a portion of the user's nose.

91. 88. A head-mounted display as described in claim 87, wherein the pantoscopic tilt assembly comprises a slot arranged to allow a pivot element to slide tangentially relative to the slot, and the pivot element configured to be locked in one or more predetermined positions relative to the slot.

92. 1. A method for pairing a head mounted display with a workstation in a medical center network, the method comprising: Introducing a communication device of the head mounted display into a first network; If the communication device of the head mounted display is previously known to the first network, initiating pairing of the communication device of the head mounted display with a first workstation of the first network using previous connection parameters; initiating pairing of the communication device of the head mounted display with the first workstation of the first network using a key exchange process that generates new connection parameters if the communication device of the head mounted display is not previously known to the first network; exchanging data between the first workstation and the communication device of the head-mounted display during a surgical procedure to enable the head-mounted display to display to a user augmented reality (AR) images including virtual reality (VR) images presented overlaid on a scene of a patient's body; A method comprising:

93. initiating pairing of the communication device of the head mounted display with a second workstation of the first network in response to pairing not being successfully completed within a predetermined time limit.

93. The method of claim 92, further comprising:

94. unpairing the communication device of the head mounted display from the first workstation of the first network; Initiating pairing of the communication device of the head mounted display with a second workstation of a second network; exchanging data between the second workstation and the communication device of the head-mounted display during a surgical procedure to enable the head-mounted display to display to a user augmented reality (AR) images including virtual reality (VR) images presented overlaid on a scene of a patient's body; 93. The method of claim 92, further comprising:

95. 93. The method of claim 92, wherein the head mounted display comprises any of the head mounted display devices of any of claims 1 to 63 or any of the head mounted display devices of claims 64 to 91.

96. 10. A method for image-guided surgery or other medical intervention substantially as described herein.

97. A method and computer software product for performing the functions of the system of any one of the preceding system claims.

98. Apparatus and computer software product for performing the method of any one of the preceding method claims.

99. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the treatment of the spine by surgical intervention.

100. Optionally, use of any of the devices, systems or methods of any one of the preceding claims for the treatment of orthopedic joints by surgical intervention, including shoulder, knee, ankle, hip or other joints.

101. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the treatment of the skull by surgical intervention.

102. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the treatment of a jaw by surgical intervention.

103. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the diagnosis of spinal abnormalities.

104. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the diagnosis of spinal cord injuries.

105. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the diagnosis of joint damage.

106. 10. Use of any of the devices, systems or methods according to any one of the preceding claims for the diagnosis of orthopedic injuries.

107. Use of any of the devices, systems, or methods of any one of the preceding claims in non-medical applications such as gaming, driving, product design, shopping, manufacturing, athletics or fitness, navigation, remote collaboration, and / or education.

108. 10. A head mounted display device, apparatus, or system substantially as described herein.

Images