Imaging modification, display and visualization using augmented and virtual reality glasses

Abstract

The invention relates to imaging modification, display and visualization using augmented and virtual reality glasses. A display system may include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system may include one or more user sensors configured to sense a user, and may include one or more environmental sensors configured to sense an ambient environment of the user. The display system may also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronic device may be configured to sense a condition involving user attention; a user intent for the situation is determined, and a user perception of a real or virtual object within the user's field of view is changed based at least in part on the user intent and / or the sensed situation involving user attention. The processing electronic device may be configured to perform at least one of enhancing or weakening user perception of a real or virtual object within a user's field of view.

Description

[0001] This application is a divisional application of an application with a filing date of July 24, 2017, PCT international application number of PCT / US2017 / 043555, Chinese national phase application number of 201780059031.6, and invention name of “IMAGING MODIFICATION, DISPLAY AND VISUALIZATION USING AUGMENTED AND VIRTUAL REALITY GLASSES”.

[0002] Priority declaration

[0003] This application claims the benefit of priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 62 / 366,599, filed on July 25, 2016, U.S. Provisional Application No. 62 / 396,071, filed on September 16, 2016, and U.S. Provisional Application No. 62 / 440,332, filed on December 29, 2016. The entire disclosure of each of these priority documents is incorporated herein by reference.

[0004] Incorporation by Reference

[0005] This application incorporates by reference the entire contents of each of the following patent applications: U.S. Provisional Application No. 62 / 366,555 filed on July 25, 2016; U.S. Provisional Application No. 62 / 352,539 filed on June 20, 2016; U.S. Provisional Application No. 62 / 333,734 filed on May 9, 2016; U.S. Application No. 14 / 212,961 filed on March 14, 2014; U.S. Application No. 14 / 331,218 filed on July 14, 2014; U.S. Application No. 15 / 072,290 filed on March 16, 2016; and U.S. Provisional Application No. 62 / 294,147 filed on February 11, 2016. Technical Field

[0006] The present disclosure relates to display systems, and more particularly to augmented reality display systems. Background Art

[0007] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which a digitally reproduced image, or a portion thereof, is presented to a user in a manner that appears to be real or can be perceived as real. Virtual reality or "VR" scenes typically involve the presentation of digital or virtual image information that is opaque to other actual real-world visual input; augmented reality or "AR" scenes typically involve the presentation of digital or virtual image information as an enhancement to the visualization of the actual world around the user. Mixed reality or "MR" scenes are "AR" type scenes, and typically involve virtual objects that are integrated into and responsive to the natural world. For example, in an MR scene, AR image content can be blocked by objects in the real world or perceived to interact with objects in the real world.

[0008] refer to Figure 1A , shows an augmented reality scene 1 in which a user of AR technology sees a real-world park-like setting 1100 featuring people, trees, buildings in the background, and a concrete platform 1120. In addition to these items, the user of the AR technology also perceives that he / she "sees" "virtual content," such as a robot statue 1110 standing on the real-world platform 1120, and a cartoon-like avatar character 1130 flying by that appears to be an avatar of a bumblebee, even though these elements 1130, 1110 do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce AR technology that promotes comfortable, natural-feeling, rich presentation of virtual image elements in addition to other virtual or real-world image elements.

[0009] The systems and methods disclosed herein address various challenges associated with AR and VR technologies. Summary of the invention

[0010] The present disclosure provides various examples of display systems. These examples include but are not limited to the following examples.

[0011] 1. A display system configured to project light into eyes of a user to display augmented reality image content in the user's field of view, the head mounted display system comprising:

[0012] a frame configured to be supported on a head of the user;

[0013] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to display augmented reality image content to the user's field of view with different amounts of divergence as if projected from different distances from the user's eyes, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0014] one or more environmental sensors configured to detect the user's surroundings;

[0015] processing electronics in communication with the display and the one or more environmental sensors, the processing electronics being configured to:

[0016] sensing a situation involving increased user focus; and

[0017] A user perception of real or virtual objects within the user's field of view is altered based at least in part on sensing the increased focus.

[0018] 2. A display system according to Example 1, wherein the processing electronic device is configured to change the user perception of a virtual object within the user's field of view by changing the image content including the object.

[0019] 3. A display system according to Example 1 or 2, wherein the processing electronic device is configured to change a user perception of an object by moving the object from a first position to a second position.

[0020] 4. A display system according to Example 3, wherein the first position is a peripheral area and the second position is a central area, or wherein the first position is a central area and the second position is a peripheral area.

[0021] 5. A display system according to any one of Examples 2 to 4, wherein changing the image content includes changing at least one of the contrast, opacity, color, color saturation, color balance, size, brightness, edges, or sharpness of the image content including the object.

[0022] 6. The display system according to any one of Examples 1 to 5 is further configured to provide an alert to the user.

[0023] 7. A display system according to Example 6, wherein the alarm is a visual alarm or an audio alarm.

[0024] 8. A display system according to any of Examples 1 to 7, wherein the one or more environmental sensors include a depth sensor, a binocular world camera pair, a geolocation sensor, a proximity sensor, or a GPS.

[0025] 9. The display system according to any one of Examples 1 to 8 further includes one or more user sensors, wherein the one or more user sensors are configured to sense the user, and the processing electronic device communicates with the one or more user sensors.

[0026] 10. The display system of Example 9, wherein the one or more user sensors include one or more cameras.

[0027] 11. A display system according to any one of Examples 1 to 10, wherein the head-mounted display includes a waveguide, the waveguide includes a diffractive optical element, the diffractive optical element is configured to output light by extracting the light from the waveguide, wherein the waveguide is one of a waveguide stack, and different waveguides in the waveguide stack are configured to output light with different wavefront divergences.

[0028] 12. A display system according to any one of Examples 1 to 11, wherein the processing electronic device is further configured to determine a user intention with respect to the situation, and to change a user perception of the real or virtual objects within the user's field of view based at least in part on the determined user intention.

[0029] 13. A display system according to any of Examples 1 to 12, wherein the processing electronic device is further configured to determine user intent with respect to the situation, and to change user perception of the real or virtual objects within the user's field of view based at least in part on sensing the increased attention.

[0030] 14. A display system according to any one of Examples 1 to 13, wherein the environmental sensor includes a sensor configured to detect radio signals.

[0031] 15. A display system according to any of Examples 1 to 14, wherein the environmental sensor includes a sensor configured to detect a Bluetooth signal from a car.

[0032] 16. A display system according to any of Examples 1 to 15, wherein the situation involving increased user attention includes driving a motor vehicle.

[0033] 17. A display system according to Example 16, wherein the processing electronic device is configured to change the user's perception of the real or virtual objects within the user's field of view based at least in part on one or more data records about the user, the one or more data records including the user's driving record.

[0034] 18. A display system according to any one of Examples 1 to 17, wherein the processing electronic device is configured to change the user's perception of the real or virtual objects within the user's field of view by changing the background.

[0035] 19. A display system according to Example 18, wherein changing the background includes increasing the homogenization of the background, thereby reducing visible features in the background.

[0036] 20. The display system of example 19, wherein increasing the homogenization of the background comprises washing out or painting over features in the background.

[0037] 21. A display system according to any of Examples 18 to 20, wherein the processing electronic device is configured to change the background by increasing the light intensity in the background so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0038] 22. A display system according to any of Examples 1 to 21, wherein the processing electronic device is configured to increase the light intensity so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0039] 23. A display system according to any of Examples 1 to 22, wherein changing the user perception of an object within the user's field of view includes highlighting the object.

[0040] 24. A display system according to Example 23, wherein highlighting the object includes superimposing a partially transparent color on the object.

[0041] 25. A display system according to any of Examples 1 to 24, further comprising one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0042] 26. A display system according to any of Examples 1 to 25, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0043] 27. A display system according to Example 25, wherein the one or more light sources are configured to guide light into the one or more waveguides.

[0044] 28. A display system according to any one of Examples 25 to 27, wherein the light source comprises a fiber scanning projector.

[0045] 29. A display system according to any of Examples 1 to 28, wherein the one or more environmental sensors include one or more outward-facing image capture systems configured to image the environment.

[0046] 30. A display system according to Examples 1 to 29, wherein the environmental sensor includes one or more outward-facing cameras.

[0047] 31. A display system according to any of Examples 1 to 30, wherein one or more environmental sensors include a distance measurement system.

[0048] 32. A display system according to Example 31, wherein the distance measurement system includes a laser rangefinder.

[0049] 33. A display system according to any one of Examples 1 to 32, further comprising an eye tracking system configured to track the position and / or movement of the user's eyes.

[0050] 34. A display system according to any of Examples 1 to 33, further comprising one or more inward-facing image capture systems configured to image the user's eyes.

[0051] 35. A display system according to any of Examples 1 to 34, wherein the head-mounted display system is configured to process image content presented to at least a first portion of the user's field of view differently than image content presented to a second portion of the user's field of view.

[0052] 36. A display system according to any of Examples 1 to 35, wherein the head-mounted display system is configured to process image content presented to at least a portion of the peripheral area of ​​the user's field of view differently than image content presented to a central area of ​​the user's field of view.

[0053] 37. A display system according to any of Examples 1 to 37, wherein changing the user perception includes strengthening or weakening (de-emphasizing).

[0054] 38. A display system according to Example 37, wherein the enhancement provided by the processing electronic device includes magnifying image content.

[0055] 39. A display system according to any of Examples 37 to 38, wherein the enhancement or attenuation provided by the processing electronic device includes changing brightness.

[0056] 40. A display system according to any of Examples 37 to 39, wherein the enhancement provided by the processing electronic device includes increasing brightness.

[0057] 41. A display system according to any of Examples 37 to 40, wherein the attenuation provided by the processing electronic device includes reducing brightness.

[0058] 42. A display system according to any of Examples 37 to 41, wherein the attenuation provided by the processing electronic device includes increasing brightness.

[0059] 43. A display system according to any of Examples 37 to 42, wherein the enhancement or attenuation provided by the processing electronic device includes changing contrast.

[0060] 44. A display system according to any of Examples 37 to 43, wherein the enhancement provided by the processing electronic device includes increasing contrast.

[0061] 45. A display system according to any of Examples 37 to 44, wherein the attenuation provided by the processing electronic device includes reducing contrast.

[0062] 46. ​​A display system according to any of Examples 37 to 45, wherein the enhancement or attenuation provided by the processing electronic device includes changing color saturation.

[0063] 47. A display system according to any of Examples 37 to 46, wherein the enhancement provided by the processing electronic device includes increasing color saturation.

[0064] 48. A display system according to any of Examples 37 to 47, wherein the attenuation provided by the processing electronic device includes reducing color saturation.

[0065] 49. A display system according to any of Examples 37 to 48, wherein the enhancement or attenuation provided by the processing electronic device includes changing sharpness.

[0066] 50. A display system according to any of Examples 37 to 49, wherein the enhancement provided by the processing electronic device includes increasing sharpness.

[0067] 51. A display system according to any of Examples 37 to 50, wherein the attenuation provided by the processing electronic device includes reducing sharpness.

[0068] 52. A display system according to any of Examples 37 to 51, wherein the enhancement or attenuation provided by the processing electronic device includes changing opacity.

[0069] 53. A display system according to any of Examples 37 to 52, wherein the enhancement provided by the processing electronic device includes increasing opacity.

[0070] 54. A display system according to any of Examples 37 to 53, wherein the attenuation provided by the processing electronic device includes reducing opacity.

[0071] 55. A display system according to any of Examples 37 to 54, wherein the attenuation provided by the processing electronic device includes increasing opacity.

[0072] 56. A display system according to any of Examples 37 to 55, wherein the enhancement includes edge enhancement features.

[0073] 57. According to a display system described in any of Examples 37 to 56, the enhancement or attenuation provided by the processing electronic device includes changing the color balance.

[0074] 1. A display system configured to project light into eyes of a user to display augmented reality image content in the user's field of view, the head mounted display system comprising:

[0075] a frame configured to be supported on a head of the user;

[0076] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0077] one or more outward-facing cameras configured to image objects in the user's environment;

[0078] one or more eye tracking sensors configured to determine where the user's eyes are looking;

[0079] processing electronics in communication with the display, the outward-facing camera, and the eye-tracking sensor to control presentation of image content on the display, the processing electronics being configured to:

[0080] Displaying image content to the user's field of view;

[0081] determining the object at which the eye is directed; and

[0082] At least one of enhancing the rendering of the object at which the eye is directed or weakening one or more features around the object at which the eye is directed is performed.

[0083] 2. A display system according to Example 1, wherein the processing electronic device is configured to display the object on the display and enhance the rendering of the object.

[0084] 3. A display system according to Example 1 or 2, wherein the processing electronic device is configured to display the one or more features surrounding the object on the display and to weaken the one or more features surrounding the object at which the eye is pointing when rendered on the display.

[0085] 4. A display system according to any one of Examples 1 to 3, wherein the one or more features surrounding the object include real features in the environment in front of the user and the head-mounted display system, and the processing electronic device is configured to attenuate the one or more real features surrounding the object at which the eye is pointing.

[0086] 5. A display system according to Example 4, wherein the processing electronic device is configured to attenuate the one or more real features surrounding the object at which the eye is pointing by increasing the intensity of light directed into the eye, so that the one or more real features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0087] 6. A display system according to any of Examples 1 to 4, wherein the processing electronic device is configured to increase the intensity of light projected into the eye so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0088] 7. A display system according to any of Examples 1 to 6, wherein the attenuation includes reducing one or more of the brightness, visibility, sharpness, or contrast of the view of the environment through the transparent portion, or changing the color of the environment through the transparent portion.

[0089] 8. A display system according to any of Examples 1 to 7, wherein the processing electronic device is configured to increase the intensity of light projected into the eye so that features in the environment in front of the user and the head-mounted display are less prominent to the user, rather than by displaying an image on the display.

[0090] 9. A display system according to any of Examples 1 to 8, wherein the attenuation includes increasing opacity so that the view of the environment through the transparent portion is attenuated.

[0091] 10. A display system according to any of Examples 1 to 9, wherein the attenuation includes reducing one or more of the brightness, visibility, sharpness, or contrast of the view of the environment through the transparent portion, or changing the color of the environment through the transparent portion, rather than by displaying an image on the display.

[0092] 11. A display system according to any of Examples 1 to 10, wherein the attenuation includes increasing opacity so that the view of the environment through the transparent portion is attenuated, rather than by displaying an image on the display.

[0093] 12. A display system according to any one of Examples 1 to 11, wherein the processing electronic device is configured to change the user's perception of the real or virtual objects within the user's field of view by changing the background.

[0094] 13. A display system according to Example 12, wherein changing the background includes increasing the homogenization of the background, thereby reducing visible features in the background.

[0095] 14. A display system according to Example 13, wherein increasing the homogenization of the background includes clearing or masking features in the background.

[0096] 15. A display system according to any of Examples 12 to 14, wherein the processing electronic device is configured to change the background by increasing the light intensity in the background so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0097] 16. A display system according to any of Examples 1 to 15, wherein the processing electronic device is configured to enhance the rendering of image content displayed by the display.

[0098] 17. A display system according to any of Examples 1 to 16, wherein the processing electronic device is configured to attenuate the rendering of image content displayed by the display.

[0099] 18. A display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0100] a frame configured to be supported on a head of the user;

[0101] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed in a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0102] processing electronics in communication with the display, the processing electronics being configured to:

[0103] Displaying image content to the user's field of view;

[0104] performing at least one of enhancing rendering of the image content displayed by the display or weakening rendering of the image content,

[0105] wherein the enhancing comprises one or more of increasing contrast, color saturation, brightness, edge visibility, opacity or sharpness of the rendered image content relative to other image content displayed by the display, highlighting or changing the color or color balance of the rendered image content, and

[0106] The attenuation includes reducing one or more of the contrast, color saturation, brightness, edge visibility, opacity or sharpness of the rendered image content relative to other image content displayed by the display, or changing the color or color balance of the rendered image content.

[0107] 19. A display system according to Example 18, wherein the processing electronic device is configured to enhance the rendering of image content displayed by the display.

[0108] 20. A display system according to Example 18 or 19, wherein the processing electronic device is configured to attenuate the rendering of image content displayed by the display.

[0109] 21. A display system according to any of Examples 18 to 20, wherein the processing electronic device is configured to enhance the rendering of the image content displayed by the display relative to surrounding image content displayed by the display.

[0110] 22. A display system according to any of Examples 18 to 21, wherein the processing electronic device is configured to attenuate rendering of image content displayed by the display relative to image content displayed by the display surrounded by the attenuated image.

[0111] 23. The display system of any of Examples 18 to 22, further comprising one or more outward-facing cameras configured to image objects in the user's environment.

[0112] 24. A display system according to any of Examples 18 to 23, further comprising one or more eye tracking sensors configured to determine where the user's eyes are looking.

[0113] 25. A head mounted display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0114] a frame configured to be supported on a head of the user;

[0115] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0116] one or more outward-facing cameras configured to image objects in the user's environment;

[0117] processing electronics in communication with the display and the outward-facing camera, the processing electronics being configured to:

[0118] Displaying image content to the user's field of view;

[0119] identifying one or more objects in the user's environment by applying object recognition to images received from the one or more outward-facing cameras; and

[0120] performing at least one of enhancing rendering of the image content displayed by the display or attenuating rendering of the image content based on the object recognition,

[0121] wherein the enhancement comprises one or more of increasing contrast, color saturation, brightness, edge visibility, opacity or sharpness of the rendered image content, highlighting, or changing the color or color balance of the rendered image content, and

[0122] The attenuation includes reducing the contrast, color saturation, brightness, edge visibility, opacity or sharpness of the rendered image content, or changing one or more of the color or color balance of the rendered image content.

[0123] 26. A display system according to Example 25, wherein the one or more outward-facing cameras are disposed on the frame.

[0124] 27. A display system according to Example 25 or 26, wherein the processing electronic device is configured to enhance the rendering of image content displayed by the display.

[0125] 28. A display system according to any of Examples 25 to 27, wherein the processing electronic device is configured to attenuate the rendering of image content displayed by the display.

[0126] 29. A head mounted display system configured to project light into a user's eyes to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0127] a frame configured to be supported on a head of the user;

[0128] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0129] one or more outward-facing cameras configured to image objects in the user's environment;

[0130] processing electronics in communication with the display and the outward-facing camera, the processing electronics being configured to:

[0131] Displaying image content to the user's field of view;

[0132] identifying one or more objects in the user's environment by applying object recognition to images received from the one or more outward-facing cameras; and

[0133] Based on the object recognition, at least a portion of a view of the environment in front of the user presented to the user's eyes through the transparent portion is attenuated.

[0134] 30. A display system according to Example 29, wherein the attenuation includes reducing one or more of the brightness, visibility, sharpness, or contrast of the view of the environment through the transparent portion, or changing the color of the environment through the transparent portion.

[0135] 31. A display system according to Example 29 or 30, wherein the attenuation includes increasing the opacity so that the view of the environment through the transparent portion is attenuated.

[0136] 32. A display system according to any of Examples 29 to 31, wherein the attenuation includes reducing one or more of the brightness, visibility, sharpness, or contrast of the view of the environment through the transparent portion, or changing the color of the environment through the transparent portion, rather than by displaying an image on the display.

[0137] 33. A display system according to any of Examples 29 to 32, wherein the attenuation includes increasing opacity so that the view of the environment through the transparent portion is attenuated, rather than by displaying an image on the display.

[0138] 34. A display system according to any of Examples 29 to 33, wherein the processing electronic device is configured to change the user perception of the real or virtual objects within the user's field of view by changing the background.

[0139] 35. A display system according to Example 34, wherein changing the background includes increasing the homogenization of the background, thereby reducing visible features in the background.

[0140] 36. A display system according to example 35, wherein increasing the homogenization of the background includes clearing or masking features in the background.

[0141] 37. A display system according to any of Examples 29 to 36, wherein the processing electronic device is configured to change the background by increasing the light intensity in the background so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0142] 38. A display system according to any of Examples 29 to 37, wherein the processing electronic device is configured to increase light intensity so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0143] 39. A head mounted display system configured to project light into a user's eyes to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0144] a frame configured to be supported on a head of the user;

[0145] one or more outward-facing sensors configured to sense the user's environment;

[0146] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0147] a processing electronic device in communication with the display and one or more storage systems including a data record about the user, the processing electronic device being configured to:

[0148] Displaying image content to the user's field of view; and

[0149] Based on output from the one or more outward-facing sensors and the one or more data records about the user, a user perception of real or virtual objects within the user's field of view is altered.

[0150] 40. A display system according to Example 39, wherein the one or more outward-facing sensors include one or more outward-facing cameras disposed on the frame.

[0151] 41. A display system according to Example 39 or 40, wherein the one or more outward-facing cameras are configured to image objects in the user's environment, and the processing electronic device is configured to identify one or more objects in the user's environment by applying object recognition to images received from the one or more outward-facing cameras, and based on the object recognition, change the user perception of the real or virtual objects within the user's field of view.

[0152] 42. A display system according to any of Examples 39 to 41, wherein the processing electronic device is configured to sense a situation involving user attention, determine the user's intention with respect to the situation, and change the user's perception of the real or virtual objects within the user's field of view based at least in part on the user's intention.

[0153] 43. A display system according to any of Examples 39 to 42, wherein changing the user perception of a real or virtual object within the user's field of view includes strengthening or weakening at least one of the user perception of the real or virtual object within the user's field of view.

[0154] 44. A display system according to any of Examples 39 to 43, wherein the one or more outward-facing cameras are configured to image objects in the user's environment, and the processing electronic device is configured to identify one or more objects in the user's environment by applying object recognition to images received from the one or more outward-facing cameras, and based on the object recognition, perform at least one of enhancing or weakening the user's perception of the real or virtual objects within the user's field of view.

[0155] 45. A display system according to any one of Examples 39 to 44, wherein the data record includes a driving record of the user.

[0156] 46. ​​A display system according to Example 45, wherein the data record includes whether the user has a driving violation.

[0157] 47. A display system according to any of Examples 45 or 46, wherein the data record includes at least one of the age of the user and whether the user is a teenage driver.

[0158] 48. A head mounted display system configured to project light into a user's eyes to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0159] a frame configured to be supported on a head of the user;

[0160] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0161] one or more outward-facing cameras configured to image objects in the user's environment;

[0162] processing electronics in communication with the display and the outward-facing camera, the processing electronics being configured to:

[0163] Displaying image content to the user's field of view;

[0164] identifying one or more objects in the user's environment by applying object recognition to images received from the one or more outward-facing cameras; and

[0165] Based on the object recognition, at least a portion of a view of the environment in front of the user is highlighted to the user's eyes through the transparent portion.

[0166] 49. A head-mounted display system according to Example 48, wherein highlighting the object includes superimposing a partially transparent color on the object.

[0167] 50. A head-mounted display system according to Example 48 or 49, wherein highlighting the object includes highlighting the object rather than by displaying images from the one or more outward-facing cameras.

[0168] 51. A head mounted display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0169] a head mounted display disposed on a frame, the display being configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0170] an outward-facing camera configured to image objects in the user's environment; and

[0171] processing electronics in communication with the display and the outward-facing camera to control presentation of image content on the display, the processing electronics being configured to:

[0172] determining that the user is operating a vehicle; and

[0173] Objects in the user's field of view are enhanced.

[0174] 52. A display system according to Example 51, wherein enhancing the object in the user's field of view includes moving image content of the object from a first position in the user's field of view to a second position.

[0175] 53. A display system according to Example 51 or 52, wherein enhancing an object in the user's field of view includes changing the perceived color of the object relative to the original color of the object without enhancement.

[0176] 54. A display system according to any of Examples 51 to 53, wherein enhancing objects in the user's field of view includes increasing one or more of contrast, color saturation, brightness, edge visibility, opacity, or sharpness of the displayed image content.

[0177] 55. A display system according to any of Examples 51 to 54, wherein enhancing an object in the user's field of view includes highlighting the object by superimposing a partially transparent color over the object.

[0178] 56. A head mounted display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0179] a frame configured to be supported on a head of the user;

[0180] a database configured to contain one or more user records;

[0181] a head-mounted display disposed on the frame, the display being configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head-mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head-mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head-mounted display; and

[0182] A processing electronic device is in communication with the display and the database, the processing electronic device being configured to reduce an amount of displayed image content based on one or more records of the user.

[0183] 57. A display system according to Example 56, wherein the one or more user records include at least one of a driving record, an accident record, a summons record, a school performance record, a criminal record, or an arrest record.

[0184] 58. A display system according to Example 56 or 57, wherein the one or more user records include the age of the user.

[0185] 59. A display system according to any of Examples 56 to 58, wherein reducing the amount of displayed image content based on one or more records of the user includes selectively enabling the user to access the displayed image content.

[0186] 60. A display system according to any of Examples 56 to 59, wherein reducing the amount of displayed image content based on one or more records of the user includes not displaying the image content.

[0187] 61. A display system according to any of Examples 56 to 59, wherein reducing the amount of image content displayed based on one or more records of the user includes substantially displaying no image content.

[0188] 62. A head mounted display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0189] a frame configured to be supported on a head of the user;

[0190] a head-mounted display disposed on the frame, the display being configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head-mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head-mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head-mounted display; and

[0191] processing electronics in communication with the display, the processing electronics being configured to:

[0192] determining that the user is proximate to the vehicle based at least in part on the established communication link with a processor of the vehicle; and

[0193] The amount of displayed image content is reduced based on the established communication link.

[0194] 63. A head-mounted display system according to Example 62, wherein determining that the user is near the vehicle includes sending and / or receiving at least one of a radio frequency signal or an infrared signal.

[0195] 64. A head-mounted display system according to Example 62, wherein determining that the user is near the vehicle includes at least one of sending and / or receiving radio frequency signals.

[0196] 65. A head-mounted display system according to Example 62, wherein determining that the user is near the vehicle includes at least one of sending and / or receiving wireless signals.

[0197] 66. A head-mounted display system according to Example 62, wherein determining that the user is near the vehicle includes at least one of sending and / or receiving Bluetooth signals.

[0198] 67. A head-mounted display system according to any of Examples 62 or 66, wherein reducing the amount of displayed image content based on the established communication link includes not displaying any image content.

[0199] 68. A head-mounted display system according to any of Examples 62 or 66, wherein reducing the amount of image content displayed based on the established communication link includes displaying substantially no image content.

[0200] 69. A head mounted display system configured to project light into a user's eyes to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0201] a frame configured to be supported on a head of the user;

[0202] a head mounted display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display;

[0203] an outward-facing camera configured to image objects in the user's environment;

[0204] an environmental sensor configured to identify one or more locations of the object; and

[0205] processing electronics in communication with the display, the outward-facing camera, and the environmental sensor, the processing electronics being configured to:

[0206] determining whether the user is operating a vehicle;

[0207] determining a risk of at least one of the vehicle colliding with the object and the object colliding with the vehicle; and

[0208] The amount of image content displayed is reduced based on the determined risk.

[0209] 70. A head-mounted display system according to Example 69, wherein determining the collision risk includes determining a rate at which the object and the vehicle are approaching each other based on the one or more positions of the object identified by the environmental sensor.

[0210] 71. A head-mounted display system according to Example 69 or 70, wherein the environmental sensor configured to identify one or more positions of the object includes at least one of a laser rangefinder, a LIDAR, a radar rangefinder, or an ultrasonic rangefinder.

[0211] 72. A head mounted display system configured to project light into eyes of a user to display augmented reality image content within the user's field of view, the head mounted display system comprising:

[0212] a display disposed on a frame, the display being configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head-mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user and the head-mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head-mounted display;

[0213] an outward-facing camera configured to image objects in the user's environment; and

[0214] processing electronics in communication with the display and the outward-facing camera to control presentation of image content on the display, the processing electronics being configured to:

[0215] determining that the user is operating a vehicle;

[0216] Displaying image content to the user's field of view with different amounts of divergence as if projected from different distances from the user's eyes; and

[0217] Based on a determination that the user is operating a vehicle, an amount of displayed graphical content is reduced.

[0218] Any of the examples above may include any one or more of the features set forth below to produce any of the following examples.

[0219] 73. A display system according to any of Examples 1 to 72, wherein the processing electronic device can be configured to change a user's perception of a virtual object within the user's field of view by changing image content including the object.

[0220] 74. A display system according to Example 73, wherein changing the image content includes changing at least one of the contrast, opacity, color, color saturation, color balance, size, brightness, edges, or sharpness of the image content including the object.

[0221] 75. A display system according to any of Examples 1 to 74, wherein the processing electronic device can be configured to change a user perception of the object by moving the object from a first position to a second position.

[0222] 76. A display system according to Example 75, wherein the first position is a peripheral area and the second position is a central area, or wherein the first position is a central area and the second position is a peripheral area.

[0223] 77. The display system of any one of Examples 1 to 76, further configured to provide an alert to the user.

[0224] 78. A display system according to Example 77, wherein the alarm is a visual alarm or an audio alarm.

[0225] 79. The display system of any one of Examples 1 to 78, further comprising one or more environmental sensors.

[0226] 80. A display system according to any of Examples 1 to 79, wherein the one or more environmental sensors include a depth sensor, a binocular world camera pair, a geolocation sensor, a proximity sensor, or a GPS.

[0227] 81. The display system of any one of Examples 1 to 79, further comprising one or more user sensors.

[0228] 82. A display system according to any of Examples 1 to 81, wherein the one or more user sensors include one or more cameras.

[0229] 83. A display system according to any of Examples 1 to 82, wherein the head-mounted display includes a waveguide, the waveguide including a diffractive optical element, the diffractive optical element configured to output light by extracting the light from the waveguide, wherein the waveguide is one of a waveguide stack, and different waveguides in the waveguide stack are configured to output light with different wavefront divergences.

[0230] 84. A display system according to any of Examples 1 to 83, wherein the processing electronic device is further configured to determine a user intention with respect to the situation, and to change a user perception of the real or virtual objects within the user's field of view based at least in part on the determined user intention.

[0231] 85. A display system according to any of Examples 1 to 84, wherein the processing electronic device is further configured to determine user intention with respect to the situation, and to change user perception of the real or virtual objects within the user's field of view based at least in part on sensing the increased attention.

[0232] 86. A display system according to any of Examples 1 to 85, wherein the environmental sensor includes a sensor configured to detect radio signals.

[0233] 87. A display system according to any of Examples 1 to 86, wherein the environmental sensor includes a sensor configured to detect a Bluetooth signal from a car.

[0234] 88. A display system according to any of Examples 1 to 87, wherein the processing electronic device is configured to determine whether the user is driving a motor vehicle.

[0235] 89. A display system according to any of Examples 1 to 88, wherein the processing electronic device is configured to change the user's perception of real or virtual objects within the user's field of view based at least in part on one or more data records about the user, the one or more data records including the user's driving record.

[0236] 90. A display system according to Examples 1 to 89, wherein the processing electronic device is configured to change the background.

[0237] 91. A display system according to Example 90, wherein changing the background includes increasing homogenization of the background, thereby reducing visible features in the background.

[0238] 92. A display system according to Example 91, wherein increasing the homogenization of the background includes clearing or masking the background.

[0239] 93. A display system according to any of Examples 1 to 92, wherein the processing electronic device is configured to change the background by increasing the light intensity in the background so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0240] 94. A display system according to any of Examples 1 to 93, wherein the processing electronic device is configured to increase the light intensity so that features in the environment in front of the user and the head-mounted display are less prominent to the user.

[0241] 95. A display system according to any of Examples 1 to 94, wherein the processing electronic device is configured to change a user's perception of an object within the user's field of view by highlighting the object.

[0242] 96. A display system according to Example 95, wherein highlighting the object includes superimposing a partially transparent color on the object.

[0243] 97. A display system according to any of Examples 1 to 96, further comprising one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0244] 98. A display system according to Example 97, wherein the one or more light sources are configured to guide light into one or more waveguides.

[0245] 99. A display system according to any of Examples 97 or 98, wherein the one or more light sources include a fiber scanning projector.

[0246] 100. A display system according to any of Examples 1 to 99, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0247] 101. The display system of any one of Examples 1 to 100, further comprising one or more sensors configured to monitor the environment.

[0248] 102. A display system according to Example 101, wherein the one or more sensors include one or more outward-facing image capture systems configured to image the environment.

[0249] 103. A display system according to Example 102, wherein the one or more outward-facing image capture systems configured to image the environment include one or more outward-facing cameras.

[0250] 104. A display system according to any of Examples 101 to 103, wherein one or more sensors include a distance measurement system.

[0251] 105. A display system according to Example 104, wherein the distance measurement system includes a laser rangefinder.

[0252] 106. The display system of any one of Examples 1 to 105, further comprising an eye tracking system configured to track the position and / or movement of the user's eyes.

[0253] 107. The display system of any of Examples 1 to 106, further comprising one or more inward-facing image capture systems configured to image the user's eyes.

[0254] 108. A display system according to any of Examples 1 to 107, wherein the head-mounted display system is configured to process image content presented to at least a first portion of the user's field of view differently than image content presented to a second portion of the user's field of view.

[0255] 109. A display system according to any of Examples 1 to 108, wherein the head-mounted display system is configured to process image content presented to at least a portion of the peripheral area of ​​the user's field of view differently than image content presented to a central area of ​​the user's field of view.

[0256] 110. A display system according to any of Examples 1 to 109, wherein the enhancement provided by the processing electronic device includes magnifying image content.

[0257] 111. A display system according to any of Examples 1 to 110, wherein the enhancement or attenuation provided by the processing electronic device includes changing brightness.

[0258] 112. A display system according to any of Examples 1 to 111, wherein the enhancement provided by the processing electronic device includes increasing brightness.

[0259] 113. A display system according to any of Examples 1 to 112, wherein the attenuation provided by the processing electronic device includes reducing brightness.

[0260] 114. A display system according to any of Examples 1 to 113, wherein the attenuation provided by the processing electronic device includes increasing brightness.

[0261] 115. A display system according to any of Examples 1 to 114, wherein the enhancement or attenuation provided by the processing electronic device includes changing contrast.

[0262] 116. A display system according to any of Examples 1 to 115, wherein the enhancement provided by the processing electronic device includes increasing contrast.

[0263] 117. A display system according to any of Examples 1 to 116, wherein the attenuation provided by the processing electronic device includes reducing contrast.

[0264] 118. A display system according to any of Examples 1 to 117, wherein the enhancement or attenuation provided by the processing electronic device includes changing color saturation.

[0265] 119. A display system according to any of Examples 1 to 118, wherein the enhancement provided by the processing electronic device includes increasing color saturation.

[0266] 120. A display system according to any of Examples 1 to 119, wherein the attenuation provided by the processing electronic device includes reducing color saturation.

[0267] 121. A display system according to any of Examples 1 to 120, wherein the enhancement or attenuation provided by the processing electronic device includes changing sharpness.

[0268] 122. A display system according to any of Examples 1 to 121, wherein the enhancement provided by the processing electronic device includes increasing sharpness.

[0269] 123. A display system according to any of Examples 1 to 122, wherein the attenuation provided by the processing electronic device includes reducing sharpness.

[0270] 124. A display system according to any of Examples 1 to 123, wherein the enhancement or attenuation provided by the processing electronic device includes changing opacity.

[0271] 125. A display system according to any of Examples 1 to 124, wherein the enhancement provided by the processing electronic device includes increasing opacity.

[0272] 126. A display system according to any of Examples 1 to 125, wherein the attenuation provided by the processing electronic device includes reducing opacity.

[0273] 127. A display system according to any of Examples 1 to 126, wherein the attenuation provided by the processing electronic device includes increasing opacity.

[0274] 128. A display system according to any of Examples 1 to 127, wherein the enhancement includes edge enhancement features.

[0275] 129. A display system according to any one of Examples 1 to 128, wherein the enhancement or attenuation provided by the processing electronic device comprises changing the color balance.

[0276] Other examples are provided below.

[0277] Example Set IA

[0278] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0279] a frame configured to be supported on a head of the user;

[0280] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0281] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0282] The head mounted display system is configured to present enhanced image content to the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0283] 2. The system of Example 1 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0284] 3. A system according to Example 1 or 2, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0285] 4. The system of Example 3, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0286] 5. A system according to any one of Examples 2 to 4, wherein the light source comprises a fiber scanning projector.

[0287] 6. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0288] 7. A system according to Example 6, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0289] 8. The system of Example 7, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0290] 9. A system according to any of Examples 6 to 8, wherein one or more sensors include a distance measuring device.

[0291] 10. The system of Example 9, wherein the distance measurement system comprises a laser rangefinder.

[0292] 11. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0293] 12. The system of Examples 1 or 11, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[0294] 13. A system according to any of the above examples, wherein the head-mounted display system is configured to process image content presented to at least a portion of the peripheral area of ​​the user's field of view differently than image content presented to the central area of ​​the user's field of view.

[0295] 14. A system according to Example 13, wherein the head-mounted display system is configured to process image content differently by magnifying image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0296] 15. A system according to any of Examples 13 to 14, wherein the head-mounted display system is configured to process image content differently by increasing the brightness of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0297] 16. A system according to any of Examples 13 to 15, wherein the head-mounted display system is configured to process image content differently by increasing the contrast of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0298] 17. A system according to any of Examples 13 to 16, wherein the head-mounted display system is configured to process image content differently by increasing the color saturation of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0299] 18. A system according to any of Examples 13 to 17, wherein the head-mounted display system is configured to process image content differently by sharpening image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0300] 19. A system according to Example 18, wherein sharpening includes edge enhancement of features in image content presented to at least a portion of the peripheral region of the user's field of view compared to image content presented to the central region of the user's field of view.

[0301] 20. A system according to any of Examples 13 to 19, wherein the head-mounted display system is configured to process image content differently by changing the color balance of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0302] 21. A head mounted display system configured to project light into an eye of a user to display augmented reality image content, the eye of the user having a field of view, the field of view having a central region and a peripheral region disposed around the central region, the head mounted display system comprising:

[0303] a frame configured to be supported on a head of the user;

[0304] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central region and the peripheral region of the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0305] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0306] The head mounted display system is configured to present attenuated image content to the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0307] 22. The system of Example 21 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0308] 23. A system according to Example 21 or 22, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0309] 24. The system of Example 23, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0310] 25. A system according to any of Examples 22 to 24, wherein the light source comprises a fiber scanning projector.

[0311] 26. The system of any one of Examples 21 to 25, further comprising one or more sensors configured to monitor the environment.

[0312] 27. A system according to Example 26, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0313] 28. A system according to Example 27, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0314] 29. A system according to any of Examples 26 to 28, wherein one or more sensors include a distance measuring device.

[0315] 30. The system of Example 29, wherein the distance measuring device comprises a laser rangefinder.

[0316] 31. The system of any one of Examples 21 to 30, further comprising an eye tracking device configured to track the position and / or movement of the user's eyes.

[0317] 32. The system of Examples 21 to 31 further comprises one or more inward-facing image capture devices configured to image the user's eyes.

[0318] 33. A system according to any of Examples 21 to 32, wherein the head-mounted display system is configured to process image content presented to at least a portion of the central area of ​​the user's field of view differently than image content presented to the peripheral area of ​​the user's field of view.

[0319] 34. A system according to Example 33, wherein the head-mounted display system is configured to process image content differently by blurring at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0320] 35. A system according to any of Examples 33 to 34, wherein the head-mounted display system is configured to process image content differently by dimming or attenuating image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0321] 36. A system according to any of Examples 33 to 35, wherein the head-mounted display system is configured to process image content differently by reducing the contrast of image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0322] 37. A system according to any of Examples 33 to 36, wherein the head-mounted display system is configured to process image content differently by reducing the color saturation of image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0323] 38. A system according to any of Examples 33 to 37, wherein the head-mounted display system is configured to process image content differently by reducing the sharpness of image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0324] 39. A system according to Example 38, wherein reducing sharpness includes removing enhancement of edges of features in image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0325] 40. A system according to any of Examples 33 to 39, wherein the head-mounted display system is configured to process image content differently by changing the color balance of image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0326] 41. A system according to any of Examples 33 to 40, wherein the head-mounted display system is configured to process image content differently by reducing image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0327] 42. A system according to Example 14, wherein the magnification is based at least in part on a resolution of the eye.

[0328] 43. A system according to Example 34, wherein the blurring includes blurring the image content presented to at least a portion of the central area of ​​the user's field of view using the same color.

[0329] 44. A system according to Example 44, wherein the same color comprises a high contrast color compared to the color in the image content presented to the peripheral area of ​​the user's field of view.

[0330] 45. A system according to any of Examples 1 to 20 or 42, wherein the system is configured to provide an alert to the user to indicate that the presented image has been enhanced.

[0331] 46. ​​A system according to any one of Examples 21 to 41 or any one of Examples 43 to 44, wherein the system is configured to provide an alert to the user to indicate that the presented image has been impaired.

[0332] 47. A system according to any of Examples 45 or 46, wherein the alarm is a visual alarm.

[0333] 48. A system according to any of Examples 45 or 46, wherein the alarm is an audio alarm.

[0334] 49. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and is configured to interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0335] 50. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0336] 51. The system of Example 50, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the eyes of the user.

[0337] 52. A system according to any of Examples 50 to 51, wherein the one or more light sources include a fiber scanning projector.

[0338] 53. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0339] 54. A system according to Example 53, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0340] 55. A system according to Example 54, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0341] 56. The system according to any of the above examples further includes a distance measuring device.

[0342] 57. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0343] 58. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0344] 59. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0345] 60. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a first image modality and a second image modality presented on the display.

[0346] 61. A system according to Example 60, wherein the first image modality comprises an MRI scan.

[0347] 62. A system according to any of Examples 60 to 61, wherein the second imaging modality comprises ultrasound.

[0348] 63. A system according to any of Examples 60 to 62, wherein the first imaging modality comprises an x-ray scan.

[0349] 64. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[0350] 65. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0351] 66. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual fiducial marker on the portion of the environment in front of the user to the user's eyes.

[0352] 67. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0353] 68. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guideline so that the virtual cutting guideline appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0354] 69. A system according to example 68, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0355] 70. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in the portion of the environment in front of the user.

[0356] 71. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0357] 72. A system according to Example 71, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0358] 73. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0359] 74. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of an image of a 3D object based on user input.

[0360] 75. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0361] 76. A system according to Example 75, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0362] 77. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the image of the transmitted portion of the environment.

[0363] 78. A system according to any of the above examples, wherein the head-mounted display is configured to alert the user to steps in the medical procedure.

[0364] 79. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0365] 80. A system according to Example 79, wherein the user's medical parameters include vital signs.

[0366] 81. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0367] 82. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0368] 83. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0369] 84. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0370] 85. A system according to Example 16, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0371] 86. A system according to Example 16, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[0372] 87. A system according to any of the above examples, wherein the head-mounted display is configured to provide an opacity at least near the presented image content.

[0373] 88. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0374] 89. A system according to example 88, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0375] 90. A system according to Example 88 or 89, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0376] Example Set IB

[0377] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0378] a frame configured to be supported on a head of the user;

[0379] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0380] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0381] The head mounted display system is configured to present enhanced image content to the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0382] 2. The system of Example 1 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0383] 3. A system according to Example 1 or 2, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0384] 4. The system of Example 3, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0385] 5. A system according to any one of Examples 2 to 4, wherein the light source comprises a fiber scanning projector.

[0386] 6. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0387] 7. A system according to Example 6, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0388] 8. The system of Example 7, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0389] 9. A system according to any of Examples 6 to 8, wherein one or more sensors include a distance measuring device.

[0390] 10. The system of Example 9, wherein the distance measuring device comprises a laser rangefinder.

[0391] 11. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0392] 12. The system of Examples 1 or 11, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[0393] 13. A system according to any of the above examples, wherein the head-mounted display system is configured to process image content presented to at least a portion of the central area of ​​the user's field of view differently than image content presented to the peripheral area of ​​the user's field of view.

[0394] 14. A system according to Example 13, wherein the head-mounted display system is configured to process image content differently by magnifying image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0395] 15. A system according to any of Examples 13 to 14, wherein the head-mounted display system is configured to process image content differently by increasing the brightness of image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0396] 16. A system according to any of Examples 13 to 15, wherein the head-mounted display system is configured to process image content differently by increasing the contrast of image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0397] 17. A system according to any of Examples 13 to 16, wherein the head-mounted display system is configured to process image content differently by increasing the color saturation of image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0398] 18. A system according to any of Examples 13 to 17, wherein the head-mounted display system is configured to process image content differently by sharpening image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0399] 19. A system according to Example 18, wherein sharpening includes edge enhancing features in image content presented to at least a portion of the central region of the user's field of view compared to image content presented to the peripheral region of the user's field of view.

[0400] 20. A system according to any of Examples 13 to 19, wherein the head-mounted display system is configured to process image content differently by changing the color balance of image content presented to at least a portion of the central area of ​​the user's field of view compared to image content presented to the peripheral area of ​​the user's field of view.

[0401] 21. A head mounted display system configured to project light into an eye of a user to display augmented reality image content, the eye of the user having a field of view, the field of view having a central region and a peripheral region disposed around the central region, the head mounted display system comprising:

[0402] a frame configured to be supported on a head of the user;

[0403] a display disposed on the frame, the display configured to project light into the user's eyes to present image content at the central region and peripheral regions of the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0404] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0405] The head mounted display system is configured to present attenuated image content to the peripheral region of the user's field of view compared to image content presented to the central region of the user's field of view.

[0406] 22. The system of Example 21 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0407] 23. A system according to Example 21 or 22, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0408] 24. The system of Example 23, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0409] 25. A system according to any of Examples 22 to 24, wherein the light source comprises a fiber scanning projector.

[0410] 26. The system of any one of Examples 21 to 25, further comprising one or more sensors configured to monitor the environment.

[0411] 27. A system according to Example 26, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0412] 28. A system according to Example 27, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0413] 29. A system according to any of Examples 26 to 28, wherein one or more sensors include a distance measuring device.

[0414] 30. The system of Example 29, wherein the distance measuring device comprises a laser rangefinder.

[0415] 31. The system of any one of Examples 21 to 30, further comprising an eye tracking device configured to track the position and / or movement of the user's eyes.

[0416] 32. The system of Examples 21 to 31 further comprises one or more inward-facing image capture devices configured to image the user's eyes.

[0417] 33. A system according to any of Examples 21 to 32, wherein the head-mounted display system is configured to process image content presented to at least a portion of the peripheral area of ​​the user's field of view differently than image content presented to the central area of ​​the user's field of view.

[0418] 34. A system according to Example 33, wherein the head-mounted display system is configured to process image content differently by blurring at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0419] 35. A system according to any of Examples 33 to 34, wherein the head-mounted display system is configured to process image content differently by dimming or attenuating image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0420] 36. A system according to any of Examples 33 to 35, wherein the head-mounted display system is configured to process image content differently by reducing the contrast of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0421] 37. A system according to any of Examples 33 to 36, wherein the head-mounted display system is configured to process image content differently by reducing the color saturation of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0422] 38. A system according to any of Examples 33 to 37, wherein the head-mounted display system is configured to process image content differently by reducing the sharpness of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0423] 39. A system according to Example 38, wherein reducing sharpness includes removing enhancement of edges of features in image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0424] 40. A system according to any of Examples 33 to 39, wherein the head-mounted display system is configured to process image content differently by changing the color balance of image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0425] 41. A system according to any of Examples 33 to 40, wherein the head-mounted display system is configured to process image content differently by reducing image content presented to at least a portion of the peripheral area of ​​the user's field of view compared to image content presented to the central area of ​​the user's field of view.

[0426] 42. A system according to Example 14, wherein the magnification is based at least in part on a resolution of the eye.

[0427] 43. A system according to Example 34, wherein the blurring includes blurring the image content presented to at least a portion of the peripheral area of ​​the user's field of view using the same color.

[0428] 44. A system according to Example 44, wherein the same color comprises a high contrast color compared to the color in the image content presented to the central area of ​​the user's field of view.

[0429] 45. A system according to any of Examples 1 to 20 or 42, wherein the system is configured to provide an alert to the user to indicate that the presented image has been enhanced.

[0430] 46. ​​A system according to any one of Examples 21 to 41 or any one of Examples 43 to 44, wherein the system is configured to provide an alert to the user to indicate that the presented image has been impaired.

[0431] 47. A system according to any of Examples 45 or 46, wherein the alarm is a visual alarm.

[0432] 48. A system according to any of Examples 45 or 46, wherein the alarm is an audio alarm.

[0433] 49. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0434] 50. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0435] 51. The system of Example 50, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the eyes of the user.

[0436] 52. A system according to any of Examples 50 to 51, wherein the one or more light sources include a fiber scanning projector.

[0437] 53. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0438] 54. A system according to Example 53, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0439] 55. A system according to Example 54, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0440] 56. The system according to any of the above examples further includes a distance measuring device.

[0441] 57. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0442] 58. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0443] 59. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0444] 60. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0445] 61. A system according to Example 60, wherein the first image modality comprises an MRI scan.

[0446] 62. A system according to any of Examples 60 to 61, wherein the second imaging modality comprises ultrasound.

[0447] 63. A system according to any of Examples 60 to 61, wherein the first imaging modality comprises an x-ray scan.

[0448] 64. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[0449] 65. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0450] 66. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[0451] 67. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0452] 68. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0453] 69. A system according to example 68, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0454] 70. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in the portion of the environment in front of the user.

[0455] 71. A system according to any of the above examples, wherein the head-mounted display is configured to use an object location database to obtain the position of objects in the portion of the environment in front of the user.

[0456] 72. A system according to Example 71, wherein the head-mounted display is configured to set a reference point based on the object location database and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0457] 73. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0458] 74. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0459] 75. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0460] 76. A system according to Example 76, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0461] 77. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0462] 78. A system according to any of the above examples, wherein the head-mounted display is configured to alert the user to steps in the medical procedure.

[0463] 79. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0464] 80. A system according to Example 79, wherein the user's medical parameters include vital signs.

[0465] 81. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0466] 82. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0467] 83. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0468] 84. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0469] 85. A system according to Example 16, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0470] 86. A system according to Example 16, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[0471] 87. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least near the presented image content.

[0472] 88. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0473] 89. A system according to example 88, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0474] 90. A system according to Example 88 or 89, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0475] Example Set IIA

[0476] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0477] a frame configured to be supported on a head of the user;

[0478] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user at multiple depth planes, at least a portion of the display comprising one or more waveguides, the one or more waveguides being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user, the central region of the user's eye's field of view corresponding to a central region in the environment in front of the user, and the peripheral region of the user's eye's field of view corresponding to a peripheral region in the environment in front of the user;

[0479] an outward-facing image capture device configured to image the at least a portion of the environment in front of the user;

[0480] one or more sensors configured to measure distance to objects in the at least a portion of the environment in front of the user;

[0481] one or more input devices configured to receive input from the user;

[0482] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0483] wherein the head-mounted display system is configured to select an object in the environment corresponding to the peripheral area of ​​the user's field of view based on input received by the one or more input devices, the one or more sensors are configured to measure the distance to the object after the selection, the outward-facing image capture device is configured to obtain an image of the object, and the display is configured to present an enhanced image of the object at a depth plane determined based on the distance measured by the one or more sensors configured to measure the distance, the enhanced image being enhanced compared to the image in other parts of the field of view, and the enhanced image being presented at a location in the peripheral area of ​​the user's field of view.

[0484] 2. The system of Example 1 further comprises one or more light sources configured to direct light into a person's eye to form an image in the eye.

[0485] 3. The system of Example 2, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0486] 4. A system according to Example 2 or 3, wherein the one or more light sources include a fiber scanning projector.

[0487] 5. A system according to Example 1, wherein the depth plane includes a first far depth plane and a second near depth plane, and when the user wears the head-mounted display, the first far depth plane is farther from the user's eyes than the second near depth plane.

[0488] 6. A system according to Example 5, wherein the enhanced image is presented on the far depth plane.

[0489] 7. A system according to Example 5 or 6, wherein the display includes an optical element having optical power so that the light projected into the eye diverges so as to present image content from the second near depth plane.

[0490] 8. A system according to Example 7, wherein the optical element having optical focal length includes a lens.

[0491] 9. A system according to Example 7 or 8, wherein the optical element having optical focal length includes a diffractive optical element.

[0492] 10. A system according to Example 1, wherein the head-mounted display system is configured to present the enhanced image content at a location in the peripheral area of ​​the user's field of view, which corresponds to a location in the peripheral area of ​​the environment in which the object is located.

[0493] 11. A system according to Example 1, wherein the head-mounted display system is configured to move the enhanced image content to a location in the peripheral area of ​​the user's field of view that does not correspond to the peripheral area in the environment in which the object is located.

[0494] 12. A system according to Example 1, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0495] 13. A system according to Example 12, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0496] 14. The system of Example 1, wherein one or more sensors comprises a distance measuring device.

[0497] 15. The system of Example 14, wherein the distance measurement system comprises a laser rangefinder.

[0498] 16. A system according to Example 1, wherein the one or more input devices configured to receive input from the user include an inward-facing eye-tracking camera configured to image the user's eyes and track their movements.

[0499] 17. A system according to any of the above examples, wherein presenting the enhanced image includes processing the image of the object differently than other portions of the user's field of view outside the location where the image of the object is formed.

[0500] 18. A system according to Example 17, wherein the differently processing the image includes magnifying the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0501] 19. A system according to any of Examples 17 to 18, wherein the differently processing the image includes increasing the brightness in the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0502] 20. A system according to any of Examples 17 to 19, wherein the differently processing the image includes increasing the contrast of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0503] 21. A system according to any of Examples 17 to 20, wherein the differently processing the image includes increasing the color saturation of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0504] 22. A system according to any of Examples 17 to 21, wherein the differently processing the image includes sharpening the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0505] 23. A system according to Example 22, wherein the sharpening includes edge enhancement of features of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0506] 24. A system according to any of Examples 17 to 23, wherein processing the image differently includes changing the color balance of the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0507] 25. A system according to any of the above examples, wherein the other portions of the field of view include other portions of the peripheral area of ​​the field of view.

[0508] 26. A system according to any of the above examples, wherein the other portion of the field of view includes at least a portion of the central area of ​​the field of view.

[0509] 27. A head mounted display system configured to project light into an eye of a user to display augmented reality image content, the eye of the user having a field of view, the field of view having a central region and a peripheral region disposed around the central region, the head mounted display system comprising:

[0510] a frame configured to be supported on a head of the user;

[0511] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user at multiple depth planes, at least a portion of the display comprising one or more waveguides, the one or more waveguides being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user, the central region of the user's eye's field of view corresponding to a central region in the environment in front of the user, and the peripheral region of the user's eye's field of view corresponding to a peripheral region in the environment in front of the user;

[0512] an outward-facing image capture device configured to image the at least a portion of the environment in front of the user;

[0513] one or more sensors configured to measure distances to objects in the at least a portion of the environment in front of the user;

[0514] one or more input devices configured to receive input from the user;

[0515] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0516] wherein the head-mounted display system is configured to select an object in the environment corresponding to the peripheral area of ​​the user's field of view based on input received by the one or more input devices, the one or more sensors are configured to measure the distance to the object after the selection, the outward-facing image capture device is configured to obtain an image of the object, and the display is configured to present the image of the object at a depth plane determined based on the distance measured by the one or more sensors configured to measure the distance, the image of the object is presented at a location in the peripheral area of ​​the user's field of view, and the display is configured to attenuate images formed in other parts of the field of view compared to the image of the object.

[0517] 28. The system of Example 27 further comprises one or more light sources configured to direct light into a person's eye to form an image in the eye.

[0518] 29. The system of Example 28, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0519] 30. The system of Examples 28 or 29, wherein the one or more light sources comprises a fiber scanning projector.

[0520] 31. A system according to Example 27, wherein the depth plane includes a first far depth plane and a second near depth plane, and when the user wears the head-mounted display, the first far depth plane is farther from the user's eyes than the second near depth plane.

[0521] 32. A system according to Example 31, wherein the image of the object is presented on the far depth plane.

[0522] 33. A system according to Example 31 or 32, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the second near depth plane.

[0523] 34. A system according to Example 33, wherein the optical element having optical focality includes a lens.

[0524] 35. A system according to Example 33 or 34, wherein the optical element having optical focal length includes a diffractive optical element.

[0525] 36. A system according to Example 27, wherein the head-mounted display system is configured to present the image of the object at a location in the peripheral area of ​​the user's field of view that corresponds to a location in the peripheral area of ​​the environment in which the object is located.

[0526] 37. A system according to Example 27, wherein the head-mounted display system is configured to move the image of the object to a location in the peripheral area of ​​the user's field of view that does not correspond to the peripheral area in the environment in which the object is located.

[0527] 38. A system according to Example 27, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0528] 39. A system according to Example 38, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0529] 40. The system of Example 27, wherein the one or more sensors include a distance measuring device.

[0530] 41. A system according to Example 40, wherein the distance measuring device includes a laser rangefinder.

[0531] 42. A system according to Example 27, wherein the one or more input devices configured to receive input from the user include an inward-facing eye-tracking camera configured to image the user's eyes and track their movements.

[0532] 43. A system according to any of Examples 27 to 42, wherein the head-mounted display system is configured to process images formed in other portions of the user's field of view outside of the location where the image of the object is formed differently than the image of the object.

[0533] 44. A system according to Example 43, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing or reducing the size of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0534] 45. A system according to any of Examples 43 to 44, wherein the head-mounted display system is configured to process images differently than the image of the object by darkening or attenuating images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0535] 46. ​​A system according to any of Examples 43 to 45, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the contrast of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0536] 47. A system according to any of Examples 43 to 46, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the color saturation of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0537] 48. A system according to any of Examples 43 to 47, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the sharpness of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0538] 49. A system according to Example 48, wherein the reducing sharpness includes removing enhancement of edges of features in the image formed in other parts of the user's field of view outside of the location where the image of the object is formed, compared to the image of the object.

[0539] 50. A system according to Example 48 or 49, wherein the reduced sharpness includes blurring the image formed in other parts of the user's field of view outside the location where the image of the object is formed compared to the image of the object.

[0540] 51. A system according to any of Examples 48 to 50, wherein the head-mounted display system is configured to process images differently than the image of the object by changing the color balance of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0541] 52. A system according to any of Examples 27 to 51, wherein the other portions of the field of view include other portions of the peripheral area of ​​the field of view.

[0542] 53. A system according to any of Examples 27 to 52, wherein the other portion of the field of view includes at least a portion of the central region of the field of view.

[0543] 54. A system according to Example 5, wherein the enhanced image is presented on the near depth plane.

[0544] 55. A system according to Example 5 or 54, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the first far depth plane.

[0545] 56. A system according to Example 18, wherein the magnification is based at least in part on a resolution of the eye.

[0546] 57. A system according to Example 31, wherein the image of the object is presented on the near depth plane.

[0547] 58. A system according to Examples 31 or 57, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the first far depth plane.

[0548] 59. A system according to Example 50, wherein the blurring includes blurring images formed in other portions of the user's field of view using the same color.

[0549] 60. A system according to Example 59, wherein the same color comprises a high contrast color compared to the color in the image of the object.

[0550] 61. A system according to any one of Examples 1 to 26 or any one of Examples 54 to 56, wherein the system is configured to provide an alert to the user to indicate that the presented image has been enhanced.

[0551] 62. A system according to any one of Examples 27 to 53 or any one of Examples 57 to 60, wherein the system is configured to provide an alert to the user to indicate that the presented image has been attenuated.

[0552] 63. A system according to any of Examples 61 or 62, wherein the alarm is a visual alarm.

[0553] 64. A system according to any of Examples 61 or 62, wherein the alarm is an audio alarm.

[0554] 65. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0555] 66. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0556] 67. The system of Example 66, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the eyes of the user.

[0557] 68. A system according to any of Examples 66 to 67, wherein the one or more light sources include a fiber scanning projector.

[0558] 69. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0559] 70. A system according to Example 69, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0560] 71. A system according to Example 70, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0561] 72. The system according to any of the above examples further includes a distance measuring device.

[0562] 73. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0563] 74. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0564] 75. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0565] 76. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0566] 77. A system according to Example 76, wherein the first imaging modality comprises an MRI scan.

[0567] 78. A system according to any of Examples 76 to 77, wherein the second imaging modality includes ultrasound.

[0568] 79. A system according to any of Examples 76 to 78, wherein the first imaging modality comprises an x-ray scan.

[0569] 80. The system according to any of the above examples further comprises an electronic transmitter suitable for generating ultrasonic waves.

[0570] 81. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0571] 82. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[0572] 83. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0573] 84. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0574] 85. A system according to example 84, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0575] 86. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in the portion of the environment in front of the user.

[0576] 87. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0577] 88. A system according to example 87, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0578] 89. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0579] 90. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0580] 91. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0581] 92. A system according to Example 91, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0582] 93. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0583] 94. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[0584] 95. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0585] 96. A system according to example 95, wherein the user's medical parameters include vital signs.

[0586] 97. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0587] 98. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0588] 99. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0589] 100. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0590] 101. A system according to Example 20, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0591] 102. A system according to Example 20, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[0592] 103. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[0593] 104. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0594] 105. A system according to example 104, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0595] 106. A system according to example 104 or 105, wherein the head mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0596] Example Set IIB

[0597] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0598] a frame configured to be supported on a head of the user;

[0599] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user at multiple depth planes, at least a portion of the display comprising one or more waveguides, the one or more waveguides being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user, the central region of the user's eye's field of view corresponding to a central region in the environment in front of the user, and the peripheral region of the user's eye's field of view corresponding to a peripheral region in the environment in front of the user;

[0600] an outward-facing image capture device configured to image the at least a portion of the environment in front of the user;

[0601] one or more sensors configured to measure distances to objects in the at least a portion of the environment in front of the user;

[0602] one or more input devices configured to receive input from the user;

[0603] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0604] wherein the head-mounted display system is configured to select an object in the environment corresponding to the central area of ​​the user's field of view based on input received by the one or more input devices, the one or more sensors are configured to measure the distance to the object after the selection, the outward-facing image capture device is configured to obtain an image of the object, and the display is configured to present an enhanced image of the object at a depth plane determined based on the distance measured by the one or more sensors configured to measure the distance, the enhanced image being enhanced compared to images in other parts of the field of view, and the enhanced image being presented at a location in the central area of ​​the user's field of view.

[0605] 2. The system of Example 1 further comprises one or more light sources configured to direct light into a person's eye to form an image in the eye.

[0606] 3. The system of Example 2, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0607] 4. A system according to Example 2 or 3, wherein the one or more light sources include a fiber scanning projector.

[0608] 5. A system according to Example 1, wherein the depth plane includes a first far depth plane and a second near depth plane, and when the user wears the head-mounted display, the first far depth plane is farther from the user's eyes than the second near depth plane.

[0609] 6. A system according to Example 5, wherein the enhanced image is presented on the far depth plane.

[0610] 7. A system according to Example 5 or 6, wherein the display includes an optical element having an optical focal length so that the light projected into the eye diverges so as to present image content from the second near depth plane.

[0611] 8. A system according to Example 7, wherein the optical element having optical focal length includes a lens.

[0612] 9. A system according to Example 7 or 8, wherein the optical element having optical focal length includes a diffractive optical element.

[0613] 10. A system according to Example 1, wherein the head-mounted display system is configured to present the enhanced image content at a position in the central area of ​​the user's field of view, which corresponds to a position in the central area of ​​the environment where the object is located.

[0614] 11. A system according to Example 1, wherein the head-mounted display system is configured to move the enhanced image content to a position in the central area of ​​the user's field of view that does not correspond to the central area of ​​the environment in which the object is located.

[0615] 12. A system according to Example 1, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0616] 13. A system according to Example 12, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0617] 14. The system of Example 1, wherein one or more sensors comprises a distance measuring device.

[0618] 15. The system of Example 14, wherein the distance measurement system comprises a laser rangefinder.

[0619] 16. A system according to Example 1, wherein the one or more input devices configured to receive input from the user include an inward-facing eye-tracking camera configured to image the user's eyes and track their movements.

[0620] 17. A system according to any of the above examples, wherein presenting the enhanced image includes processing the image of the object differently than other portions of the user's field of view outside the location where the image of the object is formed.

[0621] 18. A system according to Example 17, wherein the differently processing the image includes magnifying the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0622] 19. A system according to any of Examples 17 to 18, wherein the differently processing the image includes increasing the brightness in the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0623] 20. A system according to any of Examples 17 to 19, wherein the differently processing the image includes increasing the contrast of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0624] 21. A system according to any of Examples 17 to 20, wherein the differently processing the image includes increasing the color saturation of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0625] 22. A system according to any of Examples 17 to 21, wherein the differently processing the image includes sharpening the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0626] 23. A system according to Example 22, wherein the sharpening includes edge enhancement of features of the image of the object compared to other portions of the user's field of view outside the location where the image of the object is formed.

[0627] 24. A system according to any of Examples 17 to 23, wherein processing the image differently includes changing the color balance of the image of the object compared to other portions of the user's field of view outside of the location where the image of the object is formed.

[0628] 25. A system according to any of the above examples, wherein the other parts of the field of view include other parts of the central area of ​​the field of view.

[0629] 26. A system according to any of the above examples, wherein the other portion of the field of view includes at least a portion of the peripheral area of ​​the field of view.

[0630] 27. A head mounted display system configured to project light into an eye of a user to display augmented reality image content, the eye of the user having a field of view, the field of view having a central region and a peripheral region disposed around the central region, the head mounted display system comprising:

[0631] a frame configured to be supported on a head of the user;

[0632] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user at multiple depth planes, at least a portion of the display comprising one or more waveguides, the one or more waveguides being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user, the central region of the user's eye's field of view corresponding to a central region in the environment in front of the user, and the peripheral region of the user's eye's field of view corresponding to a peripheral region in the environment in front of the user;

[0633] an outward-facing image capture device configured to image the at least a portion of the environment in front of the user;

[0634] one or more sensors configured to measure distances to objects in the at least a portion of the environment in front of the user;

[0635] one or more input devices configured to receive input from the user;

[0636] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0637] wherein the head-mounted display system is configured to select an object in the environment corresponding to the central area of ​​the user's field of view based on input received by the one or more input devices, the one or more sensors are configured to measure the distance to the object after the selection, the outward-facing image capture device is configured to obtain an image of the object, and the display is configured to present the image of the object at a depth plane determined based on the distance measured by the one or more sensors configured to measure the distance, the image of the object being presented at a location in the central area of ​​the user's field of view, and the display is configured to attenuate images formed in other parts of the field of view compared to the image of the object.

[0638] 28. The system of Example 27 further comprises one or more light sources configured to direct light into a person's eye to form an image in the eye.

[0639] 29. The system of Example 28, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0640] 30. The system of Examples 28 or 29, wherein the one or more light sources comprises a fiber scanning projector.

[0641] 31. A system according to Example 27, wherein the depth plane includes a first far depth plane and a second near depth plane, and when the user wears the head-mounted display, the first far depth plane is farther from the user's eyes than the second near depth plane.

[0642] 32. A system according to Example 31, wherein the image of the object is presented on the far depth plane.

[0643] 33. A system according to Example 31 or 32, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the second near depth plane.

[0644] 34. A system according to Example 33, wherein the optical element having optical focality includes a lens.

[0645] 35. A system according to Example 33 or 34, wherein the optical element having optical focal length includes a diffractive optical element.

[0646] 36. A system according to Example 27, wherein the head-mounted display system is configured to present the image of the object at a position in the central region of the user's field of view, which corresponds to a position in the central region of the environment in which the object is located.

[0647] 37. A system according to Example 27, wherein the head-mounted display system is configured to move the image of the object to a position in the central area of ​​the user's field of view that does not correspond to the central area of ​​the environment in which the object is located.

[0648] 38. A system according to Example 27, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0649] 39. A system according to Example 38, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0650] 40. The system of Example 27, wherein the one or more sensors include a distance measuring device.

[0651] 41. A system according to Example 40, wherein the distance measuring device includes a laser rangefinder.

[0652] 42. A system according to Example 27, wherein the one or more input devices configured to receive input from the user include an inward-facing eye-tracking camera configured to image the user's eyes and track their movements.

[0653] 43. A system according to any of Examples 27 to 42, wherein the head-mounted display system is configured to process images formed in other portions of the user's field of view outside of the location where the image of the object is formed differently than the image of the object.

[0654] 44. A system according to Example 43, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing or reducing the size of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0655] 45. A system according to any of Examples 43 to 44, wherein the head-mounted display system is configured to process images differently than the image of the object by darkening or attenuating images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0656] 46. ​​A system according to any of Examples 43 to 45, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the contrast of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0657] 47. A system according to any of Examples 43 to 46, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the color saturation of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0658] 48. A system according to any of Examples 43 to 47, wherein the head-mounted display system is configured to process images differently than the image of the object by reducing the sharpness of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0659] 49. A system according to Example 48, wherein the reducing sharpness includes removing enhancement of edges of features in the image formed in other parts of the user's field of view outside of the location where the image of the object is formed, compared to the image of the object.

[0660] 50. A system according to Example 48 or 49, wherein the reduced sharpness includes blurring the image formed in other parts of the user's field of view outside the location where the image of the object is formed compared to the image of the object.

[0661] 51. A system according to any of Examples 48 to 50, wherein the head-mounted display system is configured to process images differently than the image of the object by changing the color balance of images formed in other portions of the user's field of view outside of the location where the image of the object is formed.

[0662] 52. A system according to any of Examples 27 to 51, wherein the other portions of the field of view include other portions of the central region of the field of view.

[0663] 53. A system according to any of Examples 27 to 52, wherein the other portion of the field of view includes at least a portion of the peripheral region of the field of view.

[0664] 54. A system according to Example 5, wherein the enhanced image is presented on the near depth plane.

[0665] 55. A system according to Example 5 or 54, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the first far depth plane.

[0666] 56. A system according to Example 18, wherein the magnification is based at least in part on a resolution of the eye.

[0667] 57. A system according to Example 31, wherein the image of the object is presented on the near depth plane.

[0668] 58. A system according to Examples 31 or 57, wherein the display includes an optical element having an optical focal length such that the light projected into the eye diverges so as to present image content from the first far depth plane.

[0669] 59. A system according to Example 50, wherein the blurring includes blurring the image formed in other portions of the user's field of view using the same color.

[0670] 60. A system according to Example 59, wherein the same color comprises a high contrast color compared to the color in the image of the object.

[0671] 61. A system according to any one of Examples 1 to 26 or any one of Examples 54 to 56, wherein the system is configured to provide an alert to the user to indicate that the presented image has been enhanced.

[0672] 62. A system according to any one of Examples 27 to 53 or any one of Examples 57 to 60, wherein the system is configured to provide an alert to the user to indicate that the presented image has been attenuated.

[0673] 63. A system according to any of Examples 61 or 62, wherein the alarm is a visual alarm.

[0674] 64. A system according to any of Examples 61 or 62, wherein the alarm is an audio alarm.

[0675] 65. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0676] 66. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0677] 67. The system of Example 66, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the eyes of the user.

[0678] 68. A system according to any of Examples 66 to 67, wherein the one or more light sources include a fiber scanning projector.

[0679] 69. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0680] 70. A system according to Example 69, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0681] 71. A system according to Example 70, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0682] 72. The system according to any of the above examples further includes a distance measuring device.

[0683] 73. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0684] 74. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0685] 75. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0686] 76. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0687] 77. A system according to Example 76, wherein the first imaging modality comprises an MRI scan.

[0688] 78. A system according to any of Examples 76 to 77, wherein the second imaging modality includes ultrasound.

[0689] 79. A system according to any of Examples 76 to 78, wherein the first imaging modality comprises an x-ray scan.

[0690] 80. The system according to any of the above examples further comprises an electronic transmitter suitable for generating ultrasonic waves.

[0691] 81. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0692] 82. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[0693] 83. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0694] 84. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0695] 85. A system according to example 84, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0696] 86. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the location of objects in the portion of the environment in front of the user.

[0697] 87. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0698] 88. A system according to example 87, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0699] 89. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0700] 90. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0701] 91. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0702] 92. A system according to Example 91, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0703] 93. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0704] 94. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[0705] 95. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0706] 96. A system according to example 95, wherein the user's medical parameters include vital signs.

[0707] 97. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0708] 98. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0709] 99. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0710] 100. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0711] 101. A system according to Example 20, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0712] 102. A system according to Example 20, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[0713] 103. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[0714] 104. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0715] 105. A system according to example 104, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0716] 106. A system according to example 104 or 105, wherein the head mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0717] Sample Set III

[0718] A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0719] a frame configured to be supported on a head of the user;

[0720] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user at multiple depth planes, at least a portion of the display comprising one or more waveguides, the one or more waveguides being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of an environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user, the central region of the user's eye's field of view corresponding to a central region in the environment in front of the user, and the peripheral region of the user's eye's field of view corresponding to a peripheral region in the environment in front of the user;

[0721] an outward-facing image capture device configured to image the at least a portion of the environment in front of the user;

[0722] one or more input devices configured to receive input from the user;

[0723] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0724] wherein the depth plane includes a first depth plane and a second depth plane, and when the user wears the head mounted display, the distance to the object corresponds more to the first depth plane than to the second depth plane,

[0725] wherein the head-mounted display system is configured to select an object in the environment within the user's field of view based on input received by the one or more input devices, the outward-facing image capture device is configured to obtain an image of the object, and the display is configured to present the image of the object at the second depth plane.

[0726] 2. A system according to Example 1, wherein the one or more input devices configured to receive input from the user include an inward-facing eye-tracking camera configured to image the user's eyes and track their movements.

[0727] 3. A system according to any of the above examples, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0728] 4. The system of Example 3, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0729] 5. The system according to any of the above examples further includes one or more sensors configured to measure the distance to objects in at least the portion of the environment in front of the user.

[0730] 6. A system according to Example 5, wherein the one or more sensors are configured to measure the distance to the object after the selection of the object.

[0731] 7. A system according to any of Examples 5 to 6, wherein one or more sensors include a distance measuring device.

[0732] 8. The system of Example 7, wherein the distance measuring device comprises a laser rangefinder.

[0733] 9. A system according to any of the above examples, wherein the first depth plane includes a far depth plane, the second depth plane includes a near depth plane, and when the user wears the head-mounted display, the far depth plane is farther from the user's eyes than the near depth plane.

[0734] 10. A system according to any one of Examples 1 to 8, wherein the first depth plane includes a near depth plane, and the second depth plane includes a far depth plane, and when the user wears the head-mounted display, the far depth plane is farther from the user's eyes than the near depth plane.

[0735] 11. A system according to any of the above examples, wherein the display is configured to present additional image content at the second depth plane.

[0736] 12. The system of example 11, wherein the additional image is a selected object.

[0737] 13. A system according to any of the above examples, wherein the image of the object presented at the second depth plane is magnified.

[0738] 14. A system according to any of the above examples, wherein the image of the object presented at the second depth plane is not magnified.

[0739] 15. A system according to Example 15, wherein the size of the image of the object presented at the second depth plane is reduced.

[0740] 16. A system according to any of the above examples, wherein the one or more input devices configured to receive input from the user include a head posture sensor.

[0741] 17. The system of Example 16, wherein the head posture sensor comprises an accelerometer or an IMU.

[0742] 18. A system according to any of the above examples, wherein the head-mounted display system is configured to present the image of the object in the central area.

[0743] 19. A system according to any of the above examples, wherein the head-mounted display system is configured to present the image of the object in the peripheral area.

[0744] 20. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0745] 21. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0746] 22. The system of Example 21, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the user's eyes.

[0747] 23. A system according to any of Examples 21 to 22, wherein the one or more light sources include a fiber scanning projector.

[0748] 24. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0749] 25. A system according to Example 24, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0750] 26. A system according to Example 25, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0751] 27. The system according to any of the above examples further includes a distance measuring device.

[0752] 28. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0753] 29. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0754] 30. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0755] 31. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0756] 32. A system according to Example 31, wherein the first image modality comprises an MRI scan.

[0757] 33. A system according to any of Examples 31 to 32, wherein the second imaging modality includes ultrasound.

[0758] 34. A system according to any of Examples 31 to 33, wherein the first imaging modality comprises an x-ray scan.

[0759] 35. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[0760] 36. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0761] 37. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on a portion of the environment in front of the user to the user's eyes.

[0762] 38. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to a real-world object in the environment.

[0763] 39. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or to give access to the area to be cut.

[0764] 40. A system according to Example 39, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0765] 41. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the orientation of objects in the portion of the environment in front of the user.

[0766] 42. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0767] 43. A system according to Example 42, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0768] 44. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0769] 45. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0770] 46. ​​A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0771] 47. A system according to Example 46, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0772] 48. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0773] 49. A system according to any of the above examples, wherein the head-mounted display is configured to alert the user to steps in the medical procedure.

[0774] 50. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0775] 51. A system according to Example 50, wherein the user's medical parameters include vital signs.

[0776] 52. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0777] 53. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0778] 54. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0779] 55. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0780] 56. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0781] 57. A system according to Example 56, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0782] 58. A system according to Example 56 or 57, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0783] Example Set IV

[0784] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0785] a frame configured to be supported on a head of the user;

[0786] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0787] one or more capture devices configured to capture lighting conditions of the environment;

[0788] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0789] The head mounted display system is configured to present image content to the user's field of view that is enhanced based at least in part on lighting conditions of the environment.

[0790] 2. The system of Example 1 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0791] 3. A system according to Example 1 or 2, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0792] 4. The system of Example 3, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0793] 5. A system according to any one of Examples 2 to 4, wherein the light source comprises a fiber scanning projector.

[0794] 6. A system according to any of Examples 1 to 5, wherein the one or more capture devices include one or more image capture devices.

[0795] 7. The system of Example 6, wherein the one or more image capture devices include one or more cameras.

[0796] 8. A system according to any of Examples 1 to 5, wherein the one or more capture devices include one or more light sensors.

[0797] 9. The system of Example 8, wherein the one or more light sensors include one or more illuminance meters.

[0798] 10. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminance of the environment.

[0799] 11. The system according to any of the above examples further includes a distance measuring device.

[0800] 12. The system of Example 11, wherein the distance measuring device comprises a laser rangefinder.

[0801] 13. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0802] 14. The system of any one of Examples 1 to 13, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[0803] 15. A system according to any of the above examples, wherein the head-mounted display system is configured to magnify the image content based at least in part on lighting conditions of the environment.

[0804] 16. A system according to Example 15, wherein the magnification is based at least in part on a resolution of the eye.

[0805] 17. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the brightness in the image content based at least in part on lighting conditions of the environment.

[0806] 18. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the contrast in the image content based at least in part on lighting conditions of the environment.

[0807] 19. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the color saturation in the image content based at least in part on the lighting conditions of the environment.

[0808] 20. A system according to any of the above examples, wherein the head-mounted display system is configured to sharpen the image content based at least in part on lighting conditions of the environment.

[0809] 21. A system according to Example 20, wherein sharpening includes adding edge enhancement features in the image content based at least in part on lighting conditions of the environment.

[0810] 22. A system according to any of the above examples, wherein the head-mounted display system is configured to change the color balance of the image content based at least in part on the lighting conditions of the environment.

[0811] 23. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been enhanced.

[0812] 24. The system of Example 23, wherein the alarm is a visual alarm.

[0813] 25. The system of Example 23, wherein the alarm is an audio alarm.

[0814] 26. A head mounted display system configured to project light into an eye of a user to display augmented reality image content, the eye of the user having a field of view, the field of view having a central region and a peripheral region disposed around the central region, the head mounted display system comprising:

[0815] a frame configured to be supported on a head of the user;

[0816] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0817] one or more capture devices configured to capture lighting conditions of the environment;

[0818] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0819] The head mounted display system is configured to present image content to the user's field of view that is attenuated based at least in part on lighting conditions of the environment.

[0820] 27. The system of Example 26 further comprises one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0821] 28. A system according to Examples 26 or 27, wherein at least the portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0822] 29. The system of Example 28, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0823] 30. A system according to any of Examples 27 to 29, wherein the light source comprises a fiber scanning projector.

[0824] 31. A system according to any of Examples 26 to 30, wherein the one or more capture devices include one or more image capture devices.

[0825] 32. A system according to example 31, wherein the one or more image capture devices include one or more cameras.

[0826] 33. A system according to any of Examples 26 to 30, wherein the one or more capture devices include one or more light sensors.

[0827] 34. The system of Example 22, wherein the one or more light sensors include one or more illuminance meters.

[0828] 35. A system according to any of Examples 26 to 34, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[0829] 36. The system of any one of Examples 26 to 35 further comprises a distance measuring device.

[0830] 37. A system according to Example 36, wherein the distance measuring device includes a laser rangefinder.

[0831] 38. The system of any one of Examples 26 to 37 further comprises an eye tracking device configured to track the position and / or movement of the user's eyes.

[0832] 39. The system of any of Examples 26 to 38, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[0833] 40. A system according to any of Examples 26 to 39, wherein the head-mounted display is configured to reduce the size of the image content based at least in part on lighting conditions of the environment.

[0834] 41. A system according to Example 40, wherein the reduction in size is based at least in part on a resolution of the eye.

[0835] 42. A system according to any of Examples 26 to 41, wherein the head-mounted display system is configured to dim or attenuate the image content based at least in part on lighting conditions of the environment.

[0836] 43. A system according to any of Examples 26 to 42, wherein the head-mounted display system is configured to reduce contrast in the image content based at least in part on lighting conditions of the environment.

[0837] 44. A system according to any of Examples 26 to 43, wherein the head-mounted display system is configured to reduce color saturation in the image content based at least in part on lighting conditions of the environment.

[0838] 45. A system according to any of Examples 26 to 44, wherein the head-mounted display system is configured to reduce the sharpness of the image content based at least in part on lighting conditions of the environment.

[0839] 46. ​​A system according to example 45, wherein reducing sharpness includes removing enhancement of edges of features in the image content based at least in part on lighting conditions of the environment.

[0840] 47. A system according to any of Examples 26 to 46, wherein the head-mounted display is configured to change the color balance of the image content based at least in part on lighting conditions of the environment.

[0841] 48. A system according to any of Examples 26 to 47, wherein the head-mounted display is configured to blur the image content based at least in part on lighting conditions of the environment.

[0842] 49. A system according to Example 48, wherein the blurring includes blurring the image content using the same color.

[0843] 50. A system according to any of Examples 26 to 49, wherein the system is configured to provide an alert to the user to indicate that the image content has been impaired.

[0844] 51. A system according to Example 50, wherein the alarm is a visual alarm.

[0845] 52. The system of Example 52, wherein the alarm is an audio alarm.

[0846] 53. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, wherein the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0847] 54. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[0848] 55. The system of Example 54, comprising one or more light sources configured to direct light into the one or more waveguides, the waveguides being configured to direct light into the user's eyes.

[0849] 56. A system according to any of Examples 54 to 55, wherein the one or more light sources include a fiber scanning projector.

[0850] 57. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0851] 58. A system according to Example 57, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0852] 59. A system according to Example 58, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0853] 60. The system according to any of the above examples further includes a distance measuring device.

[0854] 61. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0855] 62. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0856] 63. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0857] 64. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0858] 65. A system according to Example 64, wherein the first image modality comprises an MRI scan.

[0859] 66. A system according to any of Examples 64 to 65, wherein the second imaging modality includes ultrasound.

[0860] 67. A system according to any of Examples 64 to 66, wherein the first imaging modality comprises an x-ray scan.

[0861] 68. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[0862] 69. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[0863] 70. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on a portion of the environment in front of the user to the user's eyes.

[0864] 71. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0865] 72. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0866] 73. A system according to example 72, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0867] 74. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the location of objects in the portion of the environment in front of the user.

[0868] 75. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0869] 76. A system according to Example 75, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0870] 77. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0871] 78. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0872] 79. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0873] 80. A system according to Example 79, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0874] 81. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0875] 82. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[0876] 83. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[0877] 84. A system according to example 83, wherein the user's medical parameters include vital signs.

[0878] 85. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0879] 86. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0880] 87. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0881] 88. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[0882] 89. A system according to Example 18, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0883] 90. The system of Example 18, wherein increasing contrast comprises adding black, gray, white, or other color to at least one color of the image content.

[0884] 91. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[0885] 92. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0886] 93. A system according to Example 92, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0887] 94. A system according to Example 92 or 93, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0888] Example Set IVA

[0889] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0890] a frame configured to be supported on a head of the user;

[0891] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0892] one or more capture devices configured to capture lighting conditions of the environment;

[0893] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0894] The head-mounted display system is configured to project light to the position of the user's eyes so as to present image content enhanced at least in part based on the lighting conditions of the environment to a portion of the central area of ​​the user's field of view.

[0895] 2. A system according to Example 1, wherein under photopic lighting conditions, the image content is inversely enhanced based on the cone density of the projected light position.

[0896] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[0897] 4. The system of Example 1, wherein under scotopic lighting conditions, the image content is inversely enhanced based on rod density at the projected light location.

[0898] 5. The system of example 4, wherein the scotopic lighting conditions of the environment have a -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[0899] 6. A system according to Example 1, wherein under mesopic lighting conditions, the image content is enhanced based at least in part on time spent under the mesopic lighting conditions.

[0900] 7. The system of example 6, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2 luminosity.

[0901] 8. A system according to Example 6 or 7, wherein the system is configured to determine whether cones or rods are dominant in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[0902] 9. A system according to any of Examples 6 to 8, wherein when the cones are dominant in the user's eye, the image content is inversely enhanced based on the cone density of the projected light position.

[0903] 10. A system according to any of Examples 6 to 8, wherein when the rods are dominant in the user's eye, the image content is inversely enhanced based on the rod density of the projected light position of the user's eye.

[0904] 11. A system according to any of the above examples, wherein the system includes a timer or clock to monitor the time spent under the lighting conditions.

[0905] 12. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0906] 13. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project the light to the user.

[0907] 14. The system of Example 13, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0908] 15. A system according to any of Examples 12 to 14, wherein the light source comprises a fiber scanning projector.

[0909] 16. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more image capture devices.

[0910] 17. A system according to Example 16, wherein the one or more image capture devices include one or more cameras.

[0911] 18. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more light sensors.

[0912] 19. The system of Example 18, wherein the one or more light sensors include one or more illuminance meters.

[0913] 20. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[0914] 21. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate light levels at the pixels to capture the lighting conditions.

[0915] 22. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[0916] 23. The system according to any of the above examples further includes a distance measuring device.

[0917] 24. The system of Example 23, wherein the distance measuring device comprises a laser rangefinder.

[0918] 25. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[0919] 26. The system of any one of Examples 1 to 25, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[0920] 27. A system according to any of the above examples, wherein the head-mounted display system is configured to magnify the image content based at least in part on lighting conditions of the environment.

[0921] 28. A system according to Example 27, wherein the magnification is based at least in part on a resolution of the eye.

[0922] 29. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the brightness in the image content based at least in part on lighting conditions of the environment.

[0923] 30. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the contrast in the image content based at least in part on lighting conditions of the environment.

[0924] 31. A system according to Example 30, wherein the head-mounted display system is configured to increase contrast based at least in part on the contrast sensitivity of the eye.

[0925] 32. A system according to any of the above examples, wherein the head-mounted display system is configured to increase color saturation in the image content based at least in part on lighting conditions of the environment.

[0926] 33. A system according to any of the above examples, wherein the head-mounted display system is configured to sharpen the image content based at least in part on lighting conditions of the environment.

[0927] 34. A system according to example 33, wherein sharpening includes adding edge enhancement features in the image content based at least in part on lighting conditions of the environment.

[0928] 35. A system according to any of the above examples, wherein the head-mounted display system is configured to change the color balance of the image content based at least in part on the lighting conditions of the environment.

[0929] 36. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been enhanced.

[0930] 37. A system according to Example 36, wherein the alarm is a visual alarm.

[0931] 38. A system according to Example 36, wherein the alarm is an audio alarm.

[0932] 39. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, wherein the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[0933] 40. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[0934] 41. A system according to Example 40, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[0935] 42. A system according to Example 41, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[0936] 43. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[0937] 44. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[0938] 45. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[0939] 46. ​​A system according to Example 45, wherein the first image modality comprises an MRI scan.

[0940] 47. A system according to any of Examples 45 to 46, wherein the second imaging modality includes ultrasound.

[0941] 48. A system according to any of Examples 45 to 47, wherein the first imaging modality comprises an x-ray scan.

[0942] 49. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[0943] 50. The system according to any of the above examples further comprises a sensor suitable for converting ultrasonic waves into electrical signals.

[0944] 51. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[0945] 52. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[0946] 53. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[0947] 54. A system according to Example 53, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[0948] 55. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in the portion of the environment in front of the user.

[0949] 56. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[0950] 57. A system according to Example 56, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[0951] 58. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[0952] 59. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[0953] 60. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[0954] 61. A system according to Example 60, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[0955] 62. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[0956] 63. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[0957] 64. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[0958] 65. A system according to example 64, wherein the medical parameters include vital signs.

[0959] 66. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[0960] 67. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[0961] 68. A system according to Example 21, wherein the detector array includes an outward-looking camera configured to image the environment.

[0962] 69. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into the position of the user's eyes so as to present image content that is enhanced to a portion of the central area of ​​the user's field of view relative to image content presented to a portion of the peripheral area.

[0963] 70. A system according to any of the above examples, wherein the head-mounted display system is configured to project light to the position of the user's eyes so as to present image content that is enhanced relative to image content presented to a portion of the central area of ​​the user's field of view to another portion of the central area.

[0964] 71. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[0965] 72. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[0966] 73. A system according to Example 30, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[0967] 74. A system according to Example 30, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[0968] 75. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[0969] 76. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[0970] 77. A system according to Example 76, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[0971] 78. A system according to Example 76 or 77, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[0972] Example Set IVB

[0973] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[0974] a frame configured to be supported on a head of the user;

[0975] a display disposed on the frame, the display configured to project light into the user's eyes to present image content at the peripheral region of the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[0976] one or more capture devices configured to capture lighting conditions of the environment;

[0977] a processing electronic device in communication with the display to control the presentation of image content on the display,

[0978] The head-mounted display system is configured to project light to the position of the user's eyes so as to present image content enhanced at least in part based on the lighting conditions of the environment to a portion of the peripheral area of ​​the user's field of view.

[0979] 2. The system of Example 1, wherein under photopic lighting conditions, the image content is inversely enhanced based on cone density at the projected light location.

[0980] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[0981] 4. The system of Example 1, wherein under scotopic lighting conditions, the image content is inversely enhanced based on rod density at the projected light location.

[0982] 5. The system of example 4, wherein the scotopic lighting conditions of the environment have a -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[0983] 6. A system according to Example 1, wherein under interocular lighting conditions, the image content is enhanced based at least in part on time spent under the interocular lighting conditions.

[0984] 7. The system of example 6, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2luminosity.

[0985] 8. A system according to Example 6 or 7, wherein the system is configured to determine whether cones or rods are dominant in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[0986] 9. A system according to any of Examples 6 to 8, wherein when the cones are dominant in the user's eye, the image content is inversely enhanced based on the cone density of the projected light position.

[0987] 10. A system according to any of Examples 6 to 8, wherein when the rods are dominant in the user's eye, the image content is inversely enhanced based on the rod density of the projected light position of the user's eye.

[0988] 11. A system according to any of the above examples, wherein the system includes a timer or clock to monitor the time spent under the lighting conditions.

[0989] 12. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[0990] 13. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project light to the user.

[0991] 14. The system of Example 13, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[0992] 15. A system according to any of Examples 12 to 14, wherein the light source comprises a fiber scanning projector.

[0993] 16. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more image capture devices.

[0994] 17. A system according to Example 16, wherein the one or more image capture devices include one or more cameras.

[0995] 18. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more light sensors.

[0996] 19. The system of Example 18, wherein the one or more light sensors include one or more illuminance meters.

[0997] 20. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[0998] 21. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate the light level at the pixels to capture the lighting conditions.

[0999] 22. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[1000] 23. The system according to any of the above examples further includes a distance measuring device.

[1001] 24. The system of Example 23, wherein the distance measuring device comprises a laser rangefinder.

[1002] 25. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1003] 26. The system of any one of Examples 1 to 25, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[1004] 27. A system according to any of the above examples, wherein the head-mounted display system is configured to magnify the image content based at least in part on lighting conditions of the environment.

[1005] 28. A system according to Example 27, wherein the magnification is based at least in part on a resolution of the eye.

[1006] 29. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the brightness in the image content based at least in part on lighting conditions of the environment.

[1007] 30. A system according to any of the above examples, wherein the head-mounted display system is configured to increase the contrast in the image content based at least in part on lighting conditions of the environment.

[1008] 31. A system according to Example 30, wherein the head-mounted display system is configured to increase contrast based at least in part on the contrast sensitivity of the eye.

[1009] 32. A system according to any of the above examples, wherein the head-mounted display system is configured to increase color saturation in the image content based at least in part on lighting conditions of the environment.

[1010] 33. A system according to any of the above examples, wherein the head-mounted display system is configured to sharpen the image content based at least in part on lighting conditions of the environment.

[1011] 34. A system according to example 33, wherein sharpening includes adding edge enhancement features in the image content based at least in part on lighting conditions of the environment.

[1012] 35. A system according to any of the above examples, wherein the head-mounted display system is configured to change the color balance of the image content based at least in part on the lighting conditions of the environment.

[1013] 36. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been enhanced.

[1014] 37. A system according to Example 36, wherein the alarm is a visual alarm.

[1015] 38. A system according to Example 36, wherein the alarm is an audio alarm.

[1016] 39. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, wherein the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1017] 40. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1018] 41. A system according to Example 40, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1019] 42. A system according to Example 41, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1020] 43. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1021] 44. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1022] 45. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1023] 46. ​​A system according to Example 45, wherein the first image modality comprises an MRI scan.

[1024] 47. A system according to any of Examples 45 to 46, wherein the second imaging modality includes ultrasound.

[1025] 48. A system according to any of Examples 45 to 47, wherein the first imaging modality comprises an x-ray scan.

[1026] 49. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[1027] 50. The system according to any of the above examples further comprises a sensor suitable for converting ultrasonic waves into electrical signals.

[1028] 51. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on a portion of the environment in front of the user to the user's eyes.

[1029] 52. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1030] 53. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1031] 54. A system according to Example 53, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1032] 55. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in a portion of the environment in front of the user.

[1033] 56. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the location of an object in a portion of the environment in front of the user.

[1034] 57. A system according to Example 56, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1035] 58. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[1036] 59. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1037] 60. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1038] 61. A system according to Example 60, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1039] 62. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1040] 63. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[1041] 64. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[1042] 65. A system according to example 64, wherein the medical parameters include vital signs.

[1043] 66. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1044] 67. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1045] 68. A system according to Example 21, wherein the detector array includes an outward-looking camera configured to image the environment.

[1046] 69. A system according to any of the above examples, wherein the head-mounted display system is configured to project light to the position of the user's eyes so as to present image content that is enhanced relative to image content presented to a portion of the central area to a portion of the user's field of view.

[1047] 70. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into the position of the user's eyes so as to present image content that is enhanced relative to image content presented to a portion of the peripheral area to another portion of the user's field of view.

[1048] 71. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[1049] 72. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[1050] 73. A system according to Example 30, wherein increasing contrast comprises adjusting the brightness or darkness of at least one color of the image content.

[1051] 74. A system according to Example 30, wherein increasing contrast includes adding black, gray, white, or other color to at least one color of the image content.

[1052] 75. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[1053] 76. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1054] 77. A system according to Example 76, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1055] 78. A system according to Example 76 or 77, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[1056] Example Set IVC

[1057] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[1058] a frame configured to be supported on a head of the user;

[1059] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[1060] one or more capture devices configured to capture lighting conditions of the environment;

[1061] a processing electronic device in communication with the display to control the presentation of image content on the display,

[1062] The head-mounted display system is configured to present image content from a first portion of the user's field of view to a second portion of the user's field of view based at least in part on lighting conditions of the environment, and wherein under the lighting conditions of the environment, the second portion corresponds to a position of the user's eye having a higher visual acuity than the first portion.

[1063] 2. A system according to Example 1, wherein under photopic lighting conditions, the second portion corresponds to a location of the user's eye having a higher cone cell density than the first portion.

[1064] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[1065] 4. A system according to Example 2 or 3, wherein the first portion includes a portion of the peripheral region and the second portion includes a portion of the central region.

[1066] 5. A system according to Example 2 or 3, wherein the first portion includes a portion of the central region and the second portion includes another portion of the central region.

[1067] 6. A system according to Example 2 or 3, wherein the first portion includes a portion of the peripheral area and the second portion includes another portion of the peripheral area.

[1068] 7. A system according to Example 1, wherein under scotopic lighting conditions, the second portion corresponds to a location of the user's eye having a higher rod cell density than the first portion.

[1069] 8. The system of example 7, wherein the scotopic lighting conditions of the environment have a -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[1070] 9. A system according to Example 7 or 8, wherein the first portion includes a portion of the central area and the second portion includes an area of ​​the peripheral area.

[1071] 10. A system according to Example 7 or 8, wherein the first portion includes a portion of the peripheral area and the second portion includes another area of ​​the peripheral area.

[1072] 11. A system according to Example 1, wherein under interocular lighting conditions, the system is configured to present image content from the first portion to the second portion based at least in part on time spent under the interocular lighting conditions.

[1073] 12. The system of example 11, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2 luminosity.

[1074] 13. A system according to Example 11 or 12, wherein the system is configured to determine whether cones or rods dominate in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[1075] 14. A system according to any of Examples 11 to 13, wherein when the cones are dominant in the user's eye, the second portion corresponds to a location of the user's eye having a higher cone density than the first portion.

[1076] 15. A system according to Example 14, wherein the first portion includes a portion of the peripheral region and the second portion includes a portion of the central region.

[1077] 16. A system according to Example 14, wherein the first portion includes a portion of the central region and the second portion includes another portion of the central region.

[1078] 17. A system according to Example 14, wherein the first portion includes a portion of the peripheral area and the second portion includes another portion of the peripheral area.

[1079] 18. A system according to any of Examples 11 to 13, wherein when the rod cells are dominant in the user's eye, the second portion corresponds to a location of the user's eye having a higher rod cell density than the first portion.

[1080] 19. A system according to Example 18, wherein the first portion includes a portion of the central region and the second portion includes a portion of the peripheral region.

[1081] 20. A system according to Example 18, wherein the first portion includes a portion of the peripheral area and the second portion includes another portion of the peripheral area.

[1082] 21. A system according to any of the above examples, wherein the system includes a timer or clock to monitor time spent under the lighting conditions.

[1083] 22. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[1084] 23. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project the light to the user.

[1085] 24. The system of Example 23, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[1086] 25. A system according to any of Examples 22 to 24, wherein the light source comprises a fiber scanning projector.

[1087] 26. A system according to any of Examples 1 to 25, wherein the one or more capture devices include one or more image capture devices.

[1088] 27. A system according to Example 26, wherein the one or more image capture devices include one or more cameras.

[1089] 28. A system according to any of Examples 1 to 25, wherein the one or more capture devices include one or more light sensors.

[1090] 29. The system of Example 28, wherein the one or more light sensors include one or more illuminance meters.

[1091] 30. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[1092] 31. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate the light level at the pixels to capture the lighting conditions.

[1093] 32. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[1094] 33. The system according to any of the above examples further includes a distance measuring device.

[1095] 34. A system according to Example 33, wherein the distance measuring device includes a laser rangefinder.

[1096] 35. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1097] 36. The system of any one of Examples 1 to 35, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[1098] 37. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has shifted from the first portion to the second portion.

[1099] 38. A system according to Example 37, wherein the alarm is a visual alarm.

[1100] 39. The system of Example 37, wherein the alarm is an audio alarm.

[1101] 40. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1102] 41. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1103] 42. A system according to example 41, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1104] 43. A system according to Example 42, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1105] 44. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1106] 45. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1107] 46. ​​A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1108] 47. A system according to Example 46, wherein the first image modality comprises an MRI scan.

[1109] 48. A system according to any of Examples 46 to 47, wherein the second imaging modality includes ultrasound.

[1110] 49. A system according to any of Examples 46 to 48, wherein the first imaging modality comprises an x-ray scan.

[1111] 50. The system according to any of the above examples further comprises an electronic transmitter suitable for generating ultrasonic waves.

[1112] 51. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[1113] 52. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[1114] 53. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1115] 54. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1116] 55. A system according to Example 54, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1117] 56. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the orientation of objects in the portion of the environment in front of the user.

[1118] 57. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[1119] 58. A system according to Example 57, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1120] 59. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[1121] 60. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1122] 61. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1123] 62. A system according to Example 61, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1124] 63. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1125] 64. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[1126] 65. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[1127] 66. A system according to example 65, wherein the medical parameters include vital signs.

[1128] 67. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1129] 68. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1130] 69. The system of Example 31, wherein the detector array comprises an outward-looking camera configured to image the environment.

[1131] 70. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[1132] 71. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[1133] 72. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1134] 73. A system according to Example 72, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1135] 74. A system according to Example 72 or 73, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[1136] Example Set IVD

[1137] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[1138] a frame configured to be supported on a head of the user;

[1139] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content to the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[1140] one or more capture devices configured to capture lighting conditions of the environment;

[1141] a processing electronic device in communication with the display to control the presentation of image content on the display,

[1142] The head mounted display system is configured to project light to the position of the user's eyes so as to present image content to a portion of the user's field of view based at least in part on lighting conditions of the environment.

[1143] 2. The system of Example 1, wherein under photopic lighting conditions, the projected light position is based on the cone density at the projected light position.

[1144] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[1145] 4. A system according to any of Examples 2 to 3, wherein the portion of the user's field of view includes the central area.

[1146] 5. A system according to any of Examples 2 to 4, wherein the projected light position is within a range of 0 to 5 degrees from the fovea.

[1147] 6. A system according to Example 1, wherein under scotopic lighting conditions, the projected light position is based on the rod cell density at the projected light position.

[1148] 7. The system of embodiment 6, wherein the dark lighting condition of the environment has 10 -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[1149] 8. A system according to any of Examples 6 to 7, wherein the portion of the user's field of view includes the peripheral area.

[1150] 9. A system according to any of Examples 6 to 8, wherein the projected light position is within a range of 15 to 20 degrees from the fovea.

[1151] 10. A system according to any of Examples 6 to 8, wherein the projected light position is within a range of 25 to 35 degrees from the fovea.

[1152] 11. A system according to Example 1, wherein under interocular lighting conditions, the projected light position is based at least in part on time spent under the interocular lighting conditions.

[1153] 12. The system of example 11, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2 luminosity.

[1154] 13. A system according to Example 11 or 12, wherein the system is configured to determine whether cones or rods dominate in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[1155] 14. A system according to any of Examples 11 to 13, wherein when the cones are dominant in the user's eye, the projected light position is based on the cone density at the projected light position.

[1156] 15. A system according to Example 14, wherein the portion of the user's field of view includes the central area.

[1157] 16. A system according to Example 14, wherein the projected light position is within a range of 0 to 5 degrees from the fovea.

[1158] 17. A system according to any of Examples 11 to 13, wherein when the rods are dominant in the user's eye, the projected light position is based on the rod density of the projected light position.

[1159] 18. A system according to Example 17, wherein the portion of the user's field of view includes the peripheral area.

[1160] 19. A system according to Example 17, wherein the projected light position is within a range of 15 to 20 degrees from the fovea.

[1161] 20. A system according to Example 17, wherein the projected light position is within a range of 25 to 35 degrees from the fovea.

[1162] 21. A system according to any of the above examples, wherein the system includes a timer or clock to monitor time spent under the lighting conditions.

[1163] 22. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[1164] 23. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project the light to the user.

[1165] 24. The system of Example 23, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[1166] 25. A system according to any of Examples 22 to 24, wherein the light source comprises a fiber scanning projector.

[1167] 26. A system according to any of Examples 1 to 25, wherein the one or more capture devices include one or more image capture devices.

[1168] 27. A system according to Example 26, wherein the one or more image capture devices include one or more cameras.

[1169] 28. A system according to any of Examples 1 to 25, wherein the one or more capture devices include one or more light sensors.

[1170] 29. The system of Example 28, wherein the one or more light sensors include one or more illuminance meters.

[1171] 30. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[1172] 31. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate the light level at the pixels to capture the lighting conditions.

[1173] 32. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[1174] 33. The system according to any of the above examples further includes a distance measuring device.

[1175] 34. A system according to Example 33, wherein the distance measuring device includes a laser rangefinder.

[1176] 35. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1177] 36. The system of any one of Examples 1 to 35, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[1178] 37. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been presented.

[1179] 38. A system according to Example 37, wherein the alarm is a visual alarm.

[1180] 39. The system of Example 37, wherein the alarm is an audio alarm.

[1181] 40. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1182] 41. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1183] 42. A system according to example 41, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1184] 43. A system according to Example 42, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1185] 44. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1186] 45. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1187] 46. ​​A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1188] 47. A system according to Example 46, wherein the first image modality comprises an MRI scan.

[1189] 48. A system according to any of Examples 46 to 47, wherein the second imaging modality includes ultrasound.

[1190] 49. A system according to any of Examples 46 to 48, wherein the first imaging modality comprises an x-ray scan.

[1191] 50. The system according to any of the above examples further comprises an electronic transmitter suitable for generating ultrasonic waves.

[1192] 51. The system according to any of the above examples further includes a sensor suitable for converting ultrasonic waves into electrical signals.

[1193] 52. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on a portion of the environment in front of the user to the user's eyes.

[1194] 53. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1195] 54. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1196] 55. A system according to Example 54, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1197] 56. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the orientation of objects in a portion of the environment in front of the user.

[1198] 57. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in a portion of the environment in front of the user.

[1199] 58. A system according to Example 57, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1200] 59. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[1201] 60. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1202] 61. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1203] 62. A system according to Example 61, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1204] 63. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1205] 64. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[1206] 65. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[1207] 66. A system according to example 65, wherein the medical parameters include vital signs.

[1208] 67. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1209] 68. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1210] 69. The system of Example 31, wherein the detector array comprises an outward-looking camera configured to image the environment.

[1211] 70. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[1212] 71. A system according to any of the above examples, wherein the head-mounted display system is configured to project light to the location of the user's eyes so as to present image content to a portion of the user's field of view based at least in part on the density of photoreceptors at the location of the projected light.

[1213] 72. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[1214] 73. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1215] 74. A system according to Example 73, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1216] 75. A system according to Example 73 or 74, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[1217] Sample Set IVE

[1218] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[1219] a frame configured to be supported on a head of the user;

[1220] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the central area of ​​the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[1221] one or more capture devices configured to capture lighting conditions of the environment;

[1222] a processing electronic device in communication with the display to control the presentation of image content on the display,

[1223] The head-mounted display system is configured to project light to the position of the user's eyes so as to present image content that is attenuated at least in part based on the lighting conditions of the environment to a portion of the central area of ​​the user's field of view.

[1224] 2. The system of Example 1, wherein under photopic lighting conditions, the image content is attenuated based on cone density at the location of the projected light.

[1225] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[1226] 4. The system of Example 1, wherein under scotopic lighting conditions, the image content is attenuated based on rod density at the location of the projected light.

[1227] 5. The system of embodiment 4, wherein the dark lighting condition of the environment has 10 -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[1228] 6. A system according to Example 1, wherein under interocular lighting conditions, the image content is attenuated based at least in part on time spent under the interocular lighting conditions.

[1229] 7. The system of example 6, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2 luminosity.

[1230] 8. A system according to Example 6 or 7, wherein the system is configured to determine whether cones or rods are dominant in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[1231] 9. A system according to any of Examples 6 to 8, wherein when the cones are dominant in the user's eye, the image content is attenuated based on the cone density of the projected light position.

[1232] 10. A system according to any of Examples 6 to 8, wherein when the rods are dominant in the user's eye, the image content is attenuated based on the rod density of the projected light position of the user's eye.

[1233] 11. A system according to any of the above examples, wherein the system includes a timer or clock to monitor the time spent under the lighting conditions.

[1234] 12. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[1235] 13. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project the light to the user.

[1236] 14. The system of Example 13, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[1237] 15. A system according to any of Examples 12 to 14, wherein the light source comprises a fiber scanning projector.

[1238] 16. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more image capture devices.

[1239] 17. A system according to Example 16, wherein the one or more image capture devices include one or more cameras.

[1240] 18. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more light sensors.

[1241] 19. The system of Example 18, wherein the one or more light sensors include one or more illuminance meters.

[1242] 20. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[1243] 21. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate the light level at the pixels to capture the lighting conditions.

[1244] 22. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[1245] 23. The system according to any of the above examples further includes a distance measuring device.

[1246] 24. The system of Example 23, wherein the distance measuring device comprises a laser rangefinder.

[1247] 25. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1248] 26. The system of any one of Examples 1 to 25, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[1249] 27. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the size of the image content based at least in part on lighting conditions of the environment.

[1250] 28. A system according to Example 27, wherein the reduction in size is based at least in part on a resolution of the eye.

[1251] 29. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the brightness in the image content based at least in part on lighting conditions of the environment.

[1252] 30. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce contrast in the image content based at least in part on lighting conditions of the environment.

[1253] 31. A system according to Example 30, wherein the head-mounted display system is configured to reduce contrast based at least in part on the contrast sensitivity of the eye.

[1254] 32. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the color saturation in the image content based at least in part on the lighting conditions of the environment.

[1255] 33. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the sharpness of the image content based at least in part on lighting conditions of the environment.

[1256] 34. A system according to example 33, wherein sharpening includes removing enhancement of edges of features in the image content based at least in part on lighting conditions of the environment.

[1257] 35. A system according to any of the above examples, wherein the head-mounted display system is configured to change the color balance of the image content based at least in part on the lighting conditions of the environment.

[1258] 36. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been impaired.

[1259] 37. A system according to Example 36, wherein the alarm is a visual alarm.

[1260] 38. A system according to Example 36, wherein the alarm is an audio alarm.

[1261] 39. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, wherein the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1262] 40. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1263] 41. A system according to Example 40, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1264] 42. A system according to Example 41, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1265] 43. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1266] 44. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1267] 45. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1268] 46. ​​A system according to Example 45, wherein the first image modality comprises an MRI scan.

[1269] 47. A system according to any of Examples 45 to 46, wherein the second imaging modality includes ultrasound.

[1270] 48. A system according to any of Examples 45 to 47, wherein the first imaging modality comprises an x-ray scan.

[1271] 49. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[1272] 50. The system according to any of the above examples further comprises a sensor suitable for converting ultrasonic waves into electrical signals.

[1273] 51. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[1274] 52. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1275] 53. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1276] 54. A system according to Example 53, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1277] 55. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the orientation of objects in the portion of the environment in front of the user.

[1278] 56. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[1279] 57. A system according to Example 56, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1280] 58. A system according to any of the above examples, wherein the head mounted display is configured to rotate the view of the 3D image of the object around an axis based on user input

[1281] 59. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1282] 60. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1283] 61. A system according to Example 60, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1284] 62. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1285] 63. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[1286] 64. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[1287] 65. A system according to example 64, wherein the medical parameters include vital signs.

[1288] 66. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1289] 67. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1290] 68. A system according to Example 21, wherein the detector array includes an outward-looking camera configured to image the environment.

[1291] 69. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into a position of the user's eyes so as to present image content to a portion of the central area of ​​the user's field of view that is attenuated relative to image content presented to a portion of the peripheral area.

[1292] 70. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into a position of the user's eyes so as to present image content that is attenuated relative to image content presented to a portion of the central area of ​​the user's field of view to another portion of the central area.

[1293] 71. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[1294] 72. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[1295] 73. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[1296] 74. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1297] 75. A system according to example 74, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1298] 76. A system according to Example 74 or 75, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[1299] Example Set IVF

[1300] 1. A head mounted display system configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising:

[1301] a frame configured to be supported on a head of the user;

[1302] a display disposed on the frame, the display configured to project light into the user's eyes so as to present image content at the peripheral region of the user's field of view, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display system, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user;

[1303] one or more capture devices configured to capture lighting conditions of the environment;

[1304] a processing electronic device in communication with the display to control the presentation of image content on the display,

[1305] The head-mounted display system is configured to project light to the position of the user's eyes so as to present image content that is attenuated at least in part based on the lighting conditions of the environment to a portion of the peripheral area of ​​the user's field of view.

[1306] 2. The system of Example 1, wherein under photopic lighting conditions, the image content is attenuated based on cone density at the location of the projected light.

[1307] 3. The system of Example 2, wherein the photopic lighting conditions of the environment have a light intensity of 10 cd / m 2 Up to 10 8 cd / m 2 luminosity.

[1308] 4. The system of Example 1, wherein under scotopic lighting conditions, the image content is attenuated based on rod density at the location of the projected light.

[1309] 5. The system of embodiment 4, wherein the dark lighting condition of the environment has 10 -3.5 cd / m 2 Up to 10 - 6 cd / m 2 luminosity.

[1310] 6. A system according to Example 1, wherein under interocular lighting conditions, the image content is attenuated based at least in part on time spent under the interocular lighting conditions.

[1311] 7. The system of example 6, wherein the ambient lighting condition of the environment has 10 -3 cd / m 2 Up to 10 0.5 cd / m 2 luminosity.

[1312] 8. A system according to Example 6 or 7, wherein the system is configured to determine whether cones or rods are dominant in the user's eyes based at least in part on time spent under the interocular lighting conditions.

[1313] 9. A system according to any of Examples 6 to 8, wherein when the cones are dominant in the user's eye, the image content is attenuated based on the cone density of the projected light position.

[1314] 10. A system according to any of Examples 6 to 8, wherein when the rods are dominant in the user's eye, the image content is attenuated based on the rod density of the projected light position of the user's eye.

[1315] 11. A system according to any of the above examples, wherein the system includes a timer or clock to monitor the time spent under the lighting conditions.

[1316] 12. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[1317] 13. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides configured to project the light to the user.

[1318] 14. The system of Example 13, wherein the one or more light sources are configured to direct light into the one or more waveguides.

[1319] 15. A system according to any of Examples 12 to 14, wherein the light source comprises a fiber scanning projector.

[1320] 16. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more image capture devices.

[1321] 17. A system according to Example 16, wherein the one or more image capture devices include one or more cameras.

[1322] 18. A system according to any of Examples 1 to 15, wherein the one or more capture devices include one or more light sensors.

[1323] 19. The system of Example 18, wherein the one or more light sensors include one or more illuminance meters.

[1324] 20. A system according to any of the above examples, wherein the one or more capture devices are configured to measure the luminosity of the environment.

[1325] 21. A system according to any of the above examples, wherein the one or more capture devices include a detector array comprising a pixel array, wherein the detector array is configured to integrate the light level at the pixels to capture the lighting conditions.

[1326] 22. A system according to any of the above examples, wherein the one or more capture devices include one or more inward-facing cameras, wherein the inward-facing cameras are configured to detect pupil size to capture the lighting conditions.

[1327] 23. The system according to any of the above examples further includes a distance measuring device.

[1328] 24. The system of Example 23, wherein the distance measuring device comprises a laser rangefinder.

[1329] 25. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1330] 26. The system of any one of Examples 1 to 25, further comprising one or more inward-facing image capture devices configured to image the user's eyes.

[1331] 27. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the size of the image content based at least in part on lighting conditions of the environment.

[1332] 28. A system according to Example 27, wherein the reduction in size is based at least in part on a resolution of the eye.

[1333] 29. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the brightness in the image content based at least in part on lighting conditions of the environment.

[1334] 30. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce contrast in the image content based at least in part on lighting conditions of the environment.

[1335] 31. A system according to Example 30, wherein the head-mounted display system is configured to reduce contrast based at least in part on the contrast sensitivity of the eye.

[1336] 32. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the color saturation in the image content based at least in part on the lighting conditions of the environment.

[1337] 33. A system according to any of the above examples, wherein the head-mounted display system is configured to reduce the sharpness of the image content based at least in part on lighting conditions of the environment.

[1338] 34. A system according to example 33, wherein sharpening includes removing enhancement of edges of features in the image content based at least in part on lighting conditions of the environment.

[1339] 35. A system according to any of the above examples, wherein the head-mounted display system is configured to change the color balance of the image content based at least in part on the lighting conditions of the environment.

[1340] 36. A system according to any of the above examples, wherein the system is configured to provide an alert to the user to indicate that the image content has been impaired.

[1341] 37. A system according to Example 36, wherein the alarm is a visual alarm.

[1342] 38. A system according to Example 36, wherein the alarm is an audio alarm.

[1343] 39. A system according to any of the above examples, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, wherein the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1344] 40. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1345] 41. A system according to Example 40, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1346] 42. A system according to Example 41, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1347] 43. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1348] 44. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1349] 45. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1350] 46. ​​A system according to Example 45, wherein the first image modality comprises an MRI scan.

[1351] 47. A system according to any of Examples 45 to 46, wherein the second imaging modality includes ultrasound.

[1352] 48. A system according to any of Examples 45 to 47, wherein the first imaging modality comprises an x-ray scan.

[1353] 49. The system according to any of the above examples further includes an electronic transmitter suitable for generating ultrasonic waves.

[1354] 50. The system according to any of the above examples further comprises a sensor suitable for converting ultrasonic waves into electrical signals.

[1355] 51. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[1356] 52. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1357] 53. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1358] 54. A system according to Example 53, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1359] 55. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the orientation of objects in the portion of the environment in front of the user.

[1360] 56. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[1361] 57. A system according to Example 56, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1362] 58. A system according to any of the above examples, wherein the head mounted display is configured to rotate the view of the 3D image of the object around an axis based on user input

[1363] 59. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1364] 60. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1365] 61. A system according to Example 60, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1366] 62. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1367] 63. A system according to any of the above examples, wherein the head-mounted display is configured to alert a user to steps in a medical procedure.

[1368] 64. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters and provide alerts based on the medical parameters.

[1369] 65. A system according to example 64, wherein the medical parameters include vital signs.

[1370] 66. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1371] 67. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1372] 68. A system according to Example 21, wherein the detector array includes an outward-looking camera configured to image the environment.

[1373] 69. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into a position of the user's eyes so as to present image content that is attenuated relative to image content presented to a portion of the central area to a portion of the user's field of view.

[1374] 70. A system according to any of the above examples, wherein the head-mounted display system is configured to project light into a position of the user's eyes so as to present image content that is attenuated relative to image content presented to a portion of the peripheral area of ​​the user's field of view to another portion of the peripheral area.

[1375] 71. A system according to any of the above examples, wherein the system is configured to determine how to present the image content in the eyes of the user based at least in part on temporal aspects of the lighting conditions of the environment.

[1376] 72. A system according to any of the above examples, wherein the display is configured to project light into the user's eyes with different amounts of divergence so as to present the image content as if the image content comes from different depths.

[1377] 73. A system according to any of the above examples, wherein the head-mounted display is configured to provide opacity at least in the vicinity of the presented image content.

[1378] 74. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1379] 75. A system according to example 74, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1380] 76. A system according to Example 74 or 75, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient.

[1381] Sample Set V

[1382] 1. A head mounted display system for medical treatment and / or diagnosis, configured to project light into a user's eye to display augmented reality image content, the user's eye having a field of view, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display comprising:

[1383] a frame configured to be supported on a head of the user;

[1384] a display disposed on the frame, the display configured to project light into the user's eyes at multiple depth planes, at least a portion of the display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user to the user's eyes to provide a view of the portion of the environment in front of the user; and

[1385] Processing electronics communicate with the display to control presentation of image content on the display.

[1386] 2. A system according to Example 1, wherein the head-mounted display system is configured to construct a 3D representation of at least a portion of the environment in front of the user and interpret the representation of at least a portion of the environment, the portion of the environment includes a patient, and the head-mounted display is further configured to distinguish between a first structure associated with the patient and a second structure associated with the patient.

[1387] 3. A system according to any of the above examples, wherein at least a portion of the display that is transparent and positioned in front of the user's eyes includes one or more waveguides.

[1388] 4. The system of Example 3, comprising one or more light sources configured to direct light into the one or more waveguides, wherein the waveguides are configured to direct light into the user's eyes.

[1389] 5. A system according to any of Examples 3 to 4, wherein the one or more light sources include a fiber scanning projector.

[1390] 6. The system according to any of the above examples further includes one or more sensors configured to monitor the environment.

[1391] 7. A system according to Example 6, wherein the one or more sensors include one or more outward-facing image capture devices configured to image the environment.

[1392] 8. The system of Example 7, wherein the one or more outward-facing image capture devices configured to image the environment include one or more outward-facing cameras.

[1393] 9. The system according to any of the above examples further includes a distance measuring device.

[1394] 10. The system according to any of the above examples further includes an eye tracking device configured to track the position and / or movement of the user's eyes.

[1395] 11. A system according to any of the above examples, wherein the head-mounted display is configured to estimate the volume of human tissue within the user's field of view.

[1396] 12. A system according to any of the above examples, wherein the head-mounted display is configured to measure the distance between two objects in the environment.

[1397] 13. A system according to any of the above examples, wherein the head-mounted display is configured to switch between a second image modality and a first image modality presented on the display.

[1398] 14. A system according to Example 13, wherein the first image modality comprises an MRI scan.

[1399] 15. A system according to any of Examples 13 to 14, wherein the second imaging modality comprises ultrasound.

[1400] 16. A system according to any of Examples 13 to 15, wherein the first imaging modality comprises an x-ray scan.

[1401] 17. The system according to any of the above examples further comprises an electronic transmitter suitable for generating ultrasonic waves.

[1402] 18. The system according to any of the above examples further comprises a sensor suitable for converting ultrasonic waves into electrical signals.

[1403] 19. A system according to any of the above examples, wherein the head-mounted display is configured to allow a user to place a virtual reference marker on the portion of the environment in front of the user to the user's eyes.

[1404] 20. A system according to any of the above examples, wherein the head-mounted display is configured to project an image onto the display so that the image appears to be attached to real-world objects in the environment.

[1405] 21. A system according to any of the above examples, wherein the head-mounted display is configured to display a virtual cutting guide line so that the virtual cutting guide line appears to the user to be overlaid on the human body area to be cut or allows access to the area to be cut.

[1406] 22. A system according to Example 21, wherein the apparent position of the virtual cutting guide line appears to be related to the orientation of the patient's body part.

[1407] 23. A system according to any of the above examples, wherein the head-mounted display is configured to transmit signals to obtain data about the position of objects in the portion of the environment in front of the user.

[1408] 24. A system according to any of the above examples, wherein the head-mounted display is configured to use a database of object locations to obtain the orientation of objects in the portion of the environment in front of the user.

[1409] 25. A system according to Example 24, wherein the head-mounted display is configured to set a reference point based on a database of object locations and project an image into the user's eyes so that the image appears fixed relative to the reference point.

[1410] 26. A system according to any of the above examples, wherein the head-mounted display is configured to rotate a view of a 3D image of an object around an axis based on user input.

[1411] 27. A system according to any of the above examples, wherein the head-mounted display is configured to translate the view of the image of the 3D object based on user input.

[1412] 28. A system according to any of the above examples, wherein the head-mounted display is configured to display a first slice of a 3D image of an object.

[1413] 29. A system according to Example 28, wherein the head-mounted display is configured to sequentially pass through an image of the first slice and an image of a second slice of the 3D image.

[1414] 30. A system according to any of the above examples, wherein the head-mounted display is configured to transmit an image of a portion of the environment in front of the user, so that a second user of the head-mounted display can view the transmitted image of the portion of the environment.

[1415] 31. A system according to any of the above examples, wherein the head-mounted display is configured to alert the user to steps in a medical procedure.

[1416] 32. A system according to any of the above examples, wherein the head-mounted display is configured to monitor medical parameters of the user and provide alerts based on the medical parameters.

[1417] 33. A system according to Example 32, wherein the user's medical parameters include vital signs.

[1418] 34. A system according to any of the above examples, wherein the head-mounted display is configured to emit ultrasonic waves and measure signals generated by the ultrasonic waves, and wherein the head-mounted display is further configured to form an ultrasonic image based on the signals.

[1419] 35. A system according to any of the above examples, wherein the head-mounted display is configured to warn the user of objects and / or events outside the user's field of view.

[1420] 36. The system according to any of the above examples further includes one or more light sources configured to direct light into the user's eyes to form an image in the eyes.

[1421] 37. A system according to any of the above examples, wherein the one or more light sources include a fiber scanning projector.

[1422] 38. A system according to any of the above examples, wherein the head-mounted display is configured to emit energy toward the patient and sense returned energy.

[1423] 39. A system according to Example 38, wherein the energy includes ultrasound.

[1424] 40. A system according to any of the above examples, wherein the head-mounted display is configured to combine a first image modality with a second image modality different from the first image modality.

[1425] 41. A system according to Example 40, wherein the first image modality and the second image modality each include images from an MRI, CT, PET, MRA, or CTA scan.

[1426] 42. A system according to Example 40 or 41, wherein the head-mounted display is configured to align a combined image of the first image modality and the second image modality over the actual anatomy of the patient. BRIEF DESCRIPTION OF THE DRAWINGS

[1427] Figure 1A An augmented reality (AR) view viewed by a user through an AR device is shown.

[1428] Figure 1B Shows a person's field of view and visual range.

[1429] Figure 1C-1 and 1C-2 A first-person perspective showing what parts of a scene the central and peripheral fields of view may include.

[1430] Figure 1D A cross section of a human eye is shown.

[1431] Figure 1E and 1F The relative density and resolution of cones and rods in a typical human eye are shown, respectively.

[1432] Figure 1G Visual function under different lighting conditions is shown.

[1433] Figure 2 An example of a wearable display system is shown.

[1434] Figure 3 A conventional display system for simulating a three-dimensional image for a user is shown.

[1435] Figure 4 Aspects of a method of simulating a three-dimensional image using multiple depth planes are shown.

[1436] Figures 5A to 5CThe relationship between the radius of curvature and the focal radius is shown.

[1437] Figure 6 An example of a waveguide stack for outputting image information to a user is shown.

[1438] Figure 7 An example of an outgoing light beam output by a waveguide is shown.

[1439] Figure 8 An example of a stacked waveguide assembly is shown where each depth plane includes an image formed using multiple different component colors.

[1440] Fig. 9A A cross-sectional side view of an example of a set of stacked waveguides, each stacked waveguide including an incoupling optical element, is shown.

[1441] Fig. 9B Shows Fig. 9A A perspective view of an example of multiple stacked waveguides.

[1442] Fig. 9C Shows Fig. 9A and Fig. 9B A top plan view of an example of multiple stacked waveguides.

[1443] Fig. 10A A schematic diagram showing an example of various components of an augmented reality system including environment and user sensors.

[1444] Fig. 10B A schematic diagram of another example of various components of an augmented reality system including environment and user sensors is shown.

[1445] Fig.11A is a flow chart illustrating an example of a method for enhancing a user view of image content of an environment.

[1446] Fig. 11B is a flow chart illustrating another example of a method for enhancing a user view of image content of an environment.

[1447] Fig. 12A is a flow chart illustrating an example of a method for image modification.

[1448] Fig. 12B An example application of some embodiments described herein is shown.

[1449] Fig. 12C An example of location-based image modification is shown.

[1450] Fig.13A is a block diagram of an example of a wearable system.

[1451] Fig. 13B is a process flow diagram of an example of a method of rendering virtual content related to a recognized object.

[1452] Fig. 13C is a block diagram of another example of a wearable system.

[1453] Fig.14 is a flow chart illustrating an example of a method for changing a user's perception of a real or virtual object within a user's field of view based at least in part on user intent. DETAILED DESCRIPTION

[1454] The eye is a complex organ that collects and senses reflected and emitted light from our environment to provide useful information, such as the shape, features, and location of objects around us. Improving our ability to perceive objects with our eyes can help us in the pursuit of various endeavors. One example where improved vision is particularly beneficial is for medical practitioners, such as surgeons who perform medical work such as surgery, diagnosis, and / or treatment. Improved vision also helps with everyday tasks that require concentration, such as operating a motor vehicle or other means of transportation.

[1455] At any given moment, a person's view of the world or its surroundings is represented by a visual field having a central region and a peripheral region. This visual field can change as the person moves around, moves their head, or moves their eyes or gaze. Figure 1B Such a field of view 1755 including a central area and a peripheral area is shown. Figure 1B Also shown is a field of view 1765, which includes a portion of the environment around person 60 that can be perceived by person 60 (e.g., by turning their head or redirecting their gaze). The central portion of the field of view 1755 of the eyes of person 60 may be referred to as the central field of view 1745. The area within the field of view 1755 but outside the central field of view 1745 may be referred to as the peripheral field of view.

[1456] The central field of view will provide the person with a corresponding view of objects in the central region of the view of the environment. Similarly, the peripheral field of view will provide the person with a corresponding view of objects in the peripheral region of the view of the environment. In this case, what is considered central and what is considered peripheral depends on the direction the person is looking, and therefore depends on their field of view.

[1457] Figure 1C-1 and 1C-2 A first-person perspective of the portion of the scene that the central field of view and the peripheral field of view may include is shown. For example, Figure 1C-1 An example of what a person may see in his or her field of view 1755 is shown. Field of view 1755 may include objects 1721, 1722. Figure 1C-2As shown, the central field of view 1745 includes the object 1721, and Figure 1C-1 Another object 1722 is shown in the peripheral field of view (eg, a shadow area).

[1458] like Figure 1D , a schematic cross-sectional view of a human eye 1100 is shown, which includes a cornea 42, an iris 44, a lens or "crystalline lens" 46, a sclera 48, a choroid layer 50, a macula 52, a retina 54, and an optic nerve pathway 56 to the brain. The cornea 42 and the lens 46 refract and focus light onto the retina 54. The macula 52 is the center of the retina 54. At the center of the macula 52 is a portion of the retina 54 called the "fovea." The retina 54 contains photoreceptor cells called cones and rods. Near the inner surface of the retina 54 are ganglion cells, which receive visual information from the cones and rods and transmit the visual information to the brain. The visual information may include information used to form an image.

[1459] The fovea contains more photoreceptor cells than any other part of the retina 54 (approximately 120 cones per visual degree). Figure 1E The relative density of cones and rods in a typical human eye is shown, plotted as a function of angle from the center of the retina (eg, measured from the optical axis through the eye lens to the center of the retina). Figure 1E The relative density 1852 of cones is shown to be highest at the center of the retina (e.g., the fovea) and decreases significantly after several degrees away from the center. In contrast, there are essentially no rods at the center, but the density 1856 of rods increases significantly after several degrees away from the center and decreases across the retina.

[1460] Figure 1F The figure shows the relative resolution of cones and rods over the same spectrum of angles on the retina. Figure 1F As shown, similar to the density of cones, the resolution of cones 1842 is highest at the fovea and decreases as the angle away from the fovea increases. Similar to the density of rods, the resolution of rods 1846 increases first and then gradually decreases.

[1461] Because the fovea is located in the center of the eye, the central field of vision falls on the fovea. Cones are generally responsible for color (e.g., l cones or long wavelength sensitive cones can be used for red wavelengths, m cones or medium wavelength sensitive cones can be used for green wavelengths, and s cones or short wavelength sensitive cones can be used for blue wavelengths) and spatial resolution. Rods are insensitive to color and are generally responsible for basic spatial resolution (e.g., for detecting shape and movement). Because the fovea contains the highest density of cones, it provides high resolution for objects located in a person's central field of vision. As one moves away from the fovea, the cone density decreases, as does the resolution of objects located in a person's peripheral field of vision (e.g., from the near peripheral field of vision to the far peripheral field of vision). Because the cone density decreases starting from the fovea, the peripheral field of vision is often associated with poor image quality compared to the central field of vision. However, it may be desirable to allow the human eye to focus more on the central field of vision (e.g., Figure 1B 1745) as the center...

Claims

1. A head mounted display system configured to project light into eyes of a user to display augmented reality image content in a field of view of the user, the field of view of the user being a portion of a visible area surrounding the user and perceivable by the user, the field of view having a central area and a peripheral area disposed around the central area, the head mounted display system comprising: a frame configured to be supported on a head of the user; a head mounted display disposed on the frame, the head mounted display configured to project light into the user's eyes so as to display augmented reality image content to the user's field of view, at least a portion of the head mounted display being transparent and disposed at a position in front of the user's eyes when the user is wearing the head mounted display, such that the transparent portion transmits light from a portion of the environment in front of the user and the head mounted display to the user's eyes to provide a view of the portion of the environment in front of the user and the head mounted display; one or more user sensors configured to sense the user; one or more environmental sensors configured to sense the user's surrounding environment, the surrounding environment including one or more objects or events in the visible area surrounding the user but outside the field of view of the user; a processing electronic device in communication with the head mounted display, the one or more user sensors, and the one or more environmental sensors, the processing electronic device being configured to: sensing situations involving increased user attention; Based at least in part on sensing increased user attention to a real or virtual peripheral object in the peripheral region of the user's field of view, attenuating image content in portions of the field of view other than the peripheral object, wherein sensing the increased user attention to the peripheral object is based at least in part on squinting of the user's eyes toward the peripheral object or accommodation fluctuations of the user's eyes when the user's eyes are directed toward the peripheral object; and Warning the user of an event or object occurring within the visible area but outside the field of view of the user, wherein warning the user of an event or object occurring within the visible area but outside the field of view of the user comprises displaying an image of the event or the object outside the field of view of the user in the field of view of the user.

2. The head mounted display system according to claim 1, wherein: The processing electronic device is configured to attenuate the image content in the portion of the field of view excluding the peripheral object by changing at least one of the contrast, opacity, color, color saturation, color balance, size, background, brightness, edge, or sharpness of the image content in the portion of the field of view excluding the peripheral object.

3. The head mounted display system according to claim 1, wherein: The one or more environmental sensors include one or more outward-facing image capture devices, depth sensors, binocular world camera pairs, geo-positioning sensors, proximity sensors, distance measurement devices, or GPS.

4. The head mounted display system according to claim 1, wherein: The one or more user sensors include one or more cameras.

5. The head mounted display system according to claim 1, wherein: The processing electronic device is further configured to determine user intent with respect to the situation and to change user perception of the real or virtual peripheral objects within the field of view of the user based at least in part on sensing the increased user attention.

6. The head mounted display system according to claim 1, wherein: Such situations involving increased user attention include driving a motor vehicle.

7. The head mounted display system according to claim 1, wherein: The one or more environmental sensors include a sensor configured to detect radio signals.

8. The head mounted display system according to claim 1, wherein: The one or more environmental sensors include a sensor configured to detect a Bluetooth signal from the automobile.

9. The head mounted display system according to claim 6, wherein: The processing electronics is configured to attenuate image content in the portion of the field of view excluding the peripheral objects based at least in part on one or more data records about the user, the one or more data records including a driving record of the user.

Images