Serving multi-layered augmented reality images to users

Abstract

A device for supplying multi-plane images for viewing by a user includes an image generator, an image guide, and a first output port. The image generator generates a first image that will be viewed by the user as a first distance from the user's viewpoint and a second image that will be viewed by the user as a second distance from the user's viewpoint. The first image includes a number of light wavelength components, and the second image includes the number of light wavelength components. The image guide is configured to guide the first image through a first optical path to the first output port of the device, and to guide the second image through a second optical path to the first output port of the device. The first optical path corresponds to the first distance, and the second optical path corresponds to the second distance. The first optical path and the second optical path have different lengths. The first output port is configured to be connected to a first optical waveguide, which is configured to guide the number of light wavelength components to a user display device.

Description

Background Art

[0001] The present invention relates to extended reality technology, and more particularly, to technology for presenting multi-layer extended reality images to a user.

[0002] It is expected that extended reality (XR) systems will soon become a common way for people to receive and interact with various digital information. Extended reality (XR) systems (e.g., augmented reality (AR) and mixed reality (MR) systems) can be used to bridge the gap between the physical and digital worlds, thereby enhancing digital information and projecting it into the physical world.

[0003] A head mounted display (HMD) is one possible device implementation in an XR system. Conventional XR glasses rely on computing power and light sources (e.g., LED, laser, or other projection technologies) integrated into the XR glasses, which makes them heavy and bulky. Therefore, there is a need to find solutions that allow XR glasses to be smaller, less obtrusive, and smarter and provide an improved user experience. Summary of the invention

[0004] It should be emphasized that, when used in this specification, the terms "include" and "comprising" are considered to specify the presence of claimed features, integers, steps or components; but the use of these terms does not exclude the presence or addition of one or more other features, integers, steps, components or combinations thereof.

[0005] In addition, in some cases (e.g., in the claims and the summary of the invention), reference letters may be provided to facilitate identification of various steps and / or elements. However, the use of reference letters is not intended to attribute or suggest that the steps and / or elements so referenced be performed or operated in any particular order.

[0006] According to one aspect of the present invention, the foregoing and other objects are achieved in a technique (e.g., a method, an apparatus, a non-transitory computer-readable storage medium, a program device) for providing a multi-plane image for viewing by a user. The technique includes an image generator, an image guide, and a first output port. The image generator generates a first image to be viewed by the user as a first distance from the user's viewpoint and a second image to be viewed by the user as a second distance from the user's viewpoint. The first image includes a number of optical wavelength components, and the second image includes the number of optical wavelength components. The image guide guides the first image through a first optical path to the first output port of the device, and guides the second image through a second optical path to the first output port of the device, wherein the first optical path corresponds to the first distance, the second optical path corresponds to the second distance, and wherein the first optical path and the second optical path have different lengths. The first output port is connected to a first optical waveguide, which is configured to guide the number of optical wavelength components to a user display device.

[0007] In another aspect of some embodiments, the technology further includes: the first optical waveguide coupled to the first output port at a first end; and a see-through optical waveguide coupled to a second end of the first optical waveguide.

[0008] In another aspect of some embodiments, the image director includes: a first electromechanical mirror array located in the path of the first image and the second image supplied by the image generator; and a controller. The controller causes the first electromechanical mirror array to direct the first image to the first optical path, and causes the first electromechanical mirror array to direct the second image to the second optical path.

[0009] In another aspect of some embodiments, the image generator time-multiplexes the generation of the first image and the second image. When the image generator is generating the first image, the controller causes the first electromechanical mirror array to direct the first image to the first optical path; and when the image generator is generating the second image, the controller causes the first electromechanical mirror array to direct the second image to the second optical path.

[0010] In another aspect of some embodiments, the image generator generates the first image and the second image simultaneously as components of an overall image, wherein the first image is located within a first portion of the overall image and the second image is located within a second portion of the overall image; the first electromechanical mirror array has a first area portion and a second area portion; the first electromechanical mirror array is arranged in the device so that the first area portion of the first electromechanical mirror array is in the path of the first image output by the image generator and the second area portion of the first electromechanical mirror array is in the path of the second image output by the image generator. The controller causes the first area portion of the first electromechanical mirror array to direct the first image to the first optical path; and causes the second area portion of the first electromechanical mirror array to direct the second image to the second optical path.

[0011] In another aspect of some embodiments, the first image and the second image are configured to be viewed by a first eye of the user; the image generator generates a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components and the second image includes the number of light wavelength components, and wherein the third image and the fourth image are configured to be viewed by a second eye of the user. The image generator simultaneously generates the first image, the second image, the third image, and the fourth image as components of the overall image, wherein the third image is located within a third portion of the overall image and the fourth image is located within a fourth portion of the overall image. In addition, in such embodiments, the first electromechanical mirror array has a third area portion and a fourth area portion; the first electromechanical mirror array is arranged in the device so that the third area portion of the first electromechanical mirror array is in the path of the third image output by the image generator, and the fourth area portion of the first electromechanical mirror array is in the path of the fourth image output by the image generator. In such embodiments, the controller causes the third area portion of the first electromechanical mirror array to direct the third image to a third optical path corresponding to the first distance; and causes the fourth area portion of the first electromechanical mirror array to direct the fourth image to a fourth optical path corresponding to the second distance, wherein:

[0012] the first electromechanical mirror array being configured to direct the first image to the first output port via the first optical path and to direct the third image to a second output port via the third optical path, wherein the second output port is configured to connect to a second optical waveguide configured to direct the number of optical wavelength components to the user display device; and

[0013] The technique also includes a second electromechanical mirror array configured to direct the second image to the first output port via the second optical path and to direct the fourth image to the second output port via the fourth optical path.

[0014] In another aspect of some embodiments, the first image and the second image are configured to be viewed by a first eye of the user; the image generator generates a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components, the fourth image includes the number of light wavelength components, and wherein the third image and the fourth image are configured to be viewed by a second eye of the user; the image director receives the first image and the second image from the image generator during a first time period; the image director receives the first image and the second image from the image generator during a second time period An image generator receives the third image and the fourth image; the image guide is configured to guide the first image to the first output port via the first optical path during the first time period; the image guide is configured to guide the second image to the first output port via the second optical path during the first time period; the image guide is configured to guide the third image to the second output port via the third optical path during the second time period, wherein the third optical path corresponds to the first distance; and the image guide is configured to guide the fourth image to the second output port via a fourth optical path during the second time period, wherein the fourth optical path corresponds to the second distance.

[0015] In another aspect of some embodiments, the first time period includes a third time period and a fourth time period; the second time period includes a fifth time period and a sixth time period; when it is the third time period, the image guide causes the first image to be guided to the first output port via the first optical path; when it is the fourth time period, the image guide causes the second image to be guided to the first output port via the second optical path; when it is the fifth time period, the image guide causes the third image to be guided to the second output port via the third optical path; and when it is the sixth time period, the image guide causes the fourth image to be guided to the second output port via the fourth optical path.

[0016] In another aspect of some embodiments, the image guide includes a second electromechanical mirror array, which is arranged in the second optical path and the fourth optical path and is configured to: guide the second image to the first output port during the first time period; and guide the fourth image to the second output port during the second time period.

[0017] In another aspect of some embodiments, the first image and the second image are configured to be viewed by a first eye of the user; the image generator generates a third image to be viewed by the user as being at the first distance from the user's viewpoint and a fourth image to be viewed by the user as being at the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components, the fourth image includes the number of light wavelength components, and wherein the third image and the fourth image are configured to be viewed by a second eye of the user; the image director receives the first image and the third image from the image generator during a first time period; the image director receives the first image and the third image during a second time period receiving the second image and the fourth image from the image generator during the first time period; the image guide being configured to guide the first image to the first output port via the first optical path during the first time period; the image guide being configured to guide the third image to the second output port via a third optical path during the first time period, wherein the third optical path corresponds to the first distance; the image guide being configured to guide the second image to the first output port via the second optical path during the second time period; and being configured to guide the fourth image to the second output port via a fourth optical path during the second time period, wherein the fourth optical path corresponds to the second distance.

[0018] In another aspect of some embodiments, the first time period includes a third time period and a fourth time period; the second time period includes a fifth time period and a sixth time period. When it is the third time period, the image guide causes the first image to be guided to the first output port via the first optical path; when it is the fourth time period, the image guide causes the third image to be guided to the second output port via the third optical path; when it is the fifth time period, the image guide causes the second image to be guided to the first output port via the second optical path; and when it is the sixth time period, the image guide causes the fourth image to be guided to the second output port via the fourth optical path.

[0019] In another aspect of some embodiments, the image director includes a second electromechanical mirror array disposed in the second optical path and the fourth optical path. The image director is configured to: direct the second image to the first output port during the first time period; and direct the fourth image to the second output port during the second time period.

[0020] In another aspect of some embodiments, the technology also includes an autofocus lens device, which is dynamically controlled by the controller so that the first distance has a first optical length when a first control signal is generated and has a second optical length when a second control signal is generated, wherein the first optical length and the second optical length are different. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Objects and advantages of the present invention will be understood from the following detailed description read in conjunction with the accompanying drawings, in which:

[0022] FIG1 depicts a conventional head mounted display;

[0023] FIG2 depicts an arrangement in which the image generator is not integrated with the head mounted display and is therefore separate from the see-through optical waveguide;

[0024] Figure 3 , Figure 4 and Figure 5 depicts an arrangement of a remote device having an image generator which supplies images to a see-through waveguide by means of a suitable optical waveguide, and wherein time multiplexing is used to display images located on a plurality of virtual planes;

[0025] Figure 6 , Figure 7 and Figure 8 illustrates aspects of pixel resolution partitioning according to some aspects of the present technology;

[0026] Fig. 9 and Fig.10An aspect of the disclosed technology is shown in which pixel resolution partitioning is used to display images located on multiple virtual planes;

[0027] Fig.11 and Fig.12 Further aspects of pixel resolution partitioning associated with stereoscopic images having multiple virtual planes according to some aspects of the present technology are shown;

[0028] Fig.13 , Fig.14 and Fig.15 An exemplary embodiment is shown in which pixel resolution division is used in conjunction with time division multiplexing to display stereoscopic images located on multiple virtual planes;

[0029] Fig.16 and Fig.17 An alternative exemplary embodiment is shown in which pixel resolution division is used in conjunction with time division multiplexing to display stereoscopic images located on multiple virtual planes;

[0030] Fig.18 Another alternative exemplary embodiment is shown in which pixel resolution division is used to display stereoscopic images located on multiple virtual planes;

[0031] Fig.19A , Fig.19B and Fig.19C shows an exemplary arrangement for dynamically changing the position of a virtual plane according to some embodiments consistent with the present invention;

[0032] Fig. 20 In some aspects is a flow chart of actions performed in some exemplary embodiments to display multiple virtual planar images using time multiplexing;

[0033] Fig.21 In some aspects is a flow chart of actions performed in some exemplary embodiments to display multiple virtual planar images utilizing pixel resolution division;

[0034] Fig. 22 is a block diagram of arrangements according to some but not necessarily all embodiments consistent with the present invention;

[0035] Fig.23 An exemplary controller of a remote device is shown in accordance with some but not necessarily all exemplary embodiments consistent with the present invention. DETAILED DESCRIPTION

[0036] The various features of the present invention will now be described with reference to the drawings, in which like parts are identified with the same reference numerals.

[0037] Now will be described in more detail various aspects of the present invention in conjunction with a plurality of exemplary embodiments.In order to facilitate understanding of the present invention, many aspects of the present invention are described according to the action sequence that will be performed by the element of the computer system or other hardware capable of executing programming instructions.It will be appreciated that in each embodiment, various actions can be performed by a dedicated circuit (e.g., analog and / or discrete logic gates interconnected to perform a dedicated function), by one or more processors programmed with a suitable instruction set, or by a combination of the two.The term "circuit configured to perform one or more of the actions" is used herein to refer to any such embodiment (i.e., one or more dedicated circuits, one or more programming processors, or any combination thereof).In addition, the present invention may additionally be considered to be fully embodied in any form of non-transitory computer readable carrier (such as a solid-state memory, a disk or an optical disk containing a suitable computer instruction set that will cause the processor to perform the technology described herein).Therefore, various aspects of the present invention can be embodied in many different forms, and all such forms are conceived to be within the scope of the present invention.For each of the various aspects of the present invention, any such form of embodiment as described above may be referred to herein as "logic configured to perform the described actions", or alternatively as "logic to perform the described actions".

[0038] One aspect of the technology described herein involves moving all scene rendering computing power and related hardware (such as LED light sources, LCOS, and lens packaging) from the XR glasses to a separate XR computing device, which can be placed in a pocket or worn on a belt, for example. This separation can be achieved by inserting one or more optical cables between the lens packaging and the waveguide in the XR glasses.

[0039] To illustrate this aspect, reference is made to Figure 1, which depicts a conventional head mounted display 100. For example, the head mounted display 100 may be implemented as glasses or a helmet.

[0040] It is desirable to display the first virtual image 105 and the second virtual image 107 in such a manner that the first image 105 is viewed by the user as being at a first distance d1 from the user's viewpoint 109, and the second image 107 is viewed by the user as being at a second distance d2 from the user's viewpoint 109. The first distance d1 and the second distance d2 may also be referred to herein as a first virtual plane and a second virtual plane (VP). The two virtual planes are displaced from each other by a distance d3.

[0041] The first image 105 includes a certain number (n) of light wavelength components (e.g., in some but not necessarily all embodiments, three optical wavelengths corresponding to red, green, and blue-RGB), and the second image 107 includes the same number (n) of light wavelength components. To achieve this, the image generator 101 in the head mounted display 100 projects all the light wavelength components to a corresponding set of see-through waveguides 103. In this example, since there are two virtual planes and each plane has three light wavelength components, the image generator 101 projects a total of six light wavelength components (Rd1, Gd1, Bd1, Rd2, Gd2, Bd2) onto the see-through waveguides 103, and does so so that each of the light wavelength components will be guided through a corresponding one of the six see-through waveguides 103 (three see-through waveguides per virtual plane), as shown.

[0042] The see-through waveguide 103 may be implemented as, for example, a holographic waveguide, which is a display technology often used in optical see-through AR to achieve a small form factor design.

[0043] Figure 2 shows an alternative arrangement still involving an image generator 201 and a see-through waveguide 203, but in which the image generator 201 is no longer integrated with the head mounted display and is therefore separated from the see-through optical waveguide 203. Because of this separation, the image generator 201 can no longer project an image directly onto the see-through waveguide 203. To address this problem, an optical fiber bundle 205 (which is also a type of optical waveguide) is introduced.

[0044] Optical fibers provide a direct, one-to-one image transmission mechanism. Typically, there is a one-to-one correspondence between the fibers and the pixels in the complete image. If the image needs to be larger, it can be magnified by an optical lens system at some point before the image enters the holographic waveguide. However, magnifying the image in this way has the disadvantage of some pixelation (i.e., some degradation of the image quality due to insufficient pixel information for the magnified image).

[0045] Optical fibers can be made very thin. Some have been made as small as 0.8 μm. To be able to transmit a full high-definition (FHD) image through, for example, an optical fiber with a thickness of 1 μm per pixel, the size of the optical cable would be about 6 mm × about 3.3 mm ((1920 pixel resolution × 3 sub-pixels per pixel × 1 μm) × (1080 pixel resolution × 3 sub-pixels per pixel × 1 μm)), in other words, about 19 mm 2 In an XR device with one display per eye, two optical cables with these dimensions would be needed.

[0046] It has been shown that multiple wavelengths of light can be propagated through a single optical fiber. Therefore, in embodiments utilizing this technique, the number of optical fibers used can be less than the number of optical wavelength components associated with the virtual planes, since a single optical fiber is capable of carrying multiple wavelength components. However, conventional techniques still require that the number of optical fibers be repeated for each different virtual plane.

[0047] As shown, each optical fiber 205 receives a corresponding one of the optical wavelength components projected by the image generator 201 and guides the optical wavelength component to a corresponding one of the perspective waveguides 203. It should be noted that the description of the one-to-one correspondence between the optical fiber and the wavelength component is for the purpose of simplifying the discussion. In practical embodiments, the same optical fiber can be used to carry some or all of the wavelength components associated with the image pixels (e.g., all three of the red, green, and blue wavelength components), where the projectors in such systems operate in a field sequential manner so that, for example, the red portion of the image is projected first, then the green portion, and then the blue portion, and so on.

[0048] In the arrangement of Figure 2, the display system of the image generator 201 can be the same set of hardware components as the hardware components in the waveguide AR head-mounted display. Thus, for example, the display system may include a liquid crystal on silicon (LCOS) display having a light source or micro organic light emitting diode (OLED) that supplies an image to a holographic waveguide. In addition to the image generating components, optical devices for generating the distance from the virtual plane are also placed here (but not shown to avoid cluttering the graphics). The benefit of adding optics to this part of the system (i.e., separated from the display device that the user sees through) is that the optics are no longer spatially constrained as when the optics are placed in, for example, the temples of AR glasses. This means that the optical components can be made relatively large, which enables them to manipulate light in a better / simpler way. Optical properties that can be easily changed include the virtual plane, image size, and image position.

[0049] Specifically and as an example, having more room for mounting optics and other components therein means that the size and resolution of the image can be increased compared to head mounted display binding techniques. In one refinement, having a larger image size to work with enables different display surfaces to be used for different purposes. For example, in one aspect of the present technology, a central portion of a perceived image can be used to display an image, while a peripheral portion (e.g., a bottom portion) can be used to display a status field.

[0050] By separating the host system (including the image generator 201) from the surface (e.g., the see-through waveguide 203) on which the image is displayed, the processing of all images to be displayed on the see-through waveguide 203 can also be moved out of the head mounted device. As a result, the head mounted device does not require batteries or any other electrically driven components to display images to the user. This enables the head mounted device to be manufactured in a very slim manner compared to older technologies.

[0051] The arrangement described so far can be further improved and these improvements are now described in more detail.

[0052] Reference now Figure 3 , a remote device 300 having an image generator 301 supplies an image to a perspective waveguide 303 by means of a suitable optical waveguide (such as the optical fiber bundle 305 shown). Each image pixel includes multiple optical wavelength components. For example, but not limited to, each image pixel may include three optical wavelength components corresponding to red, green, and blue pixel information.

[0053] In one aspect of some but not necessarily all embodiments consistent with the present invention, there is an additional component in the remote device 300 that enables the remote device 300 to change the optical path in a dynamic manner. By doing so, the virtual distance of an object can be switched by dynamically changing the optical path inside the computing device. One benefit of doing so is that it eliminates the need to have a set of dedicated waveguide layers for each virtual plane to be depicted. For example, where FIG. 2 shows an embodiment requiring six optical fibers 205 and a corresponding set of six perspective waveguides 203 (i.e., three waveguide layers for a virtual image at distance d1, and three additional waveguide layers for a virtual image at distance d2), embodiments consistent with the present invention only require waveguides sufficient to transmit and depict the number of optical wavelength components associated with one image pixel, and these waveguides are used by all virtual planes. The benefit of reducing the number of waveguide layers is that it makes the XR device cheaper to manufacture and lighter in weight, where there is only one optical interface to the waveguides.

[0054] Therefore, the remote device 300 also includes an image guide 307, which is configured to guide different images to pass through different optical paths within the remote device 300 before the different images reach the first output port 309 of the remote device 300. For example, if the first image is to appear at a first distance d1 from the user's viewpoint and the second image is to appear at a second distance d2 from the user's viewpoint, the image guide 307 guides the first image to pass through a first optical path to the first output port 309 of the remote device 300, and guides the second image to pass through a second optical path to the first output port 309 of the remote device, wherein the first optical path corresponds to the first distance d1 and the second optical path corresponds to the second distance d2, wherein the first optical path and the second optical path have different lengths corresponding to the difference d3 between the distances d2 and d1.

[0055] Will refer to Figure 3 , Figure 4 and Figure 5 This dynamic modification of the optical path within the remote device 300 is further described. In this example, the first image and the second image will be depicted at the corresponding first virtual plane and the second virtual plane. However, the depiction of only two virtual planes is only for illustrative purposes and should not be interpreted as limiting. On the contrary, the principles on which the present technology is based are not limited to only two virtual planes, and more than two optical paths can be constructed in the remote device 300, which will achieve more than two virtual planes.

[0056] One aspect of some embodiments consistent with the present invention relies on time multiplexing, such as Figure 5 For simplicity, this example only deals with monoscopic rendering. Extensions to the principles to cover stereoscopic rendering of images are discussed later in this document.

[0057] As shown, during a first time period, the image director 307 receives (step 501) a first image M11 that will be perceived as being a first distance d1 from the user's viewpoint. Therefore, the image director 307 directs (step 503) the first image M11 to a first optical image path corresponding to the distance d1. The first image M11 appears at the end of the first optical image path and is supplied to the first output port 309 of the remote device 300, from which it is then transmitted to the see-through waveguide 303 for viewing by the user (step 505).

[0058] Next, after time Δt, during a second time period, the image director 307 receives (step 507) a second image M21 that will be perceived as being a second distance d2 from the user's viewpoint. Therefore, the image director 307 directs (step 509) the second image M21 to a second optical image path corresponding to the distance d2. The second image M21 appears at the end of the second optical image path and is supplied to the first output port 309 of the remote device 300, from which it is then transmitted to the see-through waveguide 303 for viewing by the user (step 511).

[0059] It will be appreciated that the display of the first image M11 during the first time period followed by the display of the second image M21 during the second time period occurs sufficiently rapidly that the user does not perceive a switch but rather perceives the two images as being displayed simultaneously. This corresponds to one frame of the multiplane image.

[0060] This process is repeated for subsequent frames as well. To illustrate this, Figure 5 It is also shown that during the next first time period, the image director 307 receives (step 513) a next first image M12 that will be perceived as being a first distance d1 from the user's viewpoint. Therefore, the image director 307 directs (step 515) the next first image M12 to the first optical image path corresponding to the distance d1. The next first image M12 appears at the end of the first optical image path and is supplied to the first output port 309 of the remote device 300, from which the next first image M12 is then transmitted to the see-through waveguide 303 for viewing by the user (step 517).

[0061] Next, after another time Δt, during the next second time period, the image director 307 receives (step 519) a next second image M22 that will be perceived as being a second distance d2 from the user's viewpoint. Therefore, the image director 307 directs (step 521) the next second image M22 to the second optical image path corresponding to the distance d2. The next second image M22 appears at the end of the second optical image path and is supplied to the first output port 309 of the remote device 300, from which the next second image M22 is then transmitted to the see-through waveguide 303 for viewing by the user (step 523).

[0062] In some but not necessarily all embodiments consistent with the invention, the image director 307 uses a controller to dynamically change the position of one or more electromechanical mirror arrays so that the image projected by the image generator passes through one of a plurality of possible optical paths within the remote device 300. The electromechanical mirror array may, for example, be a micro-electromechanical mirror (MEMS) as is known in the art.

[0063] To further illustrate this point, refer again to Figure 3 , which shows an image director 307 comprising a first mirror 311 and a second mirror 313 as the electromechanical mirrors just mentioned. Figure 3 The arrangement is depicted at a first time period t1. The controller 315 causes the first mirror 311 and the second mirror 313 to position themselves so that the image generated by the image generator 301 will flow through the lens package 317 (for image focusing) directly to the first output port 309. The distance from the image generator 301 to the first output port 309 corresponds to the first optical path d1.

[0064] Figure 4 The state of the remote device 300 at a later time period t2 is depicted. Note: To avoid cluttering the figure, depiction of the box representing the image director 307 has been omitted. However, it will be understood that in this and subsequent similar figures, the depicted controller, mirror and lens package are integral parts of the corresponding image director.

[0065] Accordingly, the controller 315 causes the first mirror 311 and the second mirror 313 to reposition themselves so that the image generated by the image generator 301 will flow through the second lens package 319 (for image focusing) to the third mirror 321, which deflects the image to the fourth mirror 323, which deflects the image to the second mirror 313, which has been repositioned to deflect the image to the first output port 309. The distance from the image generator 301 to the first mirror 311, then through the second lens package 319 to the third mirror 321, the fourth mirror 323, the second mirror 313, and then to the first output port 309 corresponds to the second optical path d2.

[0066] It will be understood from these figures and the discussion that the same number of optical fibers (and indeed the same actual optical fibers 305) are used to transmit the images for each of the two virtual planes VP1, VP2.

[0067] Now refer to Figures 6 to 16 A class of alternative embodiments is described in which multi-plane images are generated without using time multiplexing. These embodiments rely in part on the use of pixel resolution partitioning. First, refer to Figure 6 , a pixel array 600 formatted as an N×N arrangement of pixels may be used to represent one or more images. Note that this arrangement is for example purposes only, and in other embodiments, the array need not be in the shape of a square.

[0068] Two alternatives will be described, one for monoscopic image generation and the other for stereoscopic image generation. We start with a monoscopic example, e.g. Figure 7As shown, the pixel array 600 is conceptually divided into two partitions, which are respectively represented herein as partitions A and B. In the case where the pixel array 600 is considered to represent an overall image, each of the partitions A and B will be considered to represent a corresponding one of the two components of the overall image. Figure 8 As shown, different images are assigned to each of the components of the overall image (i.e., the partitions of the overall image) in such a way that one eye of the user can receive multiple sets of images intended for display on different virtual planes. Figure 8 A non-limiting arrangement is depicted wherein:

[0069] The image intended to be seen by one eye of the user at a distance d2 from the user's viewpoint will be projected within the first part (zone A) of the overall image

[0070] The image intended to be seen by the same eye of the user at a distance d1 from the user's viewpoint will be projected within the second part (zone B) of the overall image

[0071] Note that for illustration purposes, Figure 8 A horizontal division of the image is depicted. In alternative embodiments, other types of divisions may be used (eg vertical divisions), and in such cases the mirror control (to be described below) is adapted accordingly.

[0072] Reference now Fig. 9 To display these images to the user, the image director 307 receives (step 901) a first image M11 that will be perceived as being at a first distance d1 from the user's viewpoint during a first time period, and also receives (step 903) a second image M21 that will be perceived as being at a second distance d2 from the user's viewpoint during the first time period. Figure 8 As previously shown, the two images are configured as components of an overall image. Thus, the image director 307 directs (step 905) the first image M11 to a first optical image path corresponding to the distance d1, and simultaneously directs (step 907) the second image M21 to a second optical image path corresponding to the distance d2. The first image M11 appears at the end of the first optical image path and is supplied (step 909) to the first output port of the remote device 300, from which the first image M11 is then transmitted to the see-through waveguide 303 for viewing by the user. Simultaneously, the second image M21 appears at the end of the second optical image path and is also supplied (step 909) to the first output port of the remote device 300. Both images are then directed to the see-through waveguide 303 for viewing by the user.

[0073] This process is repeated for subsequent frames as well. To illustrate this, Fig. 9It is also shown that the image director 307 receives (step 911) another first image M12 that will be perceived as a first distance d1 from the user's viewpoint during the second time period, and also receives (step 913) a second image M22 that will be perceived as a second distance d2 from the user's viewpoint during the second time period. Figure 8 As previously shown, the two images are configured as components of the overall image. Accordingly, the image director 307 directs (step 915) the "other" first image M12 to the first optical image path corresponding to the distance d1, and simultaneously directs (step 917) the "other" second image M22 to the second optical image path corresponding to the distance d2. The "other" first image M12 appears at the end of the first optical image path and is supplied (step 919) to the first output port of the remote device 300, from which the first image M12 is then transmitted to the see-through waveguide 303 for viewing by the user. Simultaneously, the "other" second image M22 appears at the end of the second optical image path and is also supplied (step 919) to the first output port of the remote device 300. The two images are then directed to the see-through waveguide 303 for viewing by the user during the second time period.

[0074] In some but not necessarily all embodiments consistent with the present invention, the image director 307 uses a controller to dynamically change the position of a first area portion of the electromechanical mirror array so that a first image that is part of the overall image projected by the image generator passes through one of a plurality of possible optical paths within the remote device 300, and simultaneously change the position of a second area portion of the electromechanical mirror array so that a second image that is part of the overall image projected by the image generator passes through a different optical path among the plurality of possible optical paths within the remote device 300.

[0075] To further illustrate this point, refer to Fig.10, which depicts elements of an image director (not shown to avoid making the figure too cluttered), including, among other things, a controller 1015 and a first mirror 1011 (which is the electromechanical mirror just mentioned). The controller 1015 causes the first mirror 1011 to effectively operate as two mirrors by positioning the first area portion and the second area portion of the first mirror 1011 differently. The first area portion of the first mirror 1011 is in the path of a first component of the overall image generated by the image generator 1001, while the second area portion of the first mirror 1011 is in the path of a second component of the overall image generated by the image generator 1001. The mirror control is performed in such a way that the first component of the overall image generated by the image generator 1001 will be reflected by the first area portion of the mirror 1011 in a direction such that the first component flows through the first lens package 1017 (for image focusing) directly to the first output port 1009. The distance from the image generator 1001 to the first output port 1009 corresponds to the first optical path d1.

[0076] At the same time, the controller 1015 causes the second area portion of the first mirror 1011 to be angled in such a way that a second component of the overall image generated by the image generator 1001 will be reflected by the second area portion of the mirror 1011 in a direction such that the second component is reflected by the second mirror 1021, the third mirror 1023 and the fourth mirror 1025 in sequence and then reaches the first output port 1009 through the second lens package 1027 (for image focusing). The distance from the image generator 1001 to the first mirror 1011, the second mirror 1021, the third mirror 1023 and the fourth mirror 1025 and then reaches the first output port 1009 through the second lens package 1027 corresponds to the second optical path d2.

[0077] It is worth noting that embodiments that rely on pixel resolution division (such as just relative to Figure 7 , Figure 8 and Fig. 9 The embodiment described in the above description) brings up the question of how to deal with occlusion. For example, it can be seen that in Fig.10In the illustrated embodiment, an image associated with a first virtual plane travels to a first output port 1009 via a first optical path d1, and another image associated with a second virtual plane travels to the same first output port 1009 via a second optical path d2. This means that light from two different sources will converge at the first output port 1009. This is essentially a problem that can be handled by the image generator 1001 in any of a number of different ways. For example, if the image depicted at the first virtual plane VP1 is considered to be in front of the image depicted at the second virtual plane VP2, the image generator can be arranged so that the pixels associated with the second virtual plane VP2 are turned off so that the user will only see the image depicted at the first virtual plane VP1. Alternatively, when one image is in front of another image, it may be desirable to allow a certain degree of transparency. Again, this can be handled by the image generator 1001 by, for example, allowing light from two sources to mix at the first output port 1009. Alternatively, in another alternative, the image generator VP1 can adjust the image associated with the first virtual plane VP1 to produce a transparent effect while turning off the pixels associated with the second virtual plane VP2. Those skilled in the art will recognize that other solutions exist.

[0078] In another class of alternative embodiments, the principle of image resolution partitioning is extended to provide multi-plane stereoscopic images. Figures 11 to 18 Several of these embodiments are described.

[0079] In one aspect, Fig.11 As shown, Figure 6 The pixel array 600 is conceptually divided into four quadrants, hereinafter referred to as quadrants A, B, C, and D. In the case where the pixel array 600 is considered to represent an overall image, each of the quadrants A, B, C, and D will be considered to represent a corresponding one of the four components of the overall image. Fig.12 As shown, a different image is assigned to each of the components of the overall image (i.e., a quadrant of the overall image) in such a way that each eye of the user can receive a corresponding left and right image set, wherein each of the left set and the right set itself comprises a set of multiple images intended for display on different virtual planes. For example, Fig.12 A non-limiting arrangement is depicted wherein:

[0080] the image intended to be seen by the left eye at a distance d2 from the user's viewpoint will be projected in the first part of the overall image (quadrant A),

[0081] the image intended to be seen by the left eye at a distance d1 from the user's viewpoint will be projected in the second part of the overall image (quadrant C),

[0082] the image intended to be seen by the right eye at a distance d2 from the user's viewpoint will be projected in the third part of the overall image (quadrant B),

[0083] • The image intended to be seen by the right eye at a distance d1 from the user's viewpoint will be projected in the fourth part (quadrant D) of the overall image.

[0084] Reference now Fig.13 To display these images to the user, in one class of embodiments, the image director 307 receives (step 1301) a first image L1 that will be perceived by the user's left eye as being at a first distance d1 from the user's viewpoint during a first time period, and also receives (step 1303) a second image R1 that will be perceived by the user's right eye as being at a first distance d1 from the user's viewpoint during a first time period. Fig.12 As previously shown in FIG. 1 , the two images are configured as components C and D of the overall image. Accordingly, the image director 307 directs (step 1305) the first image L1 to a first optical image path 1307 corresponding to the distance d1 and leading to the left output port of the device, and simultaneously directs (step 1305) the second image R1 to a second optical image path 1309 also corresponding to the distance d1 but leading to the right output port of the device. The left and right output ports of the device are then directed to corresponding left and right perspective waveguides (not shown) for viewing by the user.

[0085] Next, during a second time period (e.g., occurring at Δt after the first time period), the image director 307 receives (step 1311) a third image L2 that will be perceived by the user's left eye as being a second distance d2 from the user's viewpoint, and also receives (step 1313) a fourth image R2 that will be perceived by the user's right eye as being a second distance d2 from the user's viewpoint during the second time period. Fig.12 As previously shown in FIG. 1 , the two images are configured as components A and B of the overall image. Accordingly, the image director 307 directs (step 1315) the third image L2 to a second optical image path corresponding to the distance d2 and leading to the left output port 1307 of the device, and simultaneously directs (step 1315) the fourth image R2 to a second optical image path also corresponding to the distance d2 but leading to the right output port 1309 of the device. As previously, the left output port 1307 and the right output port 1309 of the device are then directed to corresponding left and right see-through waveguides (not shown) for viewing by the user.

[0086] It will be appreciated that the display of the first image L1 and the second image R1 during the first time period followed by the display of the third image L2 and the fourth image R2 during the second time period occurs sufficiently quickly that the user does not perceive a switch but rather perceives the four images as being displayed simultaneously, two images at the first virtual plane VP1 and the other two images at the second virtual plane VP2. This corresponds to one frame of a stereoscopic three-dimensional image.

[0087] Now refer to Fig.14 and Fig.15 Description and References Fig.13 The discussed aspects correspond to the exemplary embodiments. Fig.14 , shows the configuration of the remote device 1400 at the first time period t1. The image generator 1401 projects the first image L1 onto the first area portion of the first electromechanical mirror 1403, and projects the second image R1 onto the second area portion of the first electromechanical mirror 1403. The controller 1405 causes the first electromechanical mirror 1403 to act in a split manner so that the first area portion of the electromechanical mirror 403 redirects the first image L1 through the first lens package 1407 to the first output port 1409, from which the first image L1 proceeds through the optical waveguide to the see-through waveguide for viewing by the user's left eye; and simultaneously causes the second area portion of the electromechanical mirror 1403 to redirect the second image R1 through the first lens package 1407 to the second output port 1411, from which the second image R1 proceeds through the optical waveguide to the see-through waveguide for viewing by the user's right eye. The distance covered by the first image L1 and the second image R1 from the image generator 1401 to the respective first output port 1409 and second output port 1411 corresponds to the first distance d1. Fig.12 As shown previously, these two images are configured as components C and D of an overall image.

[0088] Fig.15The configuration of the remote device 1400 at the second time period t2 is depicted. The image generator 1401 projects the third image L2 onto the third area portion of the first electromechanical mirror 1403, and projects the fourth image R2 onto the fourth area portion of the first electromechanical mirror 1403. The controller 1405 causes the third area portion and the fourth area portion of the first electromechanical mirror 1403 to operate in a split manner so that the third area portion and the fourth area portion of the first electromechanical mirror 1403 redirect the third image L2 and the fourth image R2 to the second mirror 1507, the second mirror 1507 redirects the image to the third mirror 1509, and the third mirror 1509 then redirects the image to the fourth mirror 1511, which is also an electromechanical mirror. The controller 1405 causes the fourth mirror 1511 to operate in a split manner so that the first area portion of the fourth mirror 1511 redirects the third image L2 through the second lens package 1513 to the first output port 1409, and the third image L2 advances from the first output port 1409 through the optical waveguide to the perspective waveguide for viewing by the user's left eye; and at the same time causes the second area portion of the fourth mirror 1511 to redirect the fourth image R2 through the second lens package 1513 to the second output port 1411, and the fourth image R2 advances from the second output port 1411 through the optical waveguide to the perspective waveguide for viewing by the user's right eye. The distance covered by the third image L2 and the fourth image R2 from the image generator 1401 through the first mirror 1403, the second mirror 1507, the third mirror 1509 and the fourth mirror 1511 and then through the second lens package 1513 to the corresponding first output port 1409 and the second output port 1411 corresponds to the second distance d2. Fig.12 As shown previously, the two images are configured as components A and B of an overall image.

[0089] It should be understood that the embodiments just described combine pixel resolution division with time multiplexing to present a stereoscopic multi-plane image to a user. In these embodiments, the image is presented to the left eye and the right eye of the user simultaneously, wherein time multiplexing is used to alternately display images associated with the first virtual plane VP1 and the second virtual plane VP2.

[0090] In other alternative embodiments, pixel resolution division and time multiplexing are again used, but this time for simultaneously presenting images associated with the first virtual plane VP1 and the second virtual plane VP2 to one eye of the user during a first time period t1, and then simultaneously presenting images associated with the first virtual plane VP1 and the second virtual plane VP2 to the other eye of the user during a second time period t2. Fig.16 and Fig.17 The illustrated exemplary embodiments illustrate these embodiments.

[0091] Fig.16Elements of an image director are depicted (not shown to avoid making the figure too cluttered), including, among others, a controller 1615 and a first mirror 1603 (which is the electromechanical mirror mentioned earlier). During a first time period t1, the controller 1615 causes the first mirror 1603 to effectively operate as two mirrors by placing the first area portion and the second area portion of the first mirror 1603 differently. The first area portion of the first mirror 1603 is in the path of a first component of the overall image generated by the image generator 1601, while the second area portion of the first mirror 1603 is in the path of a second component of the overall image generated by the image generator 1601. The mirror control is performed in such a way that the first component of the overall image generated by the image generator 1601 will be reflected by the first area portion of the mirror 1603 towards a direction that causes the first component to flow through the first lens package 1605 (for image focusing) directly to the first output port 1607. When reflected by the first mirror 1603 in this manner, the distance from the image generator 1601 to the first output port 1607 corresponds to the first optical path d1.

[0092] At the same time, still during the first time period t1, the controller 1615 causes the second area portion of the first mirror 1603 to be angled in such a way that the second component of the overall image generated by the image generator 1601 will be reflected by the second area portion of the mirror 1603 in a direction such that the second component is reflected by the second mirror 1609, the third mirror 1611 and the fourth mirror 1613 in sequence and then reaches the first output port 1607 through the second lens package 1617 (for image focusing). The distance from the image generator 1601 to the first mirror 1603, the second mirror 1609, the third mirror 1611 and the fourth mirror 1613 and then through the second lens package 1617 to the first output port 1607 corresponds to the second optical path d2. In this example, the first output port 1607 is attached to the fiber package 1619, which guides the two virtual plane images to the see-through waveguide for viewing by the user's right eye.

[0093] Next, a virtual plane image is generated for the user's left eye to view. Specifically, Fig.17 Depicted is the configuration during the next time period t2 when the controller 1615 causes the first mirror 1603 to effectively operate as two mirrors by differently placing the first area portion and the second area portion of the first mirror 1603. The first area portion of the first mirror 1603 is in the path of a first component of the overall image generated by the image generator 1601, and the second area portion of the first mirror 1603 is in the path of a second component of the overall image generated by the image generator 1601. The mirror control is different from before (i.e., with Fig.16), because now the mirror control is performed in such a way that the first component of the overall image generated by the image generator 1601 will be reflected by the first area portion of the mirror 1603 in a direction such that the first component flows through the first lens package 1605 (for image focusing) directly to the second output port 1621. When reflected by the first mirror 1603 in this way, the distance from the image generator 1601 to the second output port 1621 corresponds to the first optical path d1.

[0094] At the same time, still during the second time period t2, the controller 1615 causes the second area portion of the first mirror 1603 to be angled in such a way that the second component of the overall image generated by the image generator 1601 will be reflected by the second area portion of the mirror 1603 in a direction such that the second component is reflected by the second mirror 1609, the third mirror 1611 and the fourth mirror 1613 in sequence and then reaches the second output port 1621 through the second lens package 1617 (for image focusing). The distance from the image generator 1601 to the first mirror 1603, the second mirror 1609, the third mirror 1611 and the fourth mirror 1613 and then through the second lens package 1617 to the second output port 1621 corresponds to the second optical path d2. In this example, the second output port 1621 is attached to the second fiber package 1623, which guides the two virtual plane images to the see-through waveguide for viewing by the user's left eye.

[0095] In yet another alternative embodiment, pixel resolution division is used without temporal multiplexing to produce a stereoscopic multi-plane image for viewing by a user. Here, as before, in some of the other embodiments, a different image is assigned to each of the components of the overall image (i.e., quadrants of the overall image) in a manner that enables each eye of the user to receive a corresponding set of left and right images, wherein each of the left and right sets itself includes a set of multiple images intended for display on different virtual planes. Again referring to Fig.12 To illustrate this aspect.

[0096] Reference now Fig.18 In such embodiments, the image generator generates an image including Fig.12The image guide generates an overall image of the four components shown and projects these components (step 1801) to an image guide. The image guide (e.g., by controlling portions of a plurality of electromechanical mirror arrays) causes the first virtual plane images L1 and R1 to travel a first distance d1 to one of the two output ports (step 1803), and simultaneously causes the second virtual plane images L2 and R2 to travel a second distance d2 to the other of the two output ports (step 1805). In addition, the image guide sorts the four components of the overall image so that two components to be viewed by one eye of the user are directed to the first output port, while the other two components (which are to be viewed by the other eye of the user) are directed to the second output port (step 1807). The two output ports supply the image pairs to corresponding optical fibers, which guide the image pairs to the see-through waveguides for viewing by the left and right eyes of the user (step 1809).

[0097] Fig.19A , Fig.19B and Fig.19C Aspects of the technology related to dynamic adjustment of distances associated with virtual planes are described. Thus far, the embodiments have only described fixed virtual plane distances, such as distances d1 and d2. However, other aspects of the technology allow these distances to be dynamically adjustable. First, refer to Fig.19A , showing a configuration for displaying a first virtual plane. The image generator 1901 projects an image onto a first autofocus lens 1903. The image emerges from the first autofocus lens 1903, passes through a lens package 1905, and then goes to a second autofocus lens 1907. The image emerging from the second autofocus lens 1907 is focused onto a fiber package 1909, which directs the image to a user head mounted display, as described in the previous embodiment.

[0098] The size of the image can be varied to create the impression that the depicted image is at different distances from the user. To achieve this, Fig.19B As shown, the first autofocus lens 1903 is moved a certain amount (dx) to reach a different position (step 1911). (Note that the arrangement may include additional lenses within the lens package that also need to be moved to make the virtual distance appear to be the desired position - these additional lenses are not shown in the figure to avoid obscuring the discussion with unnecessary details known to those skilled in the art.)

[0099] In addition to changing the size of the image, moving the first autofocus lens 1903 also has the effect of blurring the image at the point where the image is projected onto the fiber package 1909. Fig.19B Shown in FIG. 1 is focus 1913 , which is the undesired volume (dz) inside the fiber package 1909 .

[0100] To compensate for this undesirable blur, the second autofocus lens 1907 is also moved a certain amount (dy) (step 1915) to a new position that re-establishes a sharp image at the entrance 1917 to the fiber package 1909.

[0101] To further promote understanding of various aspects of the present technology, in some aspects, Fig. 20 is a flow chart 2000 of actions associated with a remote device according to a plurality of the above-described embodiments utilizing time division multiplexing. In other aspects, Fig. 20 The blocks depicted in may also represent means (eg, processing means) for performing the actions described.

[0102] Initially, an image generator projects virtual plane images onto an electromechanical mirror array in a time sequence (step 2001). The mirrors are then controlled to sequentially move to different positions that reflect each sequentially presented image onto different paths (step 2003). Thus, for some arbitrary number (x) of virtual planes to be displayed to a user, the first image is reflected onto a first optical path associated with the first virtual plane (step 2005), the second image is reflected onto a second optical path associated with the second virtual plane (step 2007), and so on, until the x-th image is reflected onto the x-th optical path associated with the x-th virtual plane (step 2009).

[0103] At the end of each optical path, an image is projected onto a fiber package (step 2011). The optical fiber then directs the image onto a set of see-through waveguides configured to display a number of optical wavelength components (e.g., three in the case of an RGB image) (step 2013). A user can view the image by looking through the see-through waveguides.

[0104] To further promote understanding of other aspects of this technology, in some aspects, Fig.21 is a flowchart 2100 of actions associated with a remote device according to a plurality of the above-described embodiments utilizing pixel resolution partitioning. In other aspects, Fig.21 The blocks depicted in the may also be considered to represent means (eg, processing means) for performing the described actions.

[0105] Initially, the display generator projects (step 2101) an overall image onto an electromechanical mirror (eg, MEMS), wherein the overall image includes a corresponding component for each virtual plane to be displayed.

[0106] Different area portions of the electromechanical mirror are controlled simultaneously so that for a certain number of virtual planes (x): the first area portion of the electromechanical mirror reflects the first component onto a corresponding first path for the first virtual plane (step 2103), the second area portion of the electromechanical mirror reflects the second component onto a corresponding second path for the second virtual plane (step 2105), and so on, until the x-th area portion of the electromechanical mirror reflects the x-th component onto a corresponding x-th path for the x-th virtual plane (step 2107).

[0107] At the end of each optical path, the image is projected onto a fiber package (step 2109). The optical fiber then directs the image onto a set of see-through waveguides configured to display a number of optical wavelength components (e.g., three in the case of an RGB image) (step 2111). A user is able to view the image by looking through the see-through waveguides.

[0108] Fig. 22 is a block diagram of an apparatus 2200 according to some but not necessarily all embodiments consistent with the present invention. The apparatus 2200 includes a remote device 2201 coupled to respective left and right holographic waveguides 2203, 2205 via respective fiber optic bundles 2207, 2209.

[0109] The images are generated by a host system 2211 in the remote device 2201. The host system 2211 includes a memory 2213 coupled to a video processing unit 2215. The video processing unit 2215 generates images using data and instructions from the memory 2213, which are supplied to an interface 2217, which transmits the generated images to a display system 2219.

[0110] The display system 2219 may be configured according to the principles disclosed for any of the image directors previously described in this document.As described above, the images generated by the display system 2219 for the user's left and right eyes are transmitted to the holographic waveguides 2203, 2205 via the optical fiber sets 2207, 2209.

[0111] Aspects of the exemplary controller discussed in various embodiments are described in Fig.23 , which illustrates an exemplary controller 2301 of a remote device according to some but not necessarily all exemplary embodiments consistent with the present invention. Specifically, the controller 2301 includes circuitry configured to perform any one or any combination of the various functions described above. For example, such circuitry may be a fully hardwired circuit (e.g., one or more application specific integrated circuits - "ASICs"). However, in Fig.23In an exemplary embodiment of the invention, a programmable circuit is depicted, which includes a processor 2303, the processor 2303 is coupled to one or more memory devices 2305 (e.g., random access memory, disk drive, optical drive, read-only memory, etc.) and to an interface 2307 that enables bidirectional communication with other elements of a remote device (e.g., a control interface for one or more electromechanical mirrors). The memory device 2305 stores a program device 2309 (e.g., a set of processor instructions) that is configured to cause the processor 2303 to control other remote device elements in order to perform any of the above aspects. The memory device 2305 may also store data (not shown) representing various constant and variable parameters that may be required by the processor 2303 and / or may be generated when performing the functions of the processor 2303 (such as the functions specified by the program device 2309).

[0112] Various embodiments provide a number of advantages over prior art. For example, aspects of the exemplary embodiments enable the production of XR glasses that are lighter in weight than those known in the prior art. Furthermore, by separating the heating element from the XR glasses, the glasses become more comfortable for the user and have a smaller form factor. Furthermore, since fewer constraints need to be imposed on a wearable XR computing device placed in a pocket or worn on a belt, more power and computing resources can be added to the overall system without burdening the user.

[0113] The present invention has been described with reference to specific embodiments. However, it will be apparent to those skilled in the art that the present invention may be implemented in specific forms different from those of the embodiments described above. Therefore, the described embodiments are merely illustrative and should not be considered restrictive in any way. The scope of the present invention is further described by the appended claims, rather than merely by the preceding description, and all variations and equivalents falling within the scope of the claims are intended to be included within the scope of the present invention.

Claims

1. A device for supplying a multi-plane image for viewing by a user, the device comprising: an image generator that generates a first image to be viewed by the user as a first distance from a user's viewpoint and a second image to be viewed by the user as a second distance from the user's viewpoint, wherein the first image includes a number of optical wavelength components and the second image includes the number of optical wavelength components; an image director configured to direct the first image through a first optical path to a first output port of the device, and to direct the second image through a second optical path to the first output port of the device, wherein the first optical path corresponds to the first distance and the second optical path corresponds to the second distance, and wherein the first optical path and the second optical path have different lengths; the first output port configured to connect to a first optical waveguide configured to direct the number of optical wavelength components to a user display device; Wherein, the image director comprises: a first electromechanical mirror array located in a path of the first image and the second image supplied by the image generator; and A controller is configured to cause the first electromechanical mirror array to direct the first image to the first optical path, and to cause the first electromechanical mirror array to direct the second image to the second optical path.

2. The apparatus according to claim 1, further comprising: the first optical waveguide coupled at a first end to the first output port; as well as A see-through optical waveguide is coupled to the second end of the first optical waveguide.

3. The device according to claim 1, wherein: The image generator time-multiplexes generation of the first image and the second image; When the image generator is generating the first image, the controller causes the first electromechanical mirror array to direct the first image to the first optical path; as well as When the image generator is generating the second image, the controller causes the first electromechanical mirror array to direct the second image to the second optical path.

4. The device according to claim 1, wherein: The image generator simultaneously generates the first image and the second image as components of an overall image, wherein the first image is located within a first portion of the overall image and the second image is located within a second portion of the overall image; The first electromechanical mirror array has a first area portion and a second area portion; The first electromechanical mirror array is arranged in the device so that the first area portion of the first electromechanical mirror array is in the path of the first image output by the image generator; The first electromechanical mirror array is arranged in the device so that the second area portion of the first electromechanical mirror array is in the path of the second image output by the image generator; The controller causes the first area portion of the first electromechanical mirror array to direct the first image to the first optical path; and The controller causes the second region portion of the first electromechanical mirror array to direct the second image to the second optical path.

5. The device according to claim 4, wherein: The first image and the second image are configured for viewing by a first eye of the user; the image generator generating a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of optical wavelength components and the second image includes the number of optical wavelength components, and wherein the third image and the fourth image are configured for viewing by a second eye of the user; The image generator simultaneously generates the first image, the second image, the third image and the fourth image as components of the overall image, wherein the third image is located in a third portion of the overall image and the fourth image is located in a fourth portion of the overall image; The first electromechanical mirror array has a third area portion and a fourth area portion; the first electromechanical mirror array being arranged within the apparatus so that the third region portion of the first electromechanical mirror array is in the path of the third image output by the image generator; the first electromechanical mirror array being arranged within the apparatus so that the fourth region portion of the first electromechanical mirror array is in the path of the fourth image output by the image generator; The controller causes the third area portion of the first electromechanical mirror array to direct the third image to a third optical path corresponding to the first distance; and The controller causes the fourth region portion of the first electromechanical mirror array to guide the fourth image to a fourth optical path corresponding to the second distance, in: the first electromechanical mirror array being configured to direct the first image to the first output port via the first optical path and to direct the third image to a second output port via the third optical path, wherein the second output port is configured to connect to a second optical waveguide configured to direct the number of optical wavelength components to the user display device; and The apparatus also includes a second electromechanical mirror array configured to direct the second image to the first output port via the second optical path and to direct the fourth image to the second output port via the fourth optical path.

6. The device according to claim 1 or claim 2, wherein: The first image and the second image are configured for viewing by a first eye of the user; the image generator generating a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components, the fourth image includes the number of light wavelength components, and wherein the third image and the fourth image are configured for viewing by a second eye of the user; The image director receives the first image and the second image from the image generator during a first time period; the image director receiving the third image and the fourth image from the image generator during a second time period; the image director being configured to direct the first image to the first output port via the first optical path during the first time period; the image director being configured to direct the second image to the first output port via the second optical path during the first time period; the image director being configured to direct the third image to a second output port via a third optical path during the second time period, wherein the third optical path corresponds to the first distance; and The image director is configured to direct the fourth image to the second output port via a fourth optical path during the second time period, wherein the fourth optical path corresponds to the second distance.

7. The device according to claim 6, wherein: The first time period includes a third time period and a fourth time period; The second time period includes a fifth time period and a sixth time period; When it is the third time period, the image guide guides the first image to the first output port via the first optical path; When it is the fourth time period, the image guide causes the second image to be guided to the first output port via the second optical path; When it is the fifth time period, the image guide causes the third image to be guided to the second output port via the third optical path; as well as During the sixth time period, the image guide guides the fourth image to the second output port via the fourth optical path.

8. The device according to claim 6, wherein: The image director comprises a second electromechanical mirror array, which is arranged in the second optical path and the fourth optical path and is configured as follows: directing the second image to the first output port during the first time period; The fourth image is directed to the second output port during the second time period.

9. The device according to claim 1 or claim 2, wherein: The first image and the second image are configured for viewing by a first eye of the user; the image generator generating a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components, the fourth image includes the number of light wavelength components, and wherein the third image and the fourth image are configured for viewing by a second eye of the user; The image director receives the first image and the third image from the image generator during a first time period; the image director receiving the second image and the fourth image from the image generator during a second time period; the image director being configured to direct the first image to the first output port via the first optical path during the first time period; the image director being configured to direct the third image to a second output port via a third optical path during the first time period, wherein the third optical path corresponds to the first distance; the image director being configured to direct the second image to the first output port via the second optical path during the second time period; and The image director is configured to direct the fourth image to the second output port via a fourth optical path during the second time period, wherein the fourth optical path corresponds to the second distance.

10. The device according to claim 9, wherein: The first time period includes a third time period and a fourth time period; The second time period includes a fifth time period and a sixth time period; When it is the third time period, the image guide guides the first image to the first output port via the first optical path; When it is the fourth time period, the image guide causes the third image to be guided to the second output port via the third optical path; When it is the fifth time period, the image guide guides the second image to the first output port via the second optical path; as well as During the sixth time period, the image guide guides the fourth image to the second output port via the fourth optical path.

11. The device according to claim 9, wherein: The image director comprises a second electromechanical mirror array, which is arranged in the second optical path and the fourth optical path and is configured as follows: directing the second image to the first output port during the first time period; The fourth image is directed to the second output port during the second time period.

12. The device according to claim 1 or claim 2, wherein: The device also includes: An autofocus lens device is dynamically controlled by the controller so that the first distance has a first optical length when a first control signal is generated and has a second optical length when a second control signal is generated, wherein the first optical length and the second optical length are different.

13. A method for supplying a multi-plane image for viewing by a user, the method comprising: generating a first image to be viewed by the user as a first distance from a user's viewpoint and a second image to be viewed by the user as a second distance from the user's viewpoint, wherein the first image includes a number of optical wavelength components and the second image includes the number of optical wavelength components; directing the first image through a first optical path to a first output port of a device, and directing the second image through a second optical path to the first output port of the device, wherein the first optical path corresponds to the first distance and the second optical path corresponds to the second distance, and wherein the first optical path and the second optical path have different lengths, wherein the first output port is configured to be connected to a first optical waveguide, the first optical waveguide being configured to direct the number of optical wavelength components to a user display device; Wherein, directing the first image through the first optical path to the first output port of the device and directing the second image through the second optical path to the first output port of the device comprises: causing the first electromechanical mirror array to direct the first image to the first optical path, and causing the first electromechanical mirror array to direct the second image to the second optical path, Wherein, the first electromechanical mirror array is located in the path of the first image and the second image supplied by the image generator.

14. The method according to claim 13, wherein: Generating the first image and the second image includes: time-multiplexing the generation of the first image and the second image; When the image generator is generating the first image, causing the first electromechanical mirror array to direct the first image to the first optical path; and When the image generator is generating the second image, the first electromechanical mirror array is caused to direct the second image to the second optical path.

15. The method according to claim 13, wherein: Generating the first image and the second image includes: Simultaneously generating the first image and the second image as components of an overall image, wherein the first image is located within a first portion of the overall image and the second image is located within a second portion of the overall image; causing a first area portion of the first electromechanical mirror array to guide the first image to the first optical path; and The second area portion of the first electromechanical mirror array guides the second image to the second optical path.

16. The method of claim 15, wherein: The first image and the second image are configured for viewing by a first eye of the user; and The method comprises: generating a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of optical wavelength components and the second image includes the number of optical wavelength components, and wherein the third image and the fourth image are configured for viewing by a second eye of the user; Simultaneously generating the first image, the second image, the third image, and the fourth image as components of the overall image, wherein the third image is located within a third portion of the overall image, and the fourth image is located within a fourth portion of the overall image; causing a third area portion of the first electromechanical mirror array to guide the third image to a third optical path corresponding to the first distance; and The fourth area portion of the first electromechanical mirror array guides the fourth image to a fourth optical path corresponding to the second distance.

17. The method of claim 13, wherein: The first image and the second image are configured for viewing by a first eye of the user; and The method comprises: generating a third image to be viewed by the user as the first distance from the user's viewpoint and a fourth image to be viewed by the user as the second distance from the user's viewpoint, wherein the third image includes the number of light wavelength components, the fourth image includes the number of light wavelength components, and wherein the third image and the fourth image are configured for viewing by a second eye of the user; receiving the first image and the third image during a first time period; receiving the second image and the fourth image during a second time period; directing the first image to the first output port via the first optical path during the first time period; directing the third image to a second output port via a third optical path during the first time period, wherein the third optical path corresponds to the first distance; directing the second image to the first output port via the second optical path during the second time period; and The fourth image is directed to the second output port via a fourth optical path during the second time period, wherein the fourth optical path corresponds to the second distance.

18. The method according to any one of claims 13 to 17, further comprising: The autofocus lens arrangement is dynamically controlled so that the first distance has a first optical length when a first control signal is generated and has a second optical length when a second control signal is generated, wherein the first optical length and the second optical length are different.

Images