Display apparatus for displaying hologram and method thereof

Abstract

Techniques are provided for displaying holograms in wearable display devices. An image source projects a holographic image sequentially in the three primary colors into a waveguide, where the image is amplitude and phase modulated in a spatial light modulator (SLM). Depending on the implementation, the image source can be next to one end of the waveguide or an optical fiber is used to deliver the holographic image near the waveguide to be projected into the waveguide.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to the field of display devices, and more particularly to the architecture and design of display devices fabricated in the form of a pair of eyeglasses that can be used in a variety of applications including virtual reality and augmented reality. In particular, the present invention employs amplitude modulation (AM) and phase modulation (PM) to implement holographic images for display, for example, in a wearable display device. BACKGROUND

[0002] Virtual reality, or VR, is generally defined as a realistic and immersive simulation of a three-dimensional environment created using interactive software and hardware that is experienced or controlled by a subject's movements. A person using a virtual reality device is typically able to look around an artificially generated three-dimensional environment, walk through it, and interact with features or objects depicted on a screen or in goggles. Virtual reality artificially creates sensory experiences that can include visual, tactile, auditory, and less commonly, olfactory.

[0003] Augmented reality (AR) is a technology that adds computer-generated enhancements to a real-world environment in order to make it more meaningful through the ability to interact with it. AR is being developed into applications and used on mobile devices to blend digital components into the real world so that they enhance each other but can also be easily distinguished. AR technology is quickly becoming mainstream. It is used to display score overlays about televised sporting events and pop-up 3D e-mails, photos, or text messages on mobile devices. Leaders in this technology industry are also using AR to do exciting and revolutionary things with holograms and motion-activated commands.

[0004] The delivery methods of virtual reality and augmented reality are different when viewed separately. Most virtual reality in 2016 is displayed on computer screens, projector screens, or through virtual reality headsets (also known as head-mounted displays or HMDs). HMDs are typically in the form of head-mounted goggles with screens in front of the eyes. Virtual reality actually immerses the user in a digital world by cutting off external stimuli. In this way, the user is only focused on the digital content being displayed in the HMD. Augmented reality is increasingly used in mobile devices, such as laptops, smartphones, and tablets, to change the way the real world and digital images, graphics, cross, and interact.

[0005] In fact, VR is not always opposed to AR because they are not always operated independently of each other, but rather are often mixed together to produce a more immersive experience. For example, haptic feedback as an addition to the vibration and sensation of a graphics interaction is considered an augmentation. However, it is often used within a virtual reality scenario in order to make the experience more realistic through haptics.

[0006] Virtual reality and augmented reality are prominent examples of experiences and interactions that are driven by the desire to become immersed in a simulated platform for entertainment and gaming or to add new dimensions to the interaction between digital devices and the real world. Without a doubt, they open both the real and virtual worlds, alone or mixed together.

[0007] FIG. 1A shows an exemplary pair of goggles for delivering or displaying VR or AR applications that are common in the market today. Regardless of the design of the goggles, they appear bulky and cumbersome and create an inconvenience when worn by a user. In addition, most of the goggles are not see-through. In other words, when a user wears the goggles, he or she will not be able to see or do anything else. Therefore, there is a need for a device that can display VR and AR and also allow a user to perform other tasks when needed.

[0008] Various wearable devices are being developed for VR / AR and holographic applications. FIG. IB shows a simplified diagram of the HoloLens from Microsoft. It weighs 579 g (1.2 lbs) at which weight a wearer will feel uncomfortable after wearing it for a while. In fact, the products available in the market are typically bulky and voluminous compared to normal glasses (25 g - 100 g). There are reports that the wearable device based on the HoloLens from Microsoft will be delivered to the US army. If it is indeed equipped to the soldiers, the weight of the wearable device can greatly affect the movement of the soldiers, especially when they need to move quickly on the battlefield. Therefore, there is also a need for a wearable AR / VR viewing or display device that looks similar to a pair of normal glasses but also allows for a smaller footprint, enhanced impact performance, low cost packaging, and easier manufacturing process.

[0009] Wearable display devices provide a relatively ideal viewing environment for viewing holograms because they are often placed near the eyes and can block a considerable amount of ambient light. Therefore, there is also a need for a solution to generate holograms for projection onto a medium, such as a transparent lens, in a wearable display device.

[0010] Many eyeglass display devices use a common design that places the image forming components (e.g., LCOS) in front or near the lens frame, hoping to reduce image transmission loss and use fewer components. However, such designs often make the eyeglass display device unbalanced, with the front portion of the eyeglass display device much heavier than the back portion, thereby adding some pressure on the nose. Therefore, there is also a need to distribute the weight of such display devices when they are worn by a user.

[0011] Regardless of how the wearable display device is designed, there are still many components, wires and even batteries that must be used to make the display device function and operable. Although much effort has been made to move as many parts as possible to an attachable device or housing to drive the display device from the user's waist or pocket, necessary parts such as copper wires must be used to transmit various control signals and image data. The weight of the conductive wires, which are typically in the form of a cable, does add to the pressure on the wearer when such a display device is worn by the wearer. Therefore, there is still a need for a transmission medium that is as light as possible without sacrificing the required functionality.

[0012] There are many other needs that are not individually listed, but one or more embodiments of the application described in detail herein clearly satisfy these needs, as can be readily understood by one of skill in the art. SUMMARY

[0013] This section is intended to provide a brief overview of some aspects of the application and to introduce some preferred embodiments. To the extent that this section, the summary and the abstract can contain simplifications, omissions, and / or oversights, such simplifications, omissions, and / or oversights are not intended to limit the scope of the application, which is set forth by the claims.

[0014] The present application generally relates to the architecture and design of wearable devices that can be used for virtual reality and augmented reality applications. According to one aspect of the present application, a display device is made in the form of a pair of glasses and includes a minimum number of parts to reduce its complexity and weight. A separate casing or housing is provided to be portable to be attached or affixed to the user (e.g., a pocket or a belt). The housing includes all the necessary parts and circuitry to generate the content for virtual reality and augmented reality applications, thereby generating a minimum number of parts needed on the glasses, thus making the glasses less bulky, enhanced impact performance, lower packaging cost and easier manufacturing process. The content is optically picked up by an optical cable and transported through the optical fibers in the cable to the glasses, where the content is projected to specially made lenses for displaying the content in front of the wearer's eyes, respectively.

[0015] According to another aspect of the present application, the glasses do not include electronic components and are coupled to a housing by a transmission line that includes one or more optical fibers (the singular or plural number of optical fibers can be used interchangeably hereafter), where the optical fibers are responsible for transporting the content or optical images from one end of the optical fibers to the other end by total internal reflection within the optical fibers. The optical images are picked up by focusing lenses from a miniature display in the housing.

[0016] According to yet another aspect of the present application, the optical image is lower resolution in the optical fiber but it is accepted at twice the normal refresh rate (e.g. 120 Hz vs. 60 Hz). Wherein two frames of the lower resolution image are in succession combined at the other end of the optical fiber to generate a higher resolution image, the combined image is refreshed at the normal refresh rate.

[0017] According to yet another aspect of the present application, each lens comprises a prism in the form of a prism that propagates an optical image projected on one edge of the prism to an optical path where a user can see an image formed from the optical image. The prism is also integrated with or stacked on an optical corrective lens that is complementary or reciprocal to the lens of the prism to form the integrated lenses of the eyewear. The optical corrective lens is provided to correct the optical path from the prism to allow the user to view through the integrated lenses without optical distortion.

[0018] According to yet another aspect of the present application, an exemplary prism is a waveguide. Each lens in the integrated lenses comprises an optical waveguide that propagates an optical image projected on one end of the waveguide to the other end through an optical path where a user can see an image formed from the optical image. The waveguide can also be integrated with or stacked on an optical corrective lens to form the integrated lenses of the eyewear.

[0019] According to yet another aspect of the present application, the integrated lenses can also be coated with a multi-layer film having optical properties to enhance the optical image in front of the user's eyes.

[0020] According to yet another aspect of the present application, the eyewear comprises several electronic devices (e.g. sensors or microphones) to enable various interactions between the wearer and the displayed content. The signals captured by the devices (e.g. depth sensors) are transmitted wirelessly (e.g. RF wireless or Bluetooth) to the housing to eliminate wired connections between the eyewear and the housing.

[0021] According to yet another aspect of the present application, an optical conduit is used to transport an optical image received from an image source (e.g. a micro-display). The optical conduit is enclosed in or integrated with the temples of the display device. Depending on the implementation, the optical conduit comprising a bundle or array of optical fibers can be twisted, thinned or otherwise deformed to fit the fashion design of the temples while transporting the optical image from one end of the temple to the other.

[0022] According to yet another aspect of the present application, the optical image is a holographic image / video (hologram). The hologram is generated by phase modulation as well as amplitude modulation of the optical image by a spatial light modulation (SLM) device. Depending on the implementation, the hologram can be generated in the vicinity of the integrated lenses (at one end of the temples) or transported from an external device via an optical fiber.

[0023] According to one aspect of the present application, light propagation (e.g., from an SLM) is controlled in two different directions (e.g., 45 degrees and 0 degrees) to perform amplitude modulation (AM) and phase modulation (PM) simultaneously in a liquid crystal. According to another aspect of the present application, a mask is used to form an embossed microstructure array or pattern, where the pattern includes an array of alignment cells, a first set of alignment cells aligned in a first direction, and a second set of alignment cells aligned in a second direction. Depending on the application, the two cells from the first and second sets can correspond to a single pixel or two adjacent pixels, resulting in amplitude modulation and phase modulation within a pixel or within an array of pixels.

[0024] According to yet another aspect of the present application, a portable device can be used to house an SLM to perform AM and PM and provide a hologram for delivery via an optical fiber. Depending on the implementation, the portable device can be implemented as a standalone device or a docking unit to receive a smart phone. The portable device is primarily a control box connected to a network (e.g., the Internet) and generates control and instruction signals when controlled by a user. When a smart phone is received in the docking unit, many of the functions provided in the smart phone can be used, such as a web interface and touch screen to receive input from a user.

[0025] The present application can be implemented as an apparatus, method, system. Different implementations can yield different benefits, goals, and advantages. In one embodiment, the present application is a display device comprising: a display device comprising: a pair of eyeglass frames; at least one integrated lens, wherein the integrated lens is framed in the eyeglass frames; a spatial light modulation device to amplitude and phase modulate an optical image to produce a modulated image; and at least one holographic mirror to receive the modulated image and rotate the modulated image by 90 degrees to project the modulated image into the integrated lens, wherein the holographic mirror is optically coated to selectively allow or reflect specific wavelengths, a user wearing the display device is able to see a hologram produced by the modulated image in the integrated lens.

[0026] In another embodiment, the present application is a method of a display device, the method comprising: providing a pair of eyeglass frames comprising at least one integrated lens and a temple attached to the eyeglass frames; receiving an optical image; amplitude and phase modulating the optical image in a spatial light modulation device; generating a hologram with light intensity reflected by the spatial light modulation device illuminated by a uniform laser sheet; and projecting the hologram into the integrated lens via a mirror by a 90 degree rotation, wherein the mirror is optically coated to selectively allow or reflect specific wavelengths, a user wearing the display device is able to see a hologram produced by the modulated image in the integrated lens.

[0027] In addition to the above objects achieved and accomplished in the practice of the present application and in the following description, and resulting in the embodiments shown in the drawings, there are numerous other objects. BRIEF DESCRIPTION OF DRAWINGS

[0028] These and other features, aspects, and advantages of the present application will become better understood with reference to the following description, appended claims, and accompanying drawings where:

[0029] FIG. 1A shows an exemplary pair of glasses for applications for delivering or displaying VR / AR that are common in the market today;

[0030] FIG. IB shows a simplified diagram of the HoloLens from Microsoft;

[0031] Figure 2A An exemplary pair of glasses for applications that can be used for VR according to one embodiment of the present application is shown;

[0032] Figure 2B The use of optical fibers to transmit light from one location to another in a more efficient manner or by total internal reflection within the optical fiber along a curved path is shown;

[0033] Figure 2C Two exemplary ways of encapsulating an optical fiber or multiple optical fibers according to one embodiment of the present application is shown;

[0034] Figure 2D How an image is carried from a miniature display to an imaging medium through an optical fiber cable is shown;

[0035] Figure 2E A set of exemplary variable focus elements (VFEs) to accommodate adjustments in the projection of an image onto an optical object (e.g., an imaging medium or prism) is shown;

[0036] Figure 2F An exemplary lens for the glasses shown in Figure 2A FIG. 1A, where the lens includes two parts, a prism and an optical corrective lens or corrector, is shown;

[0037] Figure 2G Internal reflections from multiple sources (e.g., sensors, imaging media, and multiple light sources) in an irregular prism are shown;

[0038] Figure 2H A comparison of such integrated lenses with a coin and a ruler is shown;

[0039] Figure 2I A shirt where a cable is enclosed within or attached to the shirt is shown;

[0040] Figure 3A How three single color images are visually combined and perceived by the human vision as a full color image is shown;

[0041] Figure 3B An imaging medium is shown to produce three different color images under three different colors of light at wavelengths λ1, λ2, and λ3, respectively, comprising three films, each coated with one type of phosphor.

[0042] Figure 4 An optical image is shown being transported from one end of a waveguide to the other end using waveguides;

[0043] Figure 5A An exemplary functional block diagram is shown that can be used in a separate housing or enclosure to generate content about virtual reality and augmented reality for display on Figure 2A exemplary glasses of the type shown in FIG. 1;

[0044] Figure 5B An embodiment is shown in which an exemplary circuit is used in a separate housing device box (also called an image engine herein).

[0045] Figure 5C An exemplary embodiment is shown according to an embodiment of the present invention showing how a user wears a pair of designed display glasses.

[0046] Figure 5D An exemplary functional block diagram of a circuit is shown for an image engine in Figure 5B according to an embodiment employing the technology published in US Patent No. 10,147,350;

[0047] Figure 5E An example of an image engine (source) located near the end of the temple of the glasses (i.e., the hinge area) is shown;

[0048] Figure 5F A top view of a wearable display device configured to display a hologram according to an embodiment of the present invention is shown;

[0049] Figure 5G An exemplary circuit block diagram according to an embodiment employing the technology disclosed in US Patent No. 10,147,350, the contents of which are hereby incorporated by reference, is shown;

[0050] Figure 6A An array of multi-pixel cells is shown as an example, in which each pixel cell is shown to have four sub-pixel cells;

[0051] Figure 6B A concept is shown to generate an extended image from two generated frames of images;

[0052] Figure 6CAn example is shown of an expanded image that becomes twice the size with sub-pixel cells, the expansion being via a two pass processing procedure and separation to form two frames of images;

[0053] Figure 6D An example is shown of a method of splitting an image to create two frames of images that are equivalent in size to the original image by light intensity;

[0054] Figure 6E An example is shown of another embodiment of an input image that is expanded to two images of comparable size with two degrees of reduction and interleaving;

[0055] Figure 7A An example is shown of how an optical image is created using an optical cube;

[0056] Figure 7B An example is shown of a display eyewear providing images or video to an integrated lens without any other electronic components;

[0057] Figure 8A An example is shown of an exemplary LCoS structure that creates a 2-dimensional optical image (i.e., 2D varying intensity of light or modulated light to obtain grayscale of the image);

[0058] Figure 8B.1 An example is shown of an exemplary cross-sectional view of an LC layer with an alignment layer, where the alignment (rubbing) angle determines the properties of the light passing through the LC molecules;

[0059] Figure 8B.2 An example is shown of functional layers in an LCoS;

[0060] Figure 8C An example is shown of how an LCoS can be modified or redesigned to implement one embodiment of the present invention;

[0061] Figure 8D An example is shown of an exemplary 8x8 array of alignment cells (each corresponding to a pixel);

[0062] Figure 8E An example is shown of an array of alignment cells, each designed for both AM and PM, which in operation behaves as if each pixel in the entire SLM device has been split into two alternating halves, allowing half the pixels to perform AM and the other half to perform PM simultaneously at different ratios of AM to PM;

[0063] Figure 8F An example is shown of two separate graphical curves, one for reflectivity curve AM and the other for phase curve PM;

[0064] Figure 8G Shows Figure 8F exemplary alignment units for pixels, each or all of which can be steered to produce simultaneously different reflectivity curves AM (or transmittance in the case of LCoS) and phase curves PM;

[0065] Figure 8H The simulation results on a single pixel without involving neighboring pixels are shown;

[0066] Figure 8I An exemplary embodiment of a method using light to align a mask is shown; and

[0067] Figure 9 A process or procedure for creating an SLM apparatus that performs both AM and PM within a cell or array is shown according to one embodiment. DETAILED DESCRIPTION

[0068] The detailed description of the present invention is presented largely in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations of a data processing device directly or indirectly coupled to a network. These process descriptions and representations are commonly used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art.

[0069] Reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearance of the phrase "in one embodiment" in various places in this specification does not necessarily refer to the same embodiment, nor does it necessarily mean that separate or alternative embodiments are mutually exclusive of other embodiments. Furthermore, the order of blocks in a process flow diagram or diagram representing one or more embodiments of the invention does not inherently indicate any particular order or imply any limitation in the present invention.

[0070] References in this article Figures 2A to 8I Discussing Embodiments of the Invention However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes as the invention extends beyond these limited embodiments.

[0071] Referring now to the drawings, wherein like numerals refer to like parts throughout the several views. Figure 2A A pair of exemplary glasses 200 for VR / AR applications according to one embodiment of the present invention is shown. The glasses 200 appear similar to a normal pair of glasses, but include two flexible cables 202 and 204 extending from temples 206 and 208, respectively. According to one embodiment, the two flexible cables 202 and each pair of temples 206 and 208 are integrated or removably connected at one end and include one or more optical fibers.

[0072] Both flexible cables 202 are coupled at their other ends to a portable computing device 210, which computing device 210 or external box 210 includes the necessary components to generate image data to drive the micro display on which the electronic image is displayed. The image is carried through the optical fiber in the flexible cable 202 by total internal reflection therein all the way to the other end of the optical fiber, where the image is projected onto the lens in the eyeglasses 200. As will be further described, the optical image can be a hologram according to one embodiment of the present application.

[0073] According to one embodiment, each of the two flexible cables 202 includes one or more optical fibers. Optical fibers are used to transmit light from one place to another in a more efficient manner as shown in Figure 2B In one embodiment, the optical fiber is formed of thousands of strands of extremely fine quality glass or quartz having an index of refraction of about 1.7 or so. The thickness of a strand is microscopic. The strands are coated with a layer of some material of lower index of refraction. The ends of the strands are polished and firmly clamped after they have been carefully aligned. When light is incident at a small angle at one end, it is refracted into the strand (or fiber) and is incident on the interface between the fiber and the coating. At angles of incidence greater than the critical angle, the light ray undergoes total internal reflection and essentially carries the light from one end to the other, even when the fiber is bent. Depending on the implementation of the present application, a single optical fiber or multiple optical fibers arranged in parallel can be used to carry the optical image projected onto one end of the optical fiber to the other end thereof. Typically one high resolution image will require more fibers to transmit. According to one embodiment to be described below, the number of fibers used to transmit a first (low) resolution image is minimized to achieve a small number of fibers. Two such images (e.g., two consecutive images) after transmission, the two consecutive images are combined at double the refresh rate to generate a viewable second (high) resolution image.

[0074] Figure 2C Two exemplary ways of enclosing the optical fiber or fibers are shown at 230 or 232 according to one embodiment of the present application. The enclosed optical fiber can be used as Figure 2A the cable 202 or 204 in FIG. 1, and extends through each of the non-flexible temple pieces 206 and 208 all the way to the end thereof. According to one embodiment, the temple pieces 206 and 208 are made of a type of material commonly found in ordinary eyeglasses (e.g., plastic or metal), a portion of the cable 202 or 204 is embedded in or integrated with the temple piece 206 or 208, thereby creating a non-flexible portion, while another portion of the cable 202 or 204 remains flexible. According to another embodiment, the non-flexible portion and the flexible portion of the cable 202 or 204 can be removably connected by an interface or connector.

[0075] Referring now to Figure 2D which shows how an image is transported from a micro display 240 through a fiber optic cable 242 to an imaging medium 244. As will be further described below, the imaging medium 244 can be a physical thing (e.g., a film or lens) or a non-physical thing (e.g., air). The micro display is a display with an extremely small screen (e.g., less than one inch). This type of micro electronic display system was introduced commercially in the late 1990s. The most common applications for micro displays include rear-projection TVs and head-mounted displays. The micro display can be reflective or transmissive, depending on the way light is allowed to pass through the display unit. Through a lens 246, an image (not shown) displayed on the micro display 240 is picked up by one end of the fiber optic cable 242, which transports the image to the other end of the fiber optic cable 242. Another lens 248 is provided to collect the image from the fiber optic cable 242 and project the image onto the imaging medium 244. Depending on the implementation, there are different types of micro displays and imaging media. Some embodiments of the micro display and the imaging medium will be described in detail below.

[0076] Figure 2E A set of exemplary variable focus elements (VFEs) 250 are shown to accommodate adjustment of the projection of an optical image onto an optical object (e.g., an imaging medium, prism or lens). To facilitate the description of various embodiments of the present invention, it is assumed that there is an image media. As shown in Figure 2E The image 252 transported through the fiber optic cable arrives at an end surface 254 of the fiber optic cable. The image 252 is focused onto an imaging medium 258 by a set of lenses 256 referred to herein as variable focus elements (VFEs). The VFEs 256 are provided to adjust to ensure that the image 252 is precisely focused onto the imaging medium 258. Depending on the implementation, the adjustment to the VFEs 256 can be done manually or automatically according to an input (e.g., a measurement obtained from a sensor). According to one embodiment, the adjustment to the VFEs 256 is performed automatically according to a feedback signal derived from a sensed signal from a sensor directed at an eye (pupil) of a wearer of the eyewear 200. Figure 2A

[0077] Referring now to Figure 2F which shows that the VFEs 256 can be used to adjust the image 252 to be focused onto the imaging medium 258. Figure 2A ​An exemplary lens 260 of eyeglasses is shown in FIG. 26. The lens 260 comprises two parts: a prism 262 and an optical corrective lens or corrector 264. The prism 262 and the corrector 264 are stacked to form the lens 260. As the name implies, the optical corrector 264 is provided to correct the optical path from the prism 262 so that light passing through the prism 262 is collimated by the corrector 264. In other words, the refracted light from the prism 262 is corrected or de-refracted by the corrector 264. In optics, a prism is a transparent optical element with flat, polished surfaces that refract light. At least two of the flat surfaces must have some angle between them. The exact angle between the surfaces depends on the application. The traditional geometry is a triangular prism with a triangular base and rectangular sides, and in colloquial usage, a prism generally refers to this type. Prisms can be made of any material transparent to the wavelengths for which they are designed. Typical materials include glass, plastic, and fluorite. According to one embodiment, the type of the prism 262 is not really in the shape of a geometric prism, so the prism 262 is referred to herein as an arbitrary shape prism, which directs the corrector 264 to a shape that is complementary, reciprocal, or conjugate to the form of the prism 262 to form the lens 260 (i.e., an integrated lens). As further described below, in one embodiment, the prism 262 can simply be a waveguide.

[0078] On one edge of the lens 260 or the edge of the prism 262, there are at least three items that take advantage of the prism 262. Labeled 267 is an imaging medium that corresponds to Figure 2D the imaging medium 244 or Figure 2E the imaging medium 258. Depending on the implementation, the image delivered by Figure 2D the optical fiber 242 can be projected directly onto the edge of the prism 262 or formed on the imaging medium 267 before it is projected onto the edge of the prism 262. In any case, depending on the shape of the prism 262, the projected image is refracted in the prism 262 and then seen by the eye 265. In other words, a user wearing a pair of eyeglasses that uses the lens 262 can see the image displayed through or in the prism 262.

[0079] A sensor 266 is provided to image the position or movement of the pupil in the eye 265. Again, based on the refraction provided by the prism 262, the sensor 266 can find the position of the pupil. In operation, an image of the eye 265 is captured. The image is analyzed to derive the way the pupil views the image displayed through or in the lens 260. In applications of AR, the position of the pupil can be used to activate some action. As appropriate, a light source 268 is provided to illuminate the eye 265 to facilitate the image capture by the sensor 266. According to one embodiment, the light source 268 uses a near-ir source, whereby the user or his eye 265 will not be affected by the light source when it is on.

[0080] Figure 2G Internal reflections from multiple sources (e.g., sensor 266, imaging medium 267, and light source 268) are shown. Due to the prism being uniquely designed, especially in shape or having specific edges, the light rays from the sources reflect several times within the prism 268 and then impinge on the eye 265. For completeness, Figure 2H A comparison of such a lens to a coin and a ruler in size is shown.

[0081] As described above, there are different types of micro-displays, and thus different imaging media. The following table summarizes some micro-displays that can be used to facilitate the generation of an optical image that can be transported by one or more optical fibers from one end to the other end of the optical fiber by total internal reflection within the optical fiber.

[0082]

[0083] LCoS = Liquid Crystal on Silicon;

[0084] LCD = Liquid Crystal Display;

[0085] OLED = Organic Light Emitting Diode;

[0086] RGB = Red, Green, and Blue; and

[0087] SLM = Spatial Light Modulator.

[0088] In the first case shown in the above table, a full color image is actually on a silicon-based display. As Figure 2D indicated in the above table, a full color image can be picked up by a focusing lens or a set of lenses that project the full image onto one end of an optical fiber. The image is transported within the optical fiber and picked up again by another focusing lens at the other end of the optical fiber. Since the transported image is visible and full color, the imaging medium 244 can not be physically needed. Figure 2D A color image can be directly projected onto one edge of the prism 262 of the Figure 2F

[0089] In the second case shown in the above table, an LCoS is used with different light sources. Specifically, there are at least three color light sources (e.g., red, green, and blue) used sequentially. In other words, each light source generates a single color image. The image picked up by the optical fiber is only a single color image. When all three different single color images are combined, a full color image can be reproduced. The imaging medium 244 of Figure 2D is provided to reproduce a full color image from the three different single color images transported by the optical fiber respectively.

[0090] Figure 2I ​A shirt 270 is shown with a cable 272 enclosed within or attached to the shirt 270. The shirt 270 is an example of a fabric material or a multi-layer piece. Such a relatively thin cable can be embedded in the multi-layer piece. When a user wears such a shirt manufactured or designed according to one embodiment, the cable itself has less weight and the user can move around more freely.

[0091] Figure 3A Three single color images 302 are shown how they are visually combined and perceived by human vision as a full color image 304. According to one embodiment, three color light sources are used, for example, red, green and blue light sources turned on sequentially. More specifically, when the red light source is turned on, only a red image is produced as a result (for example, from a micro display). The red image is then optically picked up and transported by an optical fiber and then projected into a prism 262 of Figure 2F As the green and blue lights are then turned on sequentially, green and blue images are produced and transported by optical fibers respectively, and then projected into the prism 262 of Figure 2F It is well known that human vision has the ability to combine three single color images and perceive them as a full color image. With all three single color images projected into the prism sequentially in perfect alignment, the eye sees a full color image.

[0092] In addition in the second case shown above, the light sources can be near invisible. According to one embodiment, the three light sources produce light near the UV band. Under such illumination, three different color images can still be produced and transported, but not fully visible. Before the color images can be presented to the eye or projected into the prism, they will be converted to three primary color images, which can then be perceived as a full color image. According to one embodiment, an imaging medium 244 of Figure 2D is provided. Figure 3B Three single color images 302 are shown how they are visually combined and perceived by human vision as a full color image 304. According to one embodiment, three color light sources are used, for example, red, green and blue light sources turned on sequentially. More specifically, when the red light source is turned on, only a red image is produced as a result (for example, from a micro display). The red image is then optically picked up and transported by an optical fiber and then projected into a prism 262 of Figure 2D As the green and blue lights are then turned on sequentially, green and blue images are produced and transported by optical fibers respectively, and then projected into the prism 262 of Figure 3BThe imaging medium 312, shown producing three different color images 310 under three light sources at wavelengths λ1, λ2, and λ3, respectively, includes three film layers 314, each film layer 314 coated with one type of phosphor, a substance that exhibits luminescence. In one embodiment, three types of phosphors at wavelengths 405 nm, 435 nm, and 465 nm are used to convert three different color images produced under three light sources in the near UV band. In other words, when one such color image is projected onto a film layer coated with phosphor at wavelength 405 nm, the single color image is converted to a red image, which is then focused and projected into a prism. The same process occurs for the other two single color images through film layers coated with phosphor at wavelengths 435 nm or 465 nm, producing green and blue images. When such red, green, and blue images are projected into the prism in sequence, human vision perceives them together as a full color image.

[0093] In the third or fourth case shown in the table above, instead of using light in the human visible spectrum or near invisible light, the light source uses a laser source. There are also visible and invisible lasers. With little difference in operation from the first and second cases, the third or fourth case uses a so-called spatial light modulator (SLM) to form a full color image. A spatial light modulator is a general term describing a device for modulating the amplitude, phase, or polarization of a light wave in space and time. In other words, SLM + laser (RGB sequential) can produce three separate color images. When the color images are combined with or without an imaging medium, a full color image can be reproduced. In the case of SLM + laser (invisible), an imaging medium will be present to convert the invisible images to a full color image, in which case, appropriate film layers can be used as shown in Figure 3B

[0094] Reference is now made to Figure 4 which shows a waveguide 400 for transporting an optical image 402 from one end 404 to the other end 406 of the waveguide 400, where the waveguide 400 can be stacked with or coated with one or more film layers with one or more pieces of glass or lenses (not shown) to form or be part of a suitable lens for use in a pair of glasses for displaying images from a computing device. As known to those skilled in the art, an optical waveguide is a spatially inhomogeneous structure for guiding light, i.e., for restricting the spatial region in which light can propagate, where the waveguide contains regions of increased refractive index compared to the surrounding medium, often called a cladding.

[0095] ​The waveguide 400 is transparent and shaped at the 404 end in an appropriate manner to allow the image 402 to propagate along the waveguide 400 to the end 406 where the user 408 can view through the waveguide 400 to see the propagated image 410. According to one embodiment, one or more film layers are disposed on the waveguide 400 to magnify the propagated image 410 so that the eye 408 can see a significantly magnified image 412. One example of such a film layer is called a metalens (superlens), which is essentially a thin array of titanium dioxide nanosheets on a glass substrate.

[0096] Reference is now made to Figure 5A which shows an exemplary functional block diagram 500 that can be used in a single housing or enclosure to generate virtual reality and augmented reality related content for display on Figure 2A exemplary eyeglasses. As shown in Figure 5A two micro-displays 502 and 504 are provided to supply content to the two lenses in the eyeglasses of Figure 2A essentially the left image to the left lens and the right image to the right lens. Examples of the content are 2D or 3D images and videos or holograms. Each of the micro-displays 502 and 504 is driven by a corresponding driver 506 or 508.

[0097] The entire circuit 500 is controlled and driven by a controller 510 that is programmed to generate the content. According to one embodiment, the circuit 500 is designed to communicate with the Internet (not shown) to receive the content from other devices. Specifically, the circuit 500 includes an interface to receive sensing signals wirelessly (e.g., RF or Bluetooth) from remote sensors (e.g., the sensors 266 of Figure 2F The controller 510 is programmed to analyze the sensing signals and to provide feedback signals to control certain operations of the eyeglasses, such as a projection mechanism that includes an auto-focus and projects an optical image to the edge of the prism 262 of Figure 2F or a focusing mechanism on the waveguide 400 of Figure 4 In addition, audio is provided to be synchronized with the content, and the audio can be transmitted wirelessly to headphones (e.g., via Bluetooth).

[0098] Figure 5A An exemplary circuit 500 is shown that generates content for display in a pair of eyeglasses contemplated in one embodiment of the present invention. The circuit 500 shows that there are two micro-displays 502 and 504 that are used to provide two respective image or video streams to the two lenses of the eyeglasses in Figure 2A According to one embodiment, only one micro-display can be used to drive both lenses of the eyeglasses in Figure 2A Since those skilled in the art know how the circuit can be designed or how the circuit of Figure 5Acircuitry 500, and therefore such circuitry is not provided herein.

[0099] Figure 5B An embodiment is shown in which an exemplary circuit 500 is housed within a single housing device 516 according to this embodiment. The device 516 includes the necessary electronic components to receive an image source or video from a smartphone 518, while also being a controller to provide the required interface so that the wearer or user can manipulate what is received and shown on the display glasses, as well as how to interact with the display. Figure 5C An exemplary embodiment is shown in which a user is wearing such display glasses. According to this embodiment, the display glasses 520 do not include active electronic components (power driven) except for a pair of optical fibers 522 to deliver the image or video. The accompanying sound can be provided by the smartphone 518 directly to earphones (earbuds or Bluetooth earphones). As will be further described below, the thickness or number of optical fibers 522 from the image engine 516 to the glasses 520 to transmit or deliver low resolution images and video will again be reduced.

[0100] According to an embodiment of the present application, Figure 5D An example is shown in which some of the electronic components (i.e., the image engine) 530 in the device 516 are located near one end of the temple 532. The image engine 530 includes a light source 534 to illuminate an SLM (e.g., LCoS) 536 implemented to perform AM and PM and optical components 538 to provide the generated image (e.g., hologram) to be delivered to the lens (not shown) via optical fibers integrated in the temple 532. In operation, image data is provided to the image engine 530 via a wire 540 coupled to the device 516.

[0101] According to an embodiment of the present application, Figure 5E An example is shown in which the image engine 530 is located near the other end of the temple 532 (i.e., the hinge area in a pair of conventional glasses). In this example, image data is provided directly to the image engine 530 via a wire 540 coupled to the device 516.

[0102] Figure 5FA top view 550 of a wearable display device configured to display a hologram according to one embodiment of the present application is shown. At least one laser diode 552 is provided as a laser source to generate a laser plate 554 via an optical or mirror configuration 556, where the laser plate 554 is a uniform flat light. The light plate 554 impinges on an optical cube 558 (in one embodiment, composed of two halves) that directs the light plate 554 onto an SLM 560. As will be further described below, the light 554 is modulated according to the image displayed on the SLM 560 and further modulated in amplitude and phase. The reflected light 562 is projected onto a mirror 564 that redirects or forward rotates the reflected light 562 by 90 degrees. A user wearing the wearable display device can view a hologram 566 via the mirror 564, where the mirror 564 is optically coated to selectively allow certain wavelengths to pass or reflect. To prevent the projected hologram 566 from being reflected by a lens 568, the lens 568 is optically coated or integrated with a wavelength selective holographic mirror 570 to selectively allow certain wavelengths to pass.

[0103] Figure 5G An exemplary circuit 580 is shown according to one embodiment employing the technology published in U.S. Patent No. 10,147,350, the contents of which are hereby incorporated by reference. As shown, the circuit 550 essentially produces two low resolution images (e.g., 640x480), where the two images are diagonally shifted by one pixel and the refresh rate is 120 Hz (for the commonly used "standard" refresh rate of 60 Hz in the U.S.). The commonly used refresh rate is 60 Hz for most TVs, PC monitors, and smartphones. A refresh rate of 60 Hz means that the display is refreshed 60 times a second, in other words, the displayed image is updated (or refreshed) once every 16.67 microseconds (ms). When such two images are refreshed twice at the standard refresh rate, the perceived image resolution by a user on the integrated glasses is doubled, i.e., nominally to 1280x960. Figure 5G

[0104] ​According to one embodiment, a display image on a display glasses is originally (first) at a resolution, for example, 640x480, or is preset as a resolution efficient for transmission through an optical fiber, and is at a first refresh rate when transmitting a video. If the image is at a resolution higher than the first resolution, it can be reduced to a lower resolution. According to U.S. Patent No. 10,147,350, a duplicated but shifted by half a pixel on a diagonal image is generated to cause a second image also at the first resolution, and the two images are sequentially projected on the optical fiber 522 at twice the refresh rate of the original image, that is, a second refresh rate equal to twice (=2X) the first refresh rate. When the images are sequentially output from the other end of the optical fiber, they will be seen as a second resolution image in the waveguide, which presents twice the first resolution.

[0105] Figures 6A-6E Figures 16A-16E of U.S. Patent No. 10,147,350 are reproduced. As described above, the optical image output from the optical fiber in one embodiment of the present application will be twice the spatial resolution of the input image as seen. Referring to Figure 6A An array of pixel cells 600 (forming an image or a data image) is shown with four sub-image cells 604A, 604B, 604C, and 604D. When an input image with a first resolution (for example: 500x500) is received and displayed at the first resolution, each pixel value is stored in each pixel cell 600. In other words, sub-image cells 604A, 604B, 604C, and 604D are all written or stored with the same value and are addressed simultaneously. As shown in Figure 6A As shown, word lines (for example: WL0, WL1, or WL2) can address sub-pixels belonging to two columns in pixel 602 simultaneously, and bit lines (for example: BL0, BL1, or BL) can address sub-pixels belonging to two rows in pixel 602 simultaneously. At any time, a pixel value is written into pixel 602, and sub-image cells 604A, 604B, 604C, and 604D are all selected at this time. Ultimately, the input image is displayed at the first resolution (for example: 500x500), that is, the input image is all the same resolution.

[0106] Now assume that an input (data) image at a first resolution (for example: 500x500) is received and displayed at a second resolution (for example: 1000x1000), where the second resolution is twice the first resolution. According to one embodiment, sub-image cells are used to achieve the viewable resolution. It is extremely important to understand that this improved spatial resolution is viewable by the human eye, not the actual double resolution of the input image. To facilitate the description of the present application, Figure 6B andFigure 6C used to illustrate how the enlarged input image is expanded to achieve viewable resolution.

[0107] Now assume an input image 610 is of resolution 500x500. Via data processing 612 (e.g., enlargement and sharpening), the input image 610 is expanded to the size of an image 614 of 1000x1000. Figure 6C An example of image 616 being expanded to image 618 to be twice the size with sub-pixel cells is shown. In operation, each pixel of image 616 writes a group of cells including all (four) sub-pixel cells (e.g., exemplary sub-pixel is 2x2). Those skilled in the art will appreciate that the description herein can be immediately applied to other sub-pixel configurations (3x3, 4x4, 5x5, etc.) resulting in even more viewable resolution. According to one embodiment, a sharpening process (e.g., data processing in the middle section) is applied to expand image 618 is the underlying process (e.g., filtering, thinning, or sharpening the image edges) to achieve the purpose of generating two frames from the expanded image 618. In one embodiment, the values of each sub-pixel are recomputed mathematically to achieve a better defined edge to generate image 620, in another embodiment, the values of adjacent pixels are used as a reference to get a sharp edge. Figure 6B

[0108] The processed image 620 is then separated into two images 622 and 624 via a separation process 625. Both images 622 and 624 are of the same resolution as the input image (e.g., 500x500), where the sub-pixel cells of image 622 and 624 are written or stored with the same values. The boundary of the pixel cells of image 622 is intentionally different from the boundary of the pixel cells of image 624. In one embodiment, the boundary of the pixel cells is offset by half a pixel (equivalent to one sub-pixel in a 2x2 sub-pixel array) in the vertical direction and also offset by half a pixel (equivalent to one sub-pixel in a 2x2 sub-pixel array) in the horizontal direction. The separation process 625 is performed in a manner that when images 622 and 624 are overlapped, the combined image best matches image 620 and is four times the resolution of the input image 616. In Figure 6C ​In the example in FIG, to maintain a fixed intensity of the input image 610, the separation process 625 also includes a process of reducing the intensity of each of the two images 622 and 624 by 50%. Operationally, the intensity of the first image is reduced by N percent, where N is an integer ranging from 1 to 100, but is practically set to be around 50. As a result, the intensity of the second image is reduced by (100-N) percent. Either of the two images 622 and 624 is displayed at a refresh rate twice that of the input image 610. In other words, if the input image is displayed at 50 Hz per second, each pixel of the two images 622 and 624 is displayed at 100 Hz per second. Due to the offset of the pixel boundaries and the processing of the data, the combined image perceived by the viewer is similar to the image 620. The offset pixel boundaries between the two images 626 and 624 have the effect of "displacing" the pixel boundaries. According to another embodiment, as shown by the two pixels 626 and 628, at Figure 6C The example given in is similar to a shift of one (sub) pixel in the southeast direction.

[0109] According to the embodiment, the separation process 625 can be performed by an image algorithm or a pixel shift, where a pixel shift means Figure 6A A sub-pixel in a sub-pixel structure is shown. There are many ways to separate an NxM image into two images by intensity, each image still being NxM, so that the perceived display effect of either image will be twice the refresh rate for optimal viewing. For example, an exemplary approximation is to retain and modify the original image and reduce the intensity as the first frame, while generating the second frame from the remainder of the first frame, again at a reduced intensity. In another embodiment, the approximation is to shift the first frame (from the original or modified image) by half a (1 / 2) pixel (e.g., horizontally and vertically or diagonally) to generate the second frame, further details of which will be provided later. Figure 6C Two images 622 and 624 are shown as being generated from processing the expanded image 620, with two pixels 626 and 628 being generated simultaneously therewith according to an image processing algorithm, wherein the pixels of the first frame are diagonally shifted to produce the second frame. It should be noted that the separation process here means separating the images by their intensity to produce two frames of the same size as the original image. Figure 6DAn image of two pixels, one at full intensity (shown as black) and the other at half full intensity (shown as gray). When the two pixel image is separated into two frames of the same size as the original, the first frame has two pixels, both at half full intensity (shown as gray) and the second frame also has two pixels, one at half full intensity (shown as gray) and the other at almost zero full intensity (shown as white). There are now twice as many pixels as in the original image, which displays as a checkerboard pattern. Since each pixel is refreshed 60 times per second instead of 120 times per second, each pixel has only half as much brightness, but because there are twice as many of them, the overall brightness of the image remains the same.

[0110] Referring now to FIG. 6, Figure 6E which shows another embodiment of expanding the input image 610. The input image 610 is still assumed to be 500x500 in resolution. Through data processing 612, the input image 610 is expanded to a size of 1000x1000. In this embodiment, it should be understood that 1000x1000 is not the resolution of the expanded image. The expanded image is two 500x500 substantially reduced images 630 and 632. The expanded view 634 of the substantially reduced images 630 and 632 shows that the pixels in one image are substantially reduced to allow the pixels of the other image to be generated between the pixels. According to one embodiment of the present application, the first checkerboard image is derived from the input image and the second image is derived from the first image. As shown in the expanded view 634 of FIG. 6, Figure 6E As shown in the expanded view 634 of FIG. 6, an exemplary pixel 636 in the second image 632 is derived from three pixels 638A, 638B, and 638C. In the same manner, that is, shifting by one-half (1 / 2) pixel in a set direction can be applied to generate all of the pixels of the second image. At the end of the data processing 612, there is an interleaved image that includes both images 630 and 632, each of which is 500x500. Another separation process 625 is applied to the interleaved image to generate or store the two images 630 and 632 therein.

[0111] Referring now to FIG. 7, Figure 7AAn embodiment is shown, which illustrates how an optical cube 702 is used to generate an optical image. Based on a light source 704, the image displayed on a microdisplay (e.g., LCoS or OLED, a type of spatial light modulation device) 706 is projected as an optical image (light intensity) that is captured by a lens 708. The optical image is then transmitted to the other end of an optical fiber 710. The optical image is then projected into a waveguide or integrated lens 712 via another lens (e.g., a collimator) 714. The optical image is ultimately viewed by a human eye 716 through the waveguide lens 712.

[0112] According to one embodiment, light source 704 is a laser sheet generated from a laser point. There are many optical approaches to generating a uniform laser sheet (planar laser), the details of which will not be further described herein to avoid obscuring important aspects of the present invention. The laser sheet is used to illuminate spatial light modulation (SLM) 706, where it is amplitude- and phase-modulated. Reflected light from SLM 706 is captured by a lens (not shown) and focused onto a medium (e.g., a waveguide 712 or one end of an optical fiber). In operation, three laser sheets of three primary colors (e.g., red, green, and blue) sequentially impinge on SLM 706, assuming it is a reflective device. Each laser sheet is modulated in SLM 706, and the reflected (modulated) light is strongly coupled into waveguide 712, where the user can view the reconstructed color hologram.

[0113] Figure 7B Display glasses 720 are shown without any other power-driven electronic components, and the image engine 722 is located externally (e.g., Figure 5B The display glasses 720 may be provided with a housing 516) to provide images or videos to the integrated lenses, thereby making the display glasses 720 extremely lightweight while still being able to see all types of images / videos including holograms.

[0114] Figure 8A Shows that can be Figure 5D or 5E Graphics Engine 530 or Figure 7B An exemplary structure 800 of an LCoS used in the image engine 722 of FIG. In perspective, an LCoS generates a 2-dimensional optical image (i.e., 2D light of varying intensities or modulated light). As is well known, digital images can be transmitted via data cables, whereas optical images cannot. Generally, depending on the application, optical images are conveyed via an optical medium (e.g., air, waveguide, or optical fiber). Rather than using tiny mirrors that switch on and off, LCoS uses liquid crystals as light modulators, controlling the amount of reflected light by changing their angle.

[0115] Liquid crystals (LCs) are substances that are in a mesomorphic state (not quite liquid or solid). Their molecules usually keep their shape, like a solid, but they can also move around like a liquid. For example, nematic liquid crystals are arranged in loose parallel lines. A liquid crystal layer (or LC layer) is positioned, sandwiched, or coupled between a transparent electrode layer and a reflective electrode layer, where the reflective electrode in the reflective electrode layer includes an array of pixel electrodes and is constructed on a silicon substrate. It should be noted that between the transparent electrode layer (sometimes simply referred to as the transparent layer, such as Figure 8A and 8C ) and the reflective electrode layer (sometimes also simply referred to as the reflective layer, such as Figure 8A and 8C ) there are other layers integrated with the LC layer. As used herein, the terms "positioned," "sandwiched," or "coupled" between two layers does not mean that there is only one object between the two layers. Other layers of material or components can be added on top of the object or sandwich the object to alter, modify, or enhance the behavior, performance, or characteristics of the object, all between the two layers. When placed between two polarizing layers, twisted crystals direct the path of light. When a voltage difference is applied between the transparent electrode layer and one of the pixel electrodes, the LC molecules in between reorient with the applied electric field. By changing the direction of the light, the crystals allow or prevent it from passing through.

[0116] Liquid crystal molecules are usually much longer in length than they are wide. In a calamitic liquid crystal, the molecules are locally aligned in the same direction, resulting in optical birefringence, i.e., the refractive index along the long axis of the molecule is significantly different from the refractive index perpendicular to the long axis of the molecule. In other words, birefringence is an optical property of a material in which the refractive index depends on the polarization and direction of propagation of light. Without further detailing the molecules and / or liquid crystals and how they affect birefringence, which is beyond the scope of the present invention, it is well known that the polarization and direction of propagation of light into the LC layer determines the reflectivity or transmissivity of the light through the LC layer.

[0117] When a voltage difference is applied between the transparent electrode layer and one of the pixel electrodes, the LC molecules in between reorient with the applied electric field. By changing the direction of the light, the crystals allow or prevent it from passing through. Because LCs are birefringent, the reorientation causes a phase shift to the light, commonly referred to as a phase delay, where the phase delay can be controlled by the voltage difference that creates the electrically controlled birefringence effect (ECB mode).

[0118] When linearly polarized incident light enters the LC layer at an angle of Φ to the director axis of the liquid crystal, it splits into two beams with different polarizations, i.e., the extraordinary wave (E light) with the polarization direction parallel to the director axis of the liquid crystal, and the ordinary wave (O light) with the polarization direction perpendicular to the director axis of the liquid crystal. Because the E light and O light pass through the liquid crystal at different speeds, their refractive indices are different. Thus, when the two waves exit the liquid crystal, there is a phase difference of b between them, i.e.,

[0119]

[0120] where d is the cell gap (i.e., the thickness of the LC layer), Δn depends on the applied voltage, temperature, and the wavelength λ of the incident light v , and Δn = n e - n o , also known as birefringence.

[0121] When a homogeneous cell is sandwiched between two polarizers, the normalized light transmission is governed by the following equation:

[0122] T = cos 2 X - sin 2β sin 2(β - X) sin 2 (б / 2); Eq. (2)

[0123] where X is the angle between the polarizer and the analyzer, β is the angle between the polarizer and the LC director, and б is the phase retardation in Eq. 1. For the simplest case of β = 45 degrees and two polarizers either parallel (X = 0) or orthogonal (X = 90), the normalized light transmission simplifies to:

[0124] T ∥ = cos 2 (б / 2); Eq. (3)

[0125] T ┸ = sin 2 (б / 2); Eq. (4)

[0126] As further shown in Figure 8A , there is an essential component, the alignment layer, which dictates the macroscopic uniform alignment of the liquid crystal molecules (liquid crystal mesogens) in the vicinity of its surface, essentially orienting the LC molecules with a specific pre-tilt angle, which is the angle between the director axis of the LC molecules and the alignment layer. Figure 8B.1 An exemplary cross-sectional view of an LC layer with an alignment layer is shown, where the pre-tilt alignment dictates the properties of light passing through the LC molecules. Different pre-tilt alignment angles can yield very different modulated light, as does the thickness of the LC (e.g., the respective optical path through it). There are several ways to form a surface alignment layer. One example is the use of unidirectional mechanical rubbing of a thin polyimide coating. The thin film is spin-coated and then cured at an appropriate temperature depending on the polyimide type. Subsequently, the cured film is rubbed with a flat chamois cloth, creating micro- or nano-grooves along the rubbing direction to which the LC molecules are accordingly aligned. Figure 8B.2 Exemplary functional layers in an LCoS are shown.

[0127] Referring now to Figure 8C, it shows an example of how LCoS 800 can be modified or redesigned to implement one embodiment of the present invention. An alignment layer 802 is disposed on top of a liquid crystal layer 803 (i.e., LC layer) to configure the liquid crystal to have a predetermined pre-tilt alignment angle across the pixel array. Incident light is transmitted through the LC layer with almost zero absorption. Integration of high performance driving circuitry allows varying the applied voltage on each pixel, thereby controlling the phase delay of the incident wavefront passing through the device. Currently, there are two types of light modulation using LCoS devices, amplitude modulation (AM) and phase modulation (PM). In the AM case, the amplitude of the light signal is modulated by changing the linear polarization direction of the incident light. In the PM case, the phase delay is achieved by electrically controlled adjustment of the optical refractive index along the light path. It is not necessary to describe further the details of how the incident light is modulated by the liquid crystal (LC) or in the LC layer to avoid obscuring aspects of the present invention. One of the goals, benefits, and advantages in the present invention is to control the pre-tilt alignment angle via a modified alignment layer or an array of alignment cells integrated with the alignment layer. For ease of describing the present invention, for AM, all the alignment cells are diagonally aligned (e.g., neither horizontally nor vertically aligned, or between 20-60 degrees); for PM, all the alignment cells are horizontally aligned, meaning from 0 degrees to 360 degrees and beyond.

[0128] It is noted that throughout the description herein, the alignment layer is used as a substrate to form or hold the alignment cells (or imprinted microstructures). One skilled in the art can appreciate from the description herein that the described alignment cells can be well incorporated into the alignment layer when the alignment layer is designed or formed. For ease of describing the present invention, it is assumed that the alignment cells are formed on top of the alignment layer.

[0129] According to one embodiment of the present invention, two cells that are aligned differently are arranged in a certain manner 806 such that their alignment alternates across the alignment layer, i.e., the alignment of each alignment cell is different from the alignment of its neighboring alignment cell. In other words, the alignment of the alignment cells alternates from AM to PM. In operation, when light passes through these cells and the LC layer is applied with appropriate voltage or current, AM and PM occur simultaneously. One of the advantages, benefits, and goals in the present invention is to make AM and PM occur simultaneously in an SLM device (e.g., LCoS panel). Because all the light is simultaneously phase and amplitude modulated, the holographic image reproduced from this implementation can efficiently present high resolution.

[0130] Figure 8D An exemplary 8x8 array of alignment cells, each corresponding to one pixel, is shown. According to Figure 8DIn one embodiment shown in FIG. 8, the alignment units for AM and PM are arranged alternately across the SLM device, i.e., the pixels are alternated in odd and even rows or columns within one SLM device. In the perspective view, half of the pixels perform AM and the other half of the pixels perform PM simultaneously. In some modified embodiments, the alignment units for AM and PM can be randomly selected, or the desired pattern is designed to define certain pixels or groups of pixels for AM or PM.

[0131] According to Figure 8E another embodiment shown in FIG. 9, by splitting each pixel into two parts, it allows half of the pixels to perform AM and the other half of the pixels to perform PM simultaneously, such that the alignment units for performing AM and PM are located within a single pixel. Depending on the implementation, the percentage of one pixel for performing AM or PM can be 50% or predefined according to the desired performance, some of which are also shown in Figure 8E FIG. 10.

[0132] According to one prior art system estimation, the light efficiency of holographic display based on amplitude modulation is extremely low (e.g., approximately only 5%), while the light efficiency based on phase modulation is increased (e.g., to 95%). In the case of integration of AM and PM, the light efficiency can be significantly increased without loss of resolution.

[0133] Figure 8F Two separate graphical curves 842 and 844 are shown, one for reflectivity curve 842 and the other for phase curve. Under appropriate electrode voltage (e.g., 0-2.5V, 0-5V, etc.), half of the pixels 846 perform AM by gradually turning the corresponding LC from black to white or from white to black, while the other half of the pixels 846 perform PM by gradually turning the corresponding LC from 0 to 2π or from 2π to 0. As Figure 8E shown, the ratio of AM to PM on a single pixel is not fixed at 50:50, but can be any number (e.g., 40:60 or 70:30). Figure 8G The alignment units of the pixels 846 shown in Figure 8F are steered to produce different reflectivity and phase curves to achieve different desired results.

[0134] Figure 8HThe simulation shows the results on a single pixel 846 that does not involve adjacent pixels. The simulation shows that when the applied voltage changes from 0 V to 5 V, the liquid crystal 850 corresponding to PM (left part) is oriented differently than the liquid crystal corresponding to AM (right part). Note that the thickness or depth of the LC layer 850 is preferably twice that of its case for single modulation in one pixel. According to one embodiment, the depth of the LC layer 850 is assumed to be greater than 2D, where D is the depth of the LC layer for one pixel or array of pixels for AM or PM only. In other words, slightly more than this twice thickness to ensure that the phase shift (0~2π) is achieved. The corresponding reflectivity curve 852 and phase curve 854 are also presented for two voltages VI = 0 and V2 = 5. The physical size of the pixel 846 is assumed to be 6.4 μιη wide, so if the ratio is 50:50, then half of the width 6.4 μιη is dedicated to AM and the other half of the width 6.4 μιη is dedicated to PM.

[0135] Figure 8I An exemplary embodiment 860 using a photo-alignment mask method is shown. A photo mask 864 is added on the alignment layer 862. Given the predefined pattern imprinted on the photo mask 864, for example 50 / 50 per cell, i.e., the cell 862 is configured to cause AM and PM to occur simultaneously. The photo mask 864 is etched with UV light or other means 868. Thus, a pixel is covered with alignment cells 870 having two different alignments, one 872 for PM and the other 874 for AM. All cells of the alignment layer in a single SLM device etch the same pattern, as similarly shown in Figure 8E Figure 8I In an alternative embodiment (not shown in Figure 8I ), the alignment cells with horizontal or diagonal alignment cover only one pixel. All adjacent cells that each cover a single pixel can be aligned differently. In other words, all alignment cells alternate between aligning PM horizontally or AM diagonally, as similarly shown in Figure 8D

[0136] Figure 9 A flow or process 880 to create an SLM device that performs AM and PM simultaneously within a cell or within an array is shown, according to one embodiment. The process 880 can be better understood in conjunction with the above figures. The process 880 begins when a photo mask is added on top of the alignment layer.

[0137] An SLM (spatial light modulation) device, such as LCoS, includes an LC layer to control the passage of reflected (or transmitted) light. As described above, one embodiment is to modify or add an alignment layer on top of the LC layer. Depending on the resolution of the SLM, there are multiple alignment cells, each responsible for one pixel. Considering the properties of the underlying LC, unique control of these cells is needed in order for the LC in the LC layer to adjust the amplitude and phase of the reflected light.

[0138] At 882, a photomask is placed over the alignment layer. Figure 8D and 8E As described, there are two ways to enable AM ​​and PM to occur simultaneously within a single SLM device: one within an alignment unit, and the other within an array of alignment units. For ease of describing these two embodiments, the term "unit-based simultaneous modulation" refers to the simultaneous execution of AM and PM within a unit, i.e., the alignment unit is divided into two parts, one for AM and the other for PM, and this is the case for each alignment unit in a single SLM device. The term "array-based simultaneous modulation" means that all alignment units alternately perform AM or PM, i.e., adjacent alignment units perform different modulations.

[0139] At 884, process 880 determines how to design or configure the photomask via printing or photolithography. If it is determined that simultaneous cell-based modulation is to be implemented, process 880 proceeds to 886 where a corresponding pattern may be printed on the photomask. According to one embodiment, all cells in the array have the same pattern. According to another embodiment, from Figure 8E In this example, all cells in a row have the same pattern, while the adjacent row has a half-pixel shifted pattern, forming two alternating patterns across the row. If it is determined that array-based simultaneous modulation is to be implemented, process 880 proceeds to 888 where a corresponding pattern can be printed on a photomask. The pattern specifies that some cells are designated to perform one modulation (e.g., AM), and that cells adjacent to cells performing one modulation are designated to perform another modulation (e.g., PM). Figure 8D An exemplary portion of this pattern is shown.

[0140] Depending on the desired performance, the pattern may vary. Generally, the ratio of AM to PM within a cell is 50 / 50, but the ratio of AM to PM can be adjusted to any number as needed. Once the pattern is determined, the pattern can be imprinted onto a photomask. The details of making or imprinting the pattern onto the photomask will not be further described herein as it is well known in the art (e.g., in semiconductor manufacturing). Process 880 now proceeds to 890, where the photomask is etched. There are many ways to etch a photomask in the prior art. Again, the details of etching the photomask will not be further described herein as it is well known in the art (e.g., in semiconductor manufacturing). Because the alignment layer has designated alignment cells, an SLM device is created at 892 that performs both AM and PM within a cell, or an SLM device is created at 894 that performs both AM and PM within an array.

[0141] The present application has been described in terms of specific embodiments. Persons skilled in the art will understand that the present disclosure of implementations has been made by way of example and for purposes of illustration, various changes in the details of the embodiments of this disclosure can be made without departing from the spirit of the application. The scope of the application is to be limited only by the claims that follow.

Claims

1. A display device, characterized by comprising: It comprises: a pair of glasses frames; at least one integrated lens, wherein the integrated lens is framed in the glasses frames; a spatial light modulator to amplitude and phase modulate an optical image to produce a modulated image; and at least one holographic mirror to receive the modulated image and project the modulated image into the integrated lens after rotating the modulated image by 90 degrees, wherein the holographic mirror is optically coated to selectively allow or reflect specific wavelengths, and a user wearing the display device is able to see a hologram produced by the modulated image in the integrated lens. It further comprises a light source to illuminate the spatial light modulator, wherein the hologram is reflected light from the spatial light modulator.

2. The display device according to claim 1, wherein The light source is a uniform laser sheet, and the spatial light modulator comprises a microdisplay illuminated by the uniform laser sheet.

3. The display device according to claim 2, wherein The spatial light modulator comprises a first set of pixels to perform amplitude modulation; and a second set of pixels to perform phase modulation, wherein the first set of pixels and the second set of pixels are within a single array, the amplitude modulation and the phase modulation are performed by a liquid crystal layer sandwiched between a transparent electrode layer and a reflective electrode layer, wherein the reflective electrode comprises an array of pixel electrodes, each pixel electrode controls one pixel, and the reflective electrode is built on a silicon substrate.

4. The display device according to claim 2, wherein Within the single array, the first set of pixels and the second set of pixels are interleaved.

5. The display device according to claim 4, wherein The spatial light modulator further comprises a photomask on top of an alignment layer, the alignment layer is disposed above the liquid crystal layer, wherein the photomask has a pattern comprising an array of alignment cells, each alignment cell corresponds to one pixel, wherein a first set of the alignment cells are aligned in a first direction, and a second set of the alignment cells are aligned in a second direction.

6. The display device according to claim 5, wherein In the alignment layer, the first set of the alignment cells and the second set of the alignment cells are interleaved.

7. The display device according to claim 6, wherein The first set of the alignment cells causes phase modulation to light, and the second set of the alignment cells causes amplitude modulation to light.

8. The display device according to claim 6, wherein An image source is next to the temple, and projects the hologram into an edge of a waveguide.

9. The display device according to claim 2, wherein The image source is one end of a plurality of optical fibers, the optical fibers are encapsulated in or integrated with the temple.

10. The display device according to claim 9, wherein The optical fibers are part of the temple, and the other end of the optical fibers receives a sequence of optical images projected by a lens disposed in front of the spatial light modulator.

11. The display device according to claim 10, wherein A data image producing a two-dimensional optical image is at a first refresh rate and a first resolution, two consecutive two-dimensional optical images are displayed in the integrated lens, resulting in a combined synthetic optical image at a second refresh rate and a second resolution.

12. The display device according to claim 1, wherein The first refresh rate = 2 * the second refresh rate, and the first resolution = 1 / 2 * the second resolution.

13. The display device of claim 12, wherein, The two consecutive two-dimensional optical images from the optical fibers are used to produce a synthetic optical image viewed by a viewer of the display device.

14. The display device of claim 13, wherein, The method comprises:

15. A method for a display device, characterized by, providing a pair of glasses frames comprising at least one integrated lens and a temple attached to the glasses frames; receiving an optical image; amplitude and phase modulating the optical image in a spatial light modulator; ​ generating a hologram using light intensity reflected by the spatial light modulator illuminated by the uniform laser sheet; and projecting the hologram into the integrated lens via a mirror that is optically coated to selectively allow or reflect specific wavelengths through by a 90 degree rotation, a user wearing the display device is able to see the hologram in the integrated lens.

16. The method of claim 15, wherein, The spatial light modulator includes a microdisplay, the method further comprising: illuminating the uniform laser sheet onto the microdisplay; and amplitude and phase modulating the optical image from the uniform laser sheet.

17. The method of claim 16, wherein, The spatial light modulator includes a first set of pixels that perform amplitude modulation; and a second set of pixels that perform phase modulation, wherein the first set of pixels and the second set of pixels are within a single array, the amplitude modulation and the phase modulation are performed via a liquid crystal layer sandwiched between a transparent electrode layer and a reflective electrode layer, wherein the reflective electrode includes an array of pixel electrodes, each pixel electrode controls one pixel, the reflective electrode is built on a silicon substrate.

18. The method of claim 17, wherein, The first set of pixels are interleaved with the second set of pixels within the single array.

Images