Device with optically see-through 3D display and method for adapting device

Abstract

According to one embodiment, a method comprises: acquiring an operating condition (1072) of a wearable device, where the operating condition comprises a visual operating condition, an auditory operating condition, a positional operating condition, or an ambient operating condition, the wearable device comprising a memory and a three-dimensional display, the three-dimensional display being optically see-through; obtaining an operational preference of the wearable device from the memory of the wearable device (1074), where the operational preference includes a radius of curvature of an eye of a user of the wearable device; and calculating an exit angle and an intensity of light rays to be emitted by the three-dimensional display, the exit angle being calculated according to the operation preference of the wearable device, and the intensity being calculated according to the operation condition of the wearable device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims priority to U.S. Provisional Application No. 63 / 421,849, filed on November 2, 2022, the entire contents of which are incorporated herein by reference as if reproduced in full. Technical Field

[0002] The present invention generally relates to devices with three-dimensional (3D) displays, and in particular embodiments, to techniques and mechanisms for devices with optical see-through three-dimensional (3D) displays, and methods for adapting the devices. Background Art

[0003] The Metaverse is part of the next generation of the Internet, which is real-time, interactive, social, and persistent. The key technology of the Metaverse is augmented reality, including displays that provide real images superimposed with generated display items. Once such functions can be realized, optical combination displays can be used for augmented reality. As the interest in augmented reality has increased, the need for improved optical combination displays that support such functions has also emerged. Summary of the invention

[0004] Embodiments of the present invention describe a device having an optical see-through three-dimensional (3D) display, and a method for adapting the device, which generally achieve technical advantages.

[0005] According to one embodiment, a method includes: obtaining an operating condition of a wearable device, wherein the operating condition includes a visual operating condition, an auditory operating condition, a position operating condition or an environmental operating condition, and the wearable device includes a memory and a three-dimensional display, and the three-dimensional display is optically transparent; obtaining an operating preference of the wearable device from the memory of the wearable device, wherein the operating preference includes the radius of curvature of the eye of the user of the wearable device; calculating the exit angle and intensity of light to be emitted by the three-dimensional display, wherein the exit angle is calculated according to the operating preference of the wearable device, and the intensity is calculated according to the operating condition of the wearable device. In some embodiments of the method, calculating the exit angle and the intensity of the light is performed on the wearable device. In some embodiments of the method, calculating the exit angle and the intensity of the light is performed on an external device. In some embodiments, the method further includes: receiving the operating preference from the user of the wearable device. In some embodiments, the method further includes: displaying a three-dimensional image through the three-dimensional display. In some embodiments of the method, the operating preference further includes an indication of a preferred user interface, the preferred user interface corresponding to the operating condition, and the method further includes: adapting the user interface of the wearable device according to the operating condition and the operating preference of the wearable device. In some embodiments of the method, adapting the user interface of the wearable device includes: using a voice-based user interface when the operating condition indicates that the user of the wearable device is driving; and using a touchpad-based user interface when the operating condition indicates that the user of the wearable device is walking. In some embodiments of the method, adapting the user interface of the wearable device includes: using a touchpad-based user interface when the operating condition indicates that the user of the wearable device is in a crowded area; and using a voice-based user interface when the operating condition indicates that the user of the wearable device is in a quiet area.

[0006] According to another embodiment, a method includes: obtaining an operating condition of a wearable device; obtaining an operating preference of the wearable device; obtaining an application requirement of the wearable device; and adapting the operation of the wearable device according to the operating condition, the operating preference, and the application requirement of the wearable device. In some embodiments of the method, obtaining the operating condition of the wearable device includes: receiving an input signal from a visual sensor of the wearable device. In some embodiments of the method, obtaining the operating condition of the wearable device includes: receiving an input signal from an audio sensor of the wearable device. In some embodiments of the method, obtaining the operating condition of the wearable device includes: receiving an input signal from a position sensor of the wearable device. In some embodiments of the method, obtaining the operating condition of the wearable device includes: receiving an input signal from an environmental sensor of the wearable device. In some embodiments of the method, adapting the operation of the wearable device includes: adapting a processing system of the wearable device. In some embodiments of the method, adapting the operation of the wearable device includes: adapting the operation of a three-dimensional display of the wearable device. In some embodiments of the method, the three-dimensional display includes a light-emitting unit, and the operation of adapting the three-dimensional display includes: changing the emission angle of the light generated by the light-emitting unit. In some embodiments of the method, the operation of adapting the wearable device includes: adapting the user interface of the wearable device. In some embodiments of the method, the operation of adapting the wearable device includes: adapting the rendering of augmented reality content by the wearable device.

[0007] According to another embodiment, a wearable device includes: a processor; a non-transitory computer-readable storage medium for storing a program executed by the processor, the program including the following instructions: obtaining an operating condition of the wearable device; obtaining an operating preference of the wearable device; obtaining an application requirement of the wearable device; and adapting the operation of the wearable device according to the operating condition, the operating preference, and the application requirement of the wearable device. In some embodiments, the wearable device also includes a visual sensor, and the instructions for obtaining the operating condition of the wearable device include the following instructions: receiving an input signal from the visual sensor. In some embodiments, the wearable device also includes an audio sensor, and the instructions for obtaining the operating condition of the wearable device include the following instructions: receiving an input signal from the audio sensor. In some embodiments, the wearable device also includes a position sensor, and the instructions for obtaining the operating condition of the wearable device include the following instructions: receiving an input signal from the position sensor. In some embodiments, the wearable device also includes an environmental sensor, and the instructions for obtaining the operating condition of the wearable device include the following instructions: receiving an input signal from the environmental sensor. In some embodiments, the wearable device further includes a processing system, and the instructions for adapting the operation of the wearable device include the following instructions: adapt the processing system. In some embodiments, the wearable device further includes a three-dimensional display, and the instructions for adapting the operation of the wearable device include the following instructions: adapt the operation of the three-dimensional display. In some embodiments of the wearable device, the three-dimensional display includes a light-emitting unit, and adapting the operation of the three-dimensional display includes: changing the exit angle of light generated by the light-emitting unit. In some embodiments of the wearable device, the instructions for adapting the operation of the wearable device include the following instructions: adapt the user interface of the wearable device. In some embodiments of the wearable device, the instructions for adapting the operation of the wearable device include the following instructions: adapt the rendering of augmented reality content by the wearable device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more fully understand the present invention and its advantages, the following description is made with reference to the accompanying drawings, in which:

[0009] Figure 1A and Figure 1B is a view of glasses according to some embodiments;

[0010] Figure 2 is a block diagram of glasses according to some embodiments;

[0011] Figure 3is a flow chart of a method for displaying augmented reality content on a three-dimensional display according to some embodiments;

[0012] Figure 4 is a schematic diagram of a three-dimensional display during operation according to some embodiments;

[0013] Figure 5 is a schematic diagram of a three-dimensional display during display of a 3D image according to some embodiments;

[0014] Fig. 6A and Figure 6B is a view of a three-dimensional display according to some embodiments;

[0015] FIG. 7A to FIG. 7G is a top view of a three-dimensional display according to various embodiments;

[0016] Figures 8 to 11 is a cross-sectional view of an intermediate stage in manufacturing a three-dimensional display according to various embodiments;

[0017] Fig.12 is a cross-sectional view of an intermediate stage in manufacturing a three-dimensional display according to various embodiments;

[0018] Fig.13 is a flow chart of a method for manufacturing a three-dimensional display according to some embodiments;

[0019] Fig.14 A schematic diagram of an embodiment processing system is shown;

[0020] Fig.15 is a block diagram of a processing system for a wearable device according to some embodiments;

[0021] Fig.16 is a block diagram of a processing system for a wearable device according to some embodiments;

[0022] Fig.17 is a flow chart of a method for adapting operation of a wearable device according to some embodiments.

[0023] Unless otherwise indicated, corresponding numerals and symbols in the different figures generally refer to corresponding parts. The figures are drawn to clearly illustrate the relevant aspects of the embodiments and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0024] The manufacture and use of embodiments of the present invention will be discussed in detail below. However, it should be understood that the concepts disclosed herein can be embodied in a variety of specific contexts, and the specific embodiments discussed herein are merely illustrative and are not intended to limit the scope of the claims. In addition, it should be understood that various changes, substitutions and modifications may be made to this document without departing from the spirit and scope of the present invention as defined by the appended claims.

[0025] According to various embodiments, the device includes a three-dimensional display. In some embodiments, the device is a wearable device. The three-dimensional display can be used as an optical combination display to display augmented reality content and ordinary images that can be viewed to a user of the device. The three-dimensional display is composed of microlenses of specific sizes and spacings, so that the display has an optical see-through (OST) function while still having the required display resolution. The microlenses are arranged at intervals so that there are gaps or spaces between the microlenses, so that the user of the device perceives that the display is transparent. In addition, the three-dimensional display can be a direct light-emitting display, which can be more compact than other types of displays.

[0026] Figure 1A and Figure 1B is a view of glasses 100 according to some embodiments. Specifically, Figure 1A is a front view of the glasses 100, Figure 1B 1 is a side view of glasses 100. Glasses 100 are wearable devices with an optical see-through three-dimensional (3D) display. Glasses 100 include a frame 102, a 3D display 104, a sensor 106, and a human-machine interface device 108. As described in more detail later, the 3D display 104 is an optical see-through display that can be used as an optical combination display. The 3D display 104 is a lens for the glasses 100. The human-machine interface device 108 (described below) is used to interact with the glasses 100. Optionally, the glasses 100 include other components such as a speaker 110. Although not shown in FIG. Figure 1A and Figure 1B 100, but the glasses 100 also include a controller 112 and a transceiver 114 (see Figure 2 In one embodiment, the controller 112 and the transceiver 114 are disposed together with the human-machine interface device 108. Alternatively, they may be disposed in any portion of the frame 102 or connected to the frame 102.

[0027] The frame 102 is a spectacle frame that holds the lens (e.g., the 3D display 104) in place for the user (e.g., the wearer) of the glasses 100. The frame 102 includes: a pair of frames that surround or at least partially surround the 3D display 104 and hold the 3D display 104 in place, a nose bridge that connects the frames to each other, and ear frames connected to the sides of the frames. The frame 102 can be made of any acceptable material, such as plastic, metal, a combination thereof, etc.

[0028] The three-dimensional display 104 is coupled to the frame 102. Specifically, the three-dimensional display 104 is held by the frame of the frame 102 so that the frame 102 extends at least partially around the three-dimensional display 104. The three-dimensional display 104 is a direct-illuminated optical see-through three-dimensional display. Specifically, the three-dimensional display 104 is an optical see-through light field display. In this case, when the user of the glasses 100 perceives that the display is transparent at a desired viewing distance, the display is optically see-through. When the real-world environment is visible to the user of the glasses 100 through the three-dimensional display 104, the three-dimensional display 104 is perceived as transparent. The three-dimensional display 104 can be used to display augmented reality overlays, user interfaces (UIs), and other content to the user of the glasses 100.

[0029] Since the real world environment is visible through the three-dimensional display 104, the user of the glasses 100 can perceive the augmented reality content superimposed on the real world environment visible through the three-dimensional display 104. In addition, in this case, when the display directly emits the required light field without using a backlight for light emission, the display is a direct-emitting display. Specifically, the display emits light with the required intensity and color from each pixel to generate an image directly on the display. A direct-emitting display may be simply referred to as an emissive display.

[0030] The sensor 106 includes one or more of the following: a visual sensor, an audio sensor, a position sensor, an environmental sensor, etc. Examples of the sensor 106 include cameras such as a wide-angle camera and an infrared camera; depth sensors; position sensors such as an orientation sensor and a magnetometer; motion sensors such as an accelerometer, a gravity sensor, and a gyroscope; environmental sensors such as a light sensor, a temperature sensor, a humidity sensor, and an air pressure sensor; audio sensors such as a microphone; medical sensors such as a blood oxygen sensor, an electroencephalogram sensor, a pulse oximeter sensor, and an optical heart rate sensor; and satellite navigation sensors such as a global positioning system (GPS) sensor. Some or all of the sensors 106 are integrated or disposed in the frame 102. In some embodiments, the sensor 106 is disposed in the frame and the bridge of the nose of the frame 102. The sensor 106 may be disposed symmetrically or asymmetrically around the frame 102, and may be facing and / or facing away from the user of the glasses 100. For example, the sensor 106 may include a camera facing the user of the glasses 100, which may be used to track the position of the user's head, face, and / or eyes. Similarly, the sensors 106 may include a camera facing away from the user of the glasses 100, which may be used to track the user's gestures. Some of the sensors 106 may be networked sensors external to the frame 102, which are connected via the transceiver 114 (which is then connected to the Figure 2 ) communicates with framework 102.

[0031] The human interface device 108 includes features for interacting with the user interface of the glasses 100. The human interface device 108 may include a touch pad, buttons, etc. In some embodiments, the human interface device 108 is connected to the frame 102, for example, to the ear bracket of the frame 102. In some embodiments, the human interface device 108 is a networked human interface device external to the frame 102, which is connected to the frame 102 through the transceiver 114 (which is then connected to the frame 102). Figure 2 ) communicates with framework 102.

[0032] The speaker 110 is used to output information to the user of the glasses 100. For example, the speaker 110 can be used to play notifications or alarms. In an alternative embodiment, if the transceiver 114 (which is then Figure 2 ) with a remote speaker (e.g., using, for example, but not limited to The communication protocol In this alternative embodiment, instead of sending audio data or information to the speaker 110, the controller 112 sends data to the transceiver 114 for wireless transmission.

[0033] Figure 2is a block diagram of glasses 100 according to some embodiments. As described above, glasses 100 also include a controller 112 and a transceiver 114.

[0034] The controller 112 is used to control the components of the glasses 100 during operation. Specifically, the controller 112 is used to control the components of the glasses 100 by receiving input signals from input devices (e.g., sensors 106) of the glasses 100 and sending output signals to output devices (e.g., three-dimensional display 104 and speakers 110) of the glasses 100. The controller 112 may include control circuits, processors, application specific integrated circuits, microcontrollers, etc. For example, the controller 112 may include one or more processors and memories, such as non-transitory computer-readable storage media storing programs for execution by the processors. One or more modules within the controller 112 may be partially or fully embodied as software and / or hardware for performing any of the functions described herein. As described in more detail subsequently, the controller 112 is used to display augmented reality content on the three-dimensional display 104.

[0035] The transceiver 114 is used for the controller 112 to communicate with external devices. The transceiver 114 is used to send and receive signaling on the network. For example, the transceiver 114 can be used to communicate with an external device that assists the controller 112 in processing or receiving information for generating an image of the three-dimensional display 104. The transceiver 114 may include a transmitter and a receiver for wireless communication protocols, such as cellular protocols (e.g., 5G, long-term evolution (LTE), etc.), wireless local area network (WLAN) protocols (e.g., Wi-Fi, etc.), or any other type of wireless protocol (e.g., Bluetooth, near field communication (NFC), etc.). In these embodiments, the transceiver 114 includes one or more antennas / radiating units for sending and / or receiving communication signals.

[0036] Figure 3 is a flow chart of a method 300 for displaying augmented reality content on a three-dimensional display according to some embodiments. Figures 1A to 2 Method 300 is described. In this case, the augmented reality content includes a 3D image displayed on the three-dimensional display 104 such that the 3D image is superimposed on a real world environment visible through the three-dimensional display 104. The augmented reality content is spatially aligned with the real world environment visible through the three-dimensional display 104.

[0037] In step 302, a signal from an input device (e.g., sensor 106) of the glasses 100 is analyzed. The signal may be a sensor signal received from the sensor 106. Information obtained by analyzing the signal from the input device may be spatial alignment information for spatially aligning augmented reality content on the three-dimensional display 104. For example, the position of the head, face, and / or eyes of a user of the glasses 100 may be determined by analyzing the signal from the sensor 106 (e.g., a camera facing the user of the glasses 100).

[0038] In step 304, the augmented reality content is rendered according to the analyzed signal. Rendering the augmented reality content includes calculating the position of displaying the 3D image on the three-dimensional display 104 so that the 3D image is spatially aligned with the real world environment. For example, the spatial alignment information can be used to render the augmented reality content at a desired position on the three-dimensional display 104.

[0039] In step 306, the rendered augmented reality content is output to the three-dimensional display 104. By sending a signal to the three-dimensional display 104, a 3D image can be output to the three-dimensional display 104. Therefore, the three-dimensional display 104 displays the 3D image.

[0040] At least a portion of method 300 may be performed by controller 112. In some embodiments, each step of method 300 is performed by controller 112. In some embodiments, some steps of method 300 are performed by controller 112, and other steps of method 300 are performed by external devices, such as external processors (e.g., servers), cloud computing processors, edge computing processors, etc. For example, rendering augmented reality content may require considerable computing power. In some embodiments, steps 302 and 306 are performed by a front-end processor (e.g., controller 112), while step 304 is performed by a back-end processor (e.g., an external device). The front-end processor may communicate with the back-end processor via transceiver 114.

[0041] Furthermore, step 302 may be performed only in certain circumstances. For example, a calibration process may be performed using an input device of the glasses 100 (e.g., the sensor 106, specifically, a camera facing a user of the glasses 100). The calibration process may generate spatial alignment information for spatially aligning augmented reality content on the three-dimensional display 104. The spatial alignment information may be stored in a memory of the controller 112 and then reused for multiple rendering steps without repeatedly recalculating the spatial alignment information.

[0042] Figure 4is a schematic diagram of a three-dimensional display 104 during operation according to some embodiments. The three-dimensional display 104 is a direct-light optical see-through three-dimensional display. The three-dimensional display 104 is used to display a 3D image to a target 400. In this example, the three-dimensional display 104 is a spectacle lens and the target 400 is the user's eye.

[0043] The three-dimensional display 104 is used to display a 3D image to the target 400. Specifically, the three-dimensional display 104 is a light field display. The three-dimensional display 104 includes a plurality of 3D light emitting units 402. As described in more detail later, the 3D light emitting unit 402 includes a micro lens 604 (later described in detail). Fig. 6A and Figure 6B ) and LED array 614 (later described in Fig. 6A and Figure 6B ), the LED array 614 includes a cluster of LEDs. Each 3D light emitting unit 402 emits one or more light fields. In this case, the light field is described by a vector function that describes the amount of light flowing through multiple points in space in multiple directions. Specifically, the light field includes multiple light rays 404 defined by a full light function. Each light ray 404 has radiance, which is a measurement of the amount of light propagating along the light ray 404. In other words, each 3D light emitting unit 402 emits multiple light rays 404. In the example shown, only a portion of the light rays 404 radiated toward the target 400 are shown. The light rays 404 all propagate in a direction that forms an acute angle with a direction perpendicular to the main surface of the three-dimensional display 104. Therefore, the appearance of three-dimensional depth can be formed by the light rays 404 emitted by the 3D light emitting unit 402 (for example, through a combination of the LED array 614 and the microlens 604).

[0044] The 3D light emitting unit 402 is self-luminous and does not use a backlight for light emission. Therefore, the three-dimensional display 104 is a direct light emitting display. Using a direct light emitting display instead of a backlight display can reduce the size of the three-dimensional display 104 and can also enhance the brightness and contrast of the three-dimensional display 104. The 3D light emitting unit 402 also does not use a projector or waveguide for light emission. The power consumption of a direct light emitting display may be lower than that of a backlight display or a waveguide display.

[0045] The three-dimensional display 104 is also an optical see-through display. Therefore, the object 406 can be visible to the target 400 through the three-dimensional display 104. The 3D light emitting units 402 are arranged in a spaced relationship from each other so that each 3D light emitting unit 402 is separated from the other 3D light emitting units 402. As described in more detail later, the ratio of the distance between the 3D light emitting units 402 and the size of the 3D light emitting units 402 is a specific ratio that allows light 408 from the object 406 to pass through the three-dimensional display 104. Therefore, the three-dimensional display 104 can display a 3D image to the target 400 while also allowing the light 408 to pass through the three-dimensional display 104 and be visible to the target 400. Therefore, the augmented reality content displayed by the three-dimensional display 104 can be spatially aligned with the real world environment (e.g., the object 406) visible through the three-dimensional display 104.

[0046] Figure 5 is a schematic diagram of a three-dimensional display during display of a 3D image according to some embodiments. When displaying a 3D image through a wearable device (e.g., glasses 100), different three-dimensional displays 104 are used to display different light fields to different targets 400 (e.g., to different eyes of a user). In this embodiment, the three-dimensional display 104 is a near-eye display. The light field is generated by the three-dimensional display 104 so that the user perceives a virtual object at a display position 502 in three-dimensional space.

[0047] The clarity of the displayed 3D image is determined by the pixel density of the three-dimensional display 104. If the pixel density of the three-dimensional display 104 is insufficient to display the 3D image, the displayed position 502 of the virtual object may be different from the expected position 504 of the virtual object. When the error between the displayed position 502 of the virtual object and the expected position 504 is large, the augmented reality content may not be correctly superimposed on the real world environment visible through the three-dimensional display 104. Specifically, the displayed augmented reality content may be high-order grating diffraction patterns, which appear as ghosts to the user.

[0048] The pixel density of the three-dimensional display 104 (measured in pixels per degree (PPD)) is determined by the density of the 3D light emitting units 402 and the viewing distance between the three-dimensional display 104 and the target 400. In some embodiments where the three-dimensional display 104 is a near-eye display, the pixel density of the three-dimensional display 104 is in the range of 1 PPD to 10 PPD. A pixel density less than 1 PPD may not be sufficient to display a 3D image, resulting in a large error between the displayed position 502 and the expected position 504 of the virtual object. A pixel density greater than 10 PPD may be difficult to manufacture. In an embodiment where the viewing distance is about 10 mm and the size of the 3D light emitting unit 402 is about 50 μm, the pixel density of the three-dimensional display 104 is about 3 PPD.

[0049] Fig. 6A and Figure 6B is a view of the three-dimensional display 104 according to some embodiments. Specifically, Fig. 6A is a top view of a portion of the three-dimensional display 104, Figure 6B is a cross-sectional view of a portion of the 3D display 104. The 3D display 104 includes a display panel 602 and a plurality of micro lenses 604.

[0050] Display panel 602 Fig. 6A and Figure 6B Schematically shown in FIG. 1 , where some features are omitted for clarity of illustration (later in FIG. Figures 8 to 12 ). The display panel 602 includes a transparent layer 612, a plurality of light emitting diode arrays 614 located in / on the transparent layer 612, and a plurality of transistor arrays 616 located below the light emitting diode arrays 614.

[0051] The transparent layer 612 at least laterally surrounds the light emitting diode array 614. The transparent layer 612 can be a glass layer, a UV absorbing layer, a liquid crystal display layer, etc. The material of the transparent layer 612 can be selected based on the desired application of the three-dimensional display 104. When the three-dimensional display 104 is the lens of transparent glasses, the glass layer can provide good transparency. The UV absorbing layer is a photochromic layer that can change the transmittance according to the brightness of the ambient light. When the three-dimensional display 104 is the lens of sunglasses, the UV absorbing layer can provide good sun protection. The liquid crystal display layer can be programmable so that its transmittance can be changed as needed. For example, the liquid crystal display layer can be programmed to have high transparency in a low light environment, and can be programmed to have low transparency in a high light environment. The controller 112 (see Figure 2 ) can be used to program the liquid crystal display layer.

[0052] The light emitting diode array 614 is an array of light emitting pixels, and each array includes any number of light emitting diodes. In one embodiment, each light emitting diode array 614 includes a grid having 10 to 15 columns of light emitting diodes and 10 to 15 rows of light emitting diodes. A light emitting diode refers to a diode that emits light of a desired color without backlight during operation. Examples of suitable light emitting diodes include organic light-emitting diodes (OLED), micro light-emitting diodes (μLED), etc. The light emitting diode arrays 614 are arranged at intervals from each other. In some embodiments, the intervals between the light emitting diodes of adjacent light emitting diode arrays 614 are greater than the intervals between the light emitting diodes in the light emitting diode array 614.

[0053] The microlenses 604 are placed on the display panel 602. Specifically, the microlenses 604 are located on the light-emitting diode array 614 in a one-to-one correspondence. Therefore, each microlens 604 overlaps with the light-emitting diode array 614 below. The microlens 604 can be narrower, wider, or the same width as the light-emitting diode array 614 below. The microlens 604 can be a hemispherical lens, a spherical lens, an aspherical lens, a flat lens, a holographic lens, a metal lens (e.g., a nano-sized meta-curved lens), a Fresnel lens, etc. In addition, the microlens 604 can be a single-layer lens, a multi-element lens, etc. The microlenses 604 are arranged at intervals from each other so that the microlenses 604 do not contact each other in at least one direction. In the top view, the intervals between the microlenses 604 expose part of the transparent layer 612.

[0054] The transistor array 616 is located below the light emitting diode array 614. Specifically, the transistor array 616 is located below the light emitting diode array 614 in a one-to-one correspondence. The transistor arrays 616 are arranged in a spaced relationship from each other. The transistor array 616 includes transistors that control the diodes of the corresponding superimposed light emitting diode array 614. The transistor array 616 is formed on an opaque layer (e.g., a semiconductor layer that is opaque to visible light) that prevents light from passing through the portion of the three-dimensional display 104 where the microlens 604 is located. The microlens 604 distorts (e.g., refracts) light. Preventing light from passing through the three-dimensional display 104 where the microlens 604 is located helps reduce the distortion of light passing through the three-dimensional display 104.

[0055] Each 3D light emitting unit 402 includes a microlens 604, a light emitting diode array 614, and a transistor array 616. The microlens 604 refracts light emitted by the light emitting diode array 614. The light emitted from the 3D light emitting unit 402 can be directed to a desired direction by irradiating certain diodes of the light emitting diode array 614, so that the microlens 604 refracts the light emitted by the diode in the desired direction. The relative position between the light emitting diode and the center of the microlens 604 defines the exit angle of the light emitted from the microlens 604.

[0056] The display panel 602 has a plurality of emission regions 602E and a plurality of transparent regions 602T. The emission regions 602E are opaque regions where ambient light cannot pass through the display panel 602. Instead, the emission regions 602E emit light. An array of light-emitting diodes 614 and an array of transistors 616 are formed in the emission regions 602E. The microlenses 604 completely overlap the emission regions 602E. The transparent regions 602T are regions where light can pass through the display panel 602. The transparent regions 602T are located between adjacent emission regions 602E. The transparent layer 612 is formed in the transparent regions 602T. The microlenses 604 do not overlap the transparent regions 602T.

[0057] Although the emissive region 602E is opaque (e.g., opaque to light), the transparent region 602T is sufficiently large relative to the emissive region 602E so that the display panel 602 is optically see-through. The separation distance D between the microlenses 604 is larger (in at least one direction) relative to the size S of the microlenses 604. Specifically, the separation distance D between the microlenses 604 is greater than the size S of the microlenses 604. In some embodiments, the size S of the microlenses 604 is in the range of 20 μm to 50 μm, the separation distance D between the microlenses 604 is in the range of 30 μm to 100 μm, and the pitch P of the microlenses 604 is in the range of 100 μm to 150 μm. In some embodiments, the size S of the microlenses 604 is in the range of 5 μm to 2000 μm (e.g., 200 μm to 2000 μm), the separation distance D between the microlenses 604 is in the range of 200 μm to 2000 μm, and the pitch P of the microlenses 604 is in the range of 400 μm to 4000 μm. The size S, the separation distance D, and the pitch P may also take other values, especially when the technology scales down. The ratio of the microlens separation distance D to the microlens size S determines whether the three-dimensional display 104 is optically see-through. In some embodiments, the ratio of the microlens separation distance D to the microlens size S is in the range of 1 to 6, more specifically, in the range of 2 to 5. If the ratio of the microlens separation distance D to the microlens size S is less than 2, scattering or diffraction may occur, and the three-dimensional display 104 may display ghost images. If the ratio of the microlens separation distance D to the microlens size S is greater than 5, the pixel density of the three-dimensional display 104 may exceed the desired range (as previously described) and be insufficient to display a 3D image.

[0058] In the top view, the microlenses 604 occupy a small portion (e.g., less than half) of the area of ​​the display panel 602. Therefore, the total area of ​​the transparent regions 602T is greater than the total area of ​​the emissive regions 602E. In some embodiments, the ratio of the total area of ​​the transparent regions 602T to the total area of ​​the emissive regions 602E is in the range of 1 to 6. In some embodiments, the ratio is about 2, so that the emissive regions 602E occupy about one-third of the area of ​​the display panel 602, while the transparent regions 602T occupy about two-thirds of the area of ​​the display panel 602.

[0059] FIG. 7A to FIG. 7G 604 according to various embodiments. In the top view, the microlens 604 can have various shapes. In addition, in the top view, the microlens 604 can have various layouts. It should be understood that the light emitting diode array below the microlens 604 has the same layout as the microlens 604.

[0060] In some embodiments, Fig. 7AAs shown, the microlenses 604 are circular microlenses and are arranged in a grid microlens array. In the grid microlens array, the microlenses 604 are aligned along rows and columns in a top view. The spacing between the rows of microlenses 604 can be different from the spacing between the columns of microlenses 604. In some embodiments, the columns of microlenses 604 are closer than the rows of microlenses 604, so that the microlenses 604 are in a dense horizontal line. The spacing between the rows / columns of microlenses 604 determines the field of view of the three-dimensional display 104. For example, in the example shown where the columns of microlenses 604 are closer than the rows of microlenses 604, the horizontal field of view of the three-dimensional display 104 can be greater than the vertical field of view of the three-dimensional display 104.

[0061] In some embodiments, Figure 7B As shown, the microlenses 604 are circular microlenses and are arranged in a checkerboard microlens array. In the checkerboard microlens array, the microlenses 604 are aligned along the diagonal direction in a top view. When the microlenses 604 are in a checkerboard layout, the horizontal field of view of the three-dimensional display 104 can be equal to the vertical field of view of the three-dimensional display 104.

[0062] In some embodiments, Figure 7C As shown, the microlenses 604 are rectangular microlenses (e.g., square microlenses) and are arranged in microlens strips, wherein each microlens strip is arranged at intervals. In the microlens strips, the microlenses 604 are aligned in the same direction and densely packed along the microlens strips. The microlenses 604 within the microlens strips can be closer to each other than previously described, and in some embodiments, the microlenses 604 within the microlens strips are not arranged at intervals, but are in contact with each other.

[0063] Other acceptable microlens shapes may be used. For example, microlens 604 may have other polygonal shapes. Similarly, microlens 604 may have non-polygonal shapes. For example, microlens 604 may be a truncated circular microlens.

[0064] In some embodiments, Fig.7D As shown, the microlenses 604 are rectangular microlenses and are arranged in microlens strips, wherein some of the microlens strips are not arranged at intervals, but are in contact with each other. For example, the microlens strips can be grouped so that multiple microlens strips extend along each other and contact each other, and each group of microlens strips is arranged at intervals. When the microlens strips are grouped, the intervals between each group of microlens strips can be larger. For example, the width of each microlens strip can be about 33 μm, and three microlens strips can be grouped along each row, so that the total row width is about 100 μm, and the separation distance between the rows can be about 250 μm.

[0065] In some embodiments, Fig. 7EAs shown, microlenses 604 are hexagonal microlenses and are arranged in microlens strips, wherein some microlens strips are not arranged at intervals, but are in contact with each other. The hexagonal microlenses can be grouped into microlens strips with a honeycomb layout. The honeycomb structures are arranged at intervals. In this embodiment, the honeycomb structure is non-truncated. Therefore, in a top view, the edges of the hexagonal microlenses define the jagged edges of the microlens strips.

[0066] In some embodiments, Figure 7F As shown, the microlenses 604 are hexagonal microlenses and are arranged in a microlens strip having a honeycomb layout, wherein the honeycomb structure is truncated. Therefore, in a top view, the edges of the hexagonal microlenses define the straight edges of the microlens strip.

[0067] The microlens 604 may be arranged along a first direction (eg, Fig. 7A and FIG. 7C to FIG. 7F in the horizontal direction) than in the second direction (e.g., Fig. 7A and FIG. 7C to FIG. 7F In some embodiments, the microlens separation distance along the first direction is less than the desired separation distance (previously in Fig. 6A and Figure 6B ). The microlenses 604 may not be spaced apart in the first direction. However, the microlens separation distance along at least one direction (e.g., the second direction) is within a desired separation distance, so that the three-dimensional display 104 is optically see-through. The horizontal field of view of the human eye is greater than the vertical field of view. Therefore, in some embodiments, the microlenses 604 are spaced apart at a desired separation distance along the vertical direction, so that the gaps between the microlenses 604 are horizontally oriented, for example, oriented in the direction with a larger field of view.

[0068] The ratio of the microlens separation distance to the microlens size may vary across the three-dimensional display 104. For example, the ratio may have a first value in a first area of ​​the three-dimensional display 104, and may have a second value in a second area of ​​the three-dimensional display 104, wherein the first value is different from the second value, and both the first value and the second value are within the range previously described. In some embodiments, the ratio in a center portion of the three-dimensional display 104 is smaller than the ratio in an edge portion of the three-dimensional display 104, wherein the center portion is located between the edge portions. The edge portions may include an upper portion and a lower portion of the three-dimensional display 104. In other words, the separation distance between the microlenses 604 may be smaller in the center portion of the three-dimensional display 104 than in the edge portions of the three-dimensional display 104. FIG. 7C to FIG. 7F In some embodiments, the microlens strips in the center portion of the 3D display 104 may be closer together than the microlens strips in the edge portion of the 3D display 104 .

[0069] Other acceptable microlens layouts may be used. In some embodiments, Figure 7G As shown, the microlenses 604 are arranged in a random microlens array. Specifically, the spacing between adjacent microlenses 604 can be random. In the random microlens array, the ratio of the microlens separation distance of adjacent microlenses 604 to the microlens size can still be within the range described previously. In other words, the spacing between adjacent microlenses 604 is variable, but the spacing is within the range described previously. Randomizing the spacing between adjacent microlenses 604 can help reduce interference with light passing through the space between the microlenses 604. Although Figure 7G A random microlens array having circular microlenses is shown, but the microlenses 604 of the random microlens array may be rectangular microlenses, hexagonal microlenses, or the like.

[0070] Figures 8 to 11 is a cross-sectional view of an intermediate stage of manufacturing the three-dimensional display 104 according to various embodiments. The three-dimensional display 104 is manufactured by separately manufacturing the light emitting diode array 614 and then transferring the light emitting diode array 614 to the transparent layer 612.

[0071] exist Figure 8 , a thin film transistor layer 802 is formed. A light emitting diode layer 804 is formed on the thin film transistor layer 802. The thin film transistor layer 802 includes transistors, which can be arranged in the transistor array 616 and formed on an opaque layer such as a semiconductor substrate. The light emitting diode layer 804 includes light emitting diodes, which can be arranged in the light emitting diode array 614. The transistors of the thin film transistor layer 802 and the diodes of the light emitting diode layer 804 can be formed by an acceptable complementary metal oxide semiconductor (CMOS) process. For example, deposition, lithography, and etching processes can be performed to form various features (e.g., buffer layers, channel layers, pixel definition layers, cathodes, etc.) for the transistors of the thin film transistor layer 802 and the diodes of the light emitting diode layer 804.

[0072] The microlenses 604 are then placed on the LED layer 804. As an example of placing the microlenses 604, the microlenses 604 can be placed on a microlens sheet 806. The microlens sheet 806 can be formed of glass, etc. The microlens sheet 806 can then be placed on the LED layer 804 and can be aligned with the LED layer 804 so that the microlenses 604 are aligned with the corresponding underlying LED array 614 of the LED layer 804.

[0073] exist Fig. 9In the embodiment of the present invention, the microlens sheet 806, the light emitting diode layer 804 and the thin film transistor layer 802 are cut to form a display component 808. The display component 808 includes a transistor array 616 (part of the thin film transistor layer 802), a light emitting diode array 614 (part of the light emitting diode layer 804) and microlenses 604. ... Fig. 7A and Figure 7B ), each display element 808 is a single unit including a single transistor array 616, a single LED array 614, and a single microlens 604. In some embodiments using microlens strips (see FIG. 7C to FIG. 7F ), each display assembly 808 is a strip including a plurality of transistor arrays 616, a plurality of LED arrays 614, and a plurality of microlenses 604. A cutting process is performed along a scribe line region (e.g., between display assemblies 808). The cutting process may include performing a sawing process, a laser cutting process, etc. The cutting process separates adjacent display assemblies 808. The microlenses 604 may be trimmed by cutting the display assemblies 808. After the cutting process, the corresponding LED arrays 614, transistor arrays 616, and microlenses 604 are laterally coterminal.

[0074] exist Fig.10 In the embodiment of the present invention, a transparent layer 612 is formed. Then, the recess 810 is patterned in the transparent layer 612. The recess 810 can be patterned using acceptable photolithography and etching techniques. Fig. 7A and Figure 7B ), the recess 810 is a pit. FIG. 7C to FIG. 7F ), the concave portion 810 is a groove. The concave portion 810 is patterned at the location of the emission region 602E of the display panel 602.

[0075] exist Fig.11 In the embodiment, the display component 808 is transferred to the transparent layer 612 to form the three-dimensional display 104. After the transfer, the display component 808 is coupled to the transparent layer 612. By placing the display component 808 in the recess 810 (see Fig.10), the display assembly 808 can be transferred to the transparent layer 612. The display assembly 808 can be placed in the recess 810 by an acceptable pick and place process. Therefore, various portions of the display assembly 808 are disposed in the transparent layer 612. In some embodiments, the transistor array 616 and the light emitting diode array 614 are disposed in the transparent layer 612. Other portions of the display assembly 808 protrude from the transparent layer 612. In some embodiments, the microlenses 604 protrude from the transparent layer 612. Electrical connections to devices of the display assembly 808 (e.g., transistors of the transistor array 616) can be formed on the transparent layer 612.

[0076] In this embodiment, the microlenses 604 are placed on the LED array 614 before the display assembly 808 is placed in the recess 810 (e.g., before the display assembly 808 is cut). In another embodiment, the microlenses 604 are placed on the LED array 614 after the display assembly 808 is placed in the recess 810 (e.g., after the display assembly 808 is cut). In this embodiment, the microlens sheet 806 (see FIG. 1 ) including the microlenses 604 is Figure 8 ) can be placed on the transparent layer 612 and the display component 808.

[0077] In some embodiments, one or more surfaces of the microlens 604 are coated. The sidewalls 812 of the microlens 604 may be coated with a reflective coating, such as a metal coating, which may help block light from passing obliquely through the transparent region 602T and the microlens 604. Otherwise, such light would be deflected by the microlens 604. The circular top surface 814 of the microlens 604 may be coated with an anti-reflective coating, such as a dielectric multilayer coating, which may help increase the light transmittance through the microlens 604.

[0078] Fig.12 is a cross-sectional view of an intermediate stage in manufacturing a three-dimensional display 104 according to various embodiments. Fig.12 The steps of 804 are similar to those of 805 , except that the display assembly 808 is placed on the top surface of the transparent layer 612 instead of being placed in a recess in the transparent layer 612. When the transparent layer 612 is a layer that cannot be patterned without causing damage (e.g., a liquid crystal display layer), the recess in the transparent layer 612 can be omitted.

[0079] Fig.13 is a flow chart of a method for manufacturing a three-dimensional display according to some embodiments. The display may be a direct-emitting display for a wearable device. The method may use Figures 8 to 12 This can be done by following the appropriate steps of the process described in .

[0080] In step 1302, the thin film transistor layer and the light emitting diode layer are cut to form a first display component and a second display component. The first display component includes a first transistor array and a first light emitting diode array. The second display component includes a second transistor array and a second light emitting diode array.

[0081] In step 1304, the first display assembly and the second display assembly are transferred to the transparent layer. In some embodiments, transferring the first display assembly and the second display assembly to the transparent layer includes patterning the first recess and the second recess in the transparent layer and placing the first display assembly and the second display assembly in the first recess and the second recess, respectively. In some embodiments, transferring the first display assembly and the second display assembly to the transparent layer includes placing the first display assembly and the second display assembly on a top surface of the transparent layer.

[0082] In step 1306, a first microlens and a second microlens are placed on the first light emitting diode array and the second light emitting diode array, respectively. The first microlens and the second microlens may be placed before the first display assembly and the second display assembly are transferred to the transparent layer. In this embodiment, placing the first microlens and the second microlens includes: placing a microlens sheet on the light emitting diode layer, the microlens sheet being cut together with the light emitting diode layer. The first microlens and the second microlens may be placed after the first display assembly and the second display assembly are transferred to the transparent layer. In this embodiment, placing the first microlens and the second microlens includes: placing the microlens sheet on the transparent layer, the first microlens and the second microlens being aligned with the first light emitting diode array and the second light emitting diode array, respectively.

[0083] Fig.14 A block diagram of an embodiment processing system 1300 for performing the methods described herein is shown, which processing system can be installed in a host device. For example, the processing system 1300 can be used to implement the controller 112 (see Figure 2 As shown, the processing system 1300 may or may not include: Fig.14The processor 1404, memory 1406 and interfaces 1410 to 1414 are arranged as shown. The processor 1404 can be any component or collection of components for performing calculations and / or other processing related tasks. The memory 1406 can be any component or collection of components for storing programs and / or instructions for execution by the processor 1404. In one embodiment, the memory 1406 includes a non-transitory computer-readable medium. The interfaces 1410, 1412, 1414 can be any component or collection of components that enable the processing system 1300 to communicate with other devices / components and / or users. For example, one or more of the interfaces 1410, 1412, 1414 can be used to send data, control or management messages from the processor 1404 to applications installed on a host device and / or a remote device. For another example, one or more of the interfaces 1410, 1412, 1414 can be used to enable a user or user device (e.g., a personal computer (PC), etc.) to interact / communicate with the processing system 1300. The processing system 1300 may include Fig.14 Additional components not shown, such as long-term storage (e.g., non-volatile memory, etc.).

[0084] Embodiments can achieve advantages. Placing the microlenses 604 on the display panel 602 can cause the generated 3D light emitting unit 402 to emit a light field suitable for displaying a 3D image. Utilizing a specific ratio of the microlens separation distance D to the microlens size S (previously described) can cause the three-dimensional display 104 to be optically see-through. Because the three-dimensional display 104 of the glasses 100 is a direct-emitting display, these displays can be more compact than other types of displays (e.g., backlit displays and waveguide combination displays). Avoiding the use of a backlight in the glasses 100 can reduce the power consumption of the glasses 100. The three-dimensional display 104 is directly emitted and has a high light emission efficiency, which can reduce power consumption to 10mW. In addition, the direct-emitting display has better light transmission efficiency than the waveguide combination display. Avoiding the use of a waveguide combination display in the glasses 100 can reduce the user's exposure to excessively bright light.

[0085] Fig.15 is a block diagram of a processing system of glasses 100 according to some embodiments. Figure 3 The processing system may be used when performing the method 300 described in the embodiment of the present invention. The processing system includes a backend processing module 1502 .

[0086] The back-end processing module 1502 is used to display augmented reality content on the three-dimensional display 104. Specifically, the back-end processing module 1502 is used to receive input signals from the sensor 106 and the human-machine interface device 108, and send output signals to the speaker 110 and / or the three-dimensional display 104. The back-end processing module 1502 includes a sensor data processing and analysis module 1504, an augmented reality rendering module 1506, a playback module 1508, and a user behavior and preference analysis module 1510.

[0087] The sensor data processing and analysis module 1504 is used to receive input signals from the sensor 106 and the human interface device 108 and process the input signals. In addition, the sensor data processing and analysis module 1504 can be used to receive input from the networked sensor 1512, for example, by Figure 2 The network sensor 1512 is located outside the glasses 100 and is different from the sensor 106. For example, the network sensor 1512 can be a sensor in an external device (described below) that is paired with the glasses 100. The sensor data processing and analysis module 1504 may include: a visual sensor analysis module for analyzing signals from visual sensors such as cameras (e.g., wide-angle cameras, infrared cameras); an audio sensor analysis module for analyzing signals from audio sensors such as microphones; a position sensor analysis module for analyzing signals from position sensors such as orientation sensors and magnetometers; an environmental sensor analysis module for analyzing signals from environmental sensors such as light sensors, temperature sensors, humidity sensors, and air pressure sensors; other sensor analysis modules for analyzing signals from other types of sensors (e.g., the medical sensors described previously); and a calibration module. The calibration module is used to obtain data from an external network 1514 (e.g., via the transceiver 114, see Figure 2 ) and receiving data from the user portrait database 1516. When generating spatial alignment information for spatially aligning augmented reality content on the three-dimensional display 104 (as previously described in Figure 3 ), a calibration module may be used. Specifically, the calibration module is used to receive user preference / profile data from the user profile database 1516. The calibration module is used to calibrate the sensor analysis module of the sensor data processing and analysis module 1504 based on the data from the external network 1514 and the user preference / profile data from the user profile database 1516.

[0088] The augmented reality rendering module 1506 is used to receive the processed input signal from the sensor data processing and analysis module 1504 and render the augmented reality content. In addition, the augmented reality rendering module 1506 can be used to receive input from the external network 1514 and receive user preference / portrait data from the user portrait database 1516. The augmented reality rendering module 1506 may include a graphics and video rendering module, an audio rendering module, a voice rendering module, or a text rendering module, etc. The augmented reality rendering module 1506 is used to render augmented reality content according to its various inputs. Rendering augmented reality content includes calculating the position of displaying a 3D image on the three-dimensional display 104 so that the 3D image is spatially aligned with the real world environment. For example, spatial alignment information can be used to render augmented reality content at a desired position on the three-dimensional display 104. In some embodiments, the text rendering module is used for speech-to-text processing so that the processed speech can be rendered as text through the three-dimensional display 104. In some embodiments, the speech rendering module is used for text-to-speech processing so that the text recognized by the sensor 106 can be played as speech through the speaker 110.

[0089] The playback module 1508 is used to receive the augmented reality content from the augmented reality rendering module 1506 and output the augmented reality content to the three-dimensional display 104. In addition, the augmented reality rendering module 1506 can be used to receive calibration information (e.g., spatial alignment information) from the calibration module of the sensor data processing and analysis module 1504, and receive user preference / portrait data from the user portrait database 1516. According to the calibration information and the user preference / portrait data, the augmented reality content can be output to the three-dimensional display 104.

[0090] The user behavior and preference analysis module 1510 is used to store user preference / portrait data in a user portrait database 1516. The user portrait database 1516 can be a relational database, a key-value database, a graph database, etc. As described in more detail later, the user preference / portrait data can be used to adapt the operation of the glasses 100 in real time. For example, a user can specify how the glasses 100 should adapt to environmental changes, and those preferences of the user can be stored in the user portrait database 1516. In addition, the user portrait database 1516 can store multiple portraits so that different users can use the same pair of glasses 100. The user preference / portrait data can be generated by receiving preferences from the user (e.g., through a user interface) or by analyzing the user's behavior and automatically generating the user preference / portrait data.

[0091] In some embodiments, the playback module 1508 is also used to transmit notifications from the backend processing module 1502 to the glasses 100. The notifications can be in one or more forms such as text, graphics, audio, tactile, etc. The notifications can be transmitted according to the user's preferences stored in the user portrait database 1516. For example, when the user specifies a preference for audio notifications in certain environments, the audio notifications can be transmitted to the speaker 110.

[0092] The back-end processing module 1502 can be implemented by a front-end processor (e.g., the controller 112) or a back-end processor (e.g., an external device). In this embodiment, the back-end processing module 1502 is implemented by the controller 112 (see Figure 2 ), so that the processing is performed at the glasses 100. Performing all processing by the controller 112 can reduce latency, thereby improving the performance of the glasses 100. In another embodiment (later in Fig.16 ), the back-end processing module 1502 is implemented by an external device. The external device can be a device paired with the glasses 100 to assist the glasses 100 in performing processing. For example, the external device can be a smart phone or a smart watch paired with the glasses 100.

[0093] Fig.16 is a block diagram of a processing system of glasses 100 according to some embodiments. Figure 3 The processing system may be used when the method 300 described in Fig.15 The embodiment of the present invention is similar to that of the embodiment of the present invention, except that the processing system further includes a front-end processing module 1602, and the various aspects of the processing system are divided into a front-end processor and a back-end processor. The back-end processing module 1502 can be implemented by a back-end processor, and the front-end processing module 1602 can be implemented by a front-end processor. The separate processing of the front-end processor and the back-end processor advantageously reduces the processing of the front-end processor (e.g., the controller 112), thereby reducing the power consumption of the glasses 100 and extending the battery life.

[0094] The front-end processing module 1602 is a sensor data preprocessing module, which is used to receive input signals from the sensor 106 and the human-machine interface device 108 and process the input signals. Figure 2 ) forwards the processed input signal to the back-end processing module 1502 (e.g., sensor data processing and analysis module 1504, see Fig.15). Alternatively or additionally, the input signals from some sensors 106 may be directly forwarded to the back-end processing module 1502 without pre-processing by the front-end processing module 1602. Thus, the input signals may be collected and pre-processed by the controller 112 and then forwarded to the external device for rendering and playing the augmented reality content. The output from the back-end processing module 1502 may then be transmitted to the transceiver 114 (see Figure 2 ) is forwarded to the glasses 100 for output to the 3D display 104 and / or the speaker 110. In some embodiments, the output from the back-end processing module 1502 is sent to the front-end processing module 1602 and then forwarded to the 3D display 104 and / or the speaker 110. The output may be additionally processed on the front-end processing module 1602 before being forwarded to the 3D display 104 and / or the speaker 110.

[0095] As previously described, user preference / profile data may be used to adapt the operation of the glasses 100 in real time. More specifically, the operating conditions of the glasses 100 may be determined by sensors 108 or the like. The operation of the glasses 100 may be adapted according to the operating conditions. In addition, the operation of the glasses 100 may be adapted according to the application requirements of the glasses 100. How the operation of the glasses 100 is adapted to the operating conditions and application requirements may be specified by the user preference / profile data.

[0096] Fig.17 is a flow chart of a method according to some embodiments for adapting the operation of the eyewear 100. The method may be implemented using appropriate modules / steps from the previously described modules / steps.

[0097] In step 1702, the operating conditions of the glasses 100 are obtained. The operating conditions may include visual operating conditions (e.g., ambient light intensity), auditory operating conditions (e.g., ambient sound intensity), position operating conditions (e.g., speed), environmental operating conditions (e.g., temperature, humidity, air pressure, etc.), combinations thereof, etc. Acquiring the operating conditions may include: receiving and processing input signals from sensors, such as visual sensors, audio sensors, position sensors, environmental sensors, combinations thereof, etc.

[0098] The operating conditions can be used to determine how the glasses 100 are used or the activities that the user of the glasses 100 is performing. For example, a low speed can indicate that the user of the glasses 100 is walking; a high speed can indicate that the user of the glasses 100 is driving; a low ambient light intensity plus low humidity can indicate that the user of the glasses 100 is indoors; a high ambient light intensity plus high humidity can indicate that the user of the glasses 100 is outdoors; a high ambient sound intensity can indicate that the user of the glasses 100 is in a crowded area; a low ambient sound intensity can indicate that the user of the glasses 100 is in a quiet area; etc. When the value of the operating condition exceeds a predetermined threshold, it can be considered "large"; when its value is below a predetermined threshold, it can be considered "small".

[0099] In step 1704, the operation preference of the glasses 100 is obtained. The operation preference is different from the operation condition. The operation preference is a preference with a condition, wherein the specified condition can be satisfied based on the operation condition (e.g., vision, hearing, location, environment, etc.) of the glasses 100. Specifically, the operation preference indicates how the operation of the glasses 100 should change based on the operation condition, for example, by indicating the user interface that should be used under certain operation conditions. In some embodiments, the operation preference indicates that the first user interface should be used under the first operation condition and the second user interface should be used under the second operation condition. For example, the operation preference can be conditioned on the ambient sound intensity, wherein the operation preference indicates that the first user interface (e.g., touchpad) should be used when the user is in a crowded area, and the second user interface (e.g., voice command) should be used when the user is in a quiet area. Similarly, the operation preference can be conditioned on the speed, wherein the operation preference indicates that the first user interface (e.g., voice command) should be used when the user is driving, and the second user interface (e.g., gesture tracking) should be used when the user is walking. The operation preference includes an indication of a preferred user interface, which corresponds to the operation condition. Additionally, the operational preferences may indicate that when the user is outdoors, a first (eg, high) display brightness and color difference should be used, and when the user is indoors, a second (eg, low) display brightness and color difference should be used.

[0100] Acquiring the operation preference may include: Fig.15 ) retrieves user preference / profile data, wherein the user preference / profile data includes operating preferences. In some embodiments, the operating preferences are retrieved from a memory of the glasses 100. The operating preferences may have been previously received from the user (e.g., via a user interface) and stored in the user profile database 1516. For example, a user of the glasses 100 may use the human interface device 108 to specify operating preferences, for example, by indicating a user interface that should be used under given operating conditions.

[0101] In step 1706, the application requirements of the glasses 100 are obtained. Application requirements are different from operating preferences and operating conditions. Given the current program being executed by the controller 112, the application requirements are used to indicate which parameters of the glasses 100 should be prioritized. For example, the application requirements may include an indication of whether high processing speed or low power consumption should be prioritized for a given application.

[0102] In step 1708, the operation of the glasses 100 is adapted according to the operating preferences, operating conditions, and application requirements of the glasses 100. One or more aspects of the operation of the glasses 100 may be adapted, including device processing (step 1710), display (step 1712), user interface (step 1714), rendering of augmented reality content (step 1716), etc., or a combination thereof.

[0103] In step 1710, the processing system of the glasses 100 may be adapted. Specifically, the back-end processing may be switched between a front-end processor (e.g., the controller 112) and a back-end processor (e.g., an external device), as previously described in Fig.15 and Fig.16 For example, the processing system can switch to a back-end processor (e.g., a cloud computing processor) or a front-end processor based on an operating preference. Similarly, the processing system can switch to a back-end processor (e.g., a cloud computing processor) or a front-end processor based on whether the application requirements require low power consumption or high processing speed, respectively.

[0104] In step 1712, the operation of the three-dimensional display 104 may be adapted. Specifically, the brightness, contrast, color difference, light field generation, etc. of the three-dimensional display 104 may be changed. In some embodiments, the operating preferences indicate that the brightness of the three-dimensional display 104 should be increased when the user is outdoors, and the brightness of the three-dimensional display 104 should be reduced when the user is indoors. In some embodiments, the operating preferences indicate that the brightness of the three-dimensional display 104 should be reduced when the application requirements require low power consumption. In some embodiments, the operating preferences indicate that the contrast of the three-dimensional display 104 should be increased when the user is outdoors, and the contrast of the three-dimensional display 104 should be reduced when the user is indoors. In some embodiments, the operating preferences indicate that the three-dimensional display 104 should use black and white color difference (e.g., for color blind users).

[0105] Reference Figure 3To Figure 6, in some embodiments, the light 404 emitted by the three-dimensional display 104 can be generated according to the operating preferences and operating conditions. Specifically, the intensity of the emitted light 404 can be calculated using visual operating conditions such as ambient light intensity. Similarly, the operating preferences can include an indication of the radius of curvature of the user's eyes, and the exit angle of the light 404 emitted from the microlens 604 can be changed according to the radius of curvature. When outputting augmented reality content to the three-dimensional display 104, the exit angle can be changed by changing the diodes of the illuminated light-emitting diode array 614 so that the light emitted by the diode is refracted by the microlens 604 in the desired direction. Therefore, displaying a 3D image by a three-dimensional display 104 can be used to adapt to users with impaired vision. In this embodiment, the operation of adapting the three-dimensional display 104 includes: calculating the exit angle of the light to be emitted by the three-dimensional display 104 when displaying the 3D image, wherein the exit angle is calculated according to the radius of curvature, and further calculating the intensity of the light, wherein the intensity is calculated according to the ambient light intensity.

[0106] In step 1714, the user interface of the glasses 100 can be adapted. The glasses 100 can be interacted with through more than one type of user interface. The glasses 100 can have a voice-based user interface, a touchpad-based user interface, a tracking-based user interface, a gesture-based user interface, and the like. The voice-based user interface can be interacted with through the user's voice. The touchpad-based user interface can be interacted with through a touchpad (e.g., a human-machine interface device 108). The tracking-based user interface can be interacted with by tracking the position of the user's head, face, and / or eyes. The user can interact with the gesture-based user interface through gestures. In some embodiments, the operating preference indicates the user interface that should be used for the operating condition. For example, the operating preference can indicate that when the user of the glasses 100 is driving, a voice-based user interface should be used, and when the user of the glasses 100 is walking, a touchpad-based user interface should be used. Similarly, the operating preference can indicate that when the user of the glasses 100 is in a crowded area, a touchpad-based user interface should be used, and when the user of the glasses 100 is in a quiet area, a voice-based user interface should be used.

[0107] The user interface can display feedback to the user of the glasses 100 through the three-dimensional display 104. In some embodiments, the operating preferences indicate how the user interface should display feedback. For example, the operating preferences can indicate the font size, color, etc. that should be used to display the user interface feedback through the three-dimensional display 104.

[0108] In step 1716, the rendering of the augmented reality content by the glasses 100 can be adapted. The glasses 100 can display multiple types of augmented reality content. Specifically, the glasses 100 can have selective zooming and object compensation functions, wherein the desired objects in the environment can be enhanced. For example, text recognition can be performed on a sign in the environment, and the augmented reality content can include text from the sign, displayed in a large and / or color-coded font. Similarly, an image of an object in the environment can be taken, and the augmented reality content can include a high-resolution / zoomed copy of the image. Therefore, the desired object in the environment may be more clearly visible to some users (e.g., myopic users). In some embodiments, the operation preference indicates the type of augmented reality content that should be displayed, for example, by indicating whether selective zooming and / or object compensation should be performed, and on which type of object the selective zooming and / or object compensation should be performed. The operation preference can also indicate the font size, color, etc. applied to text recognition.

[0109] Some variations are contemplated. For example, in step 1708, the operation of the glasses 100 may be adapted based on some or all of the operating preferences, operating conditions, and application requirements of the glasses 100. In some embodiments, the user interface of the glasses 100 may be adapted based on operating conditions alone or in combination with operating preferences.

[0110] It should be understood that one or more steps of the embodiment method provided herein can be performed by corresponding units or modules. For example, a signal can be sent by a sending unit or a sending module. A signal can be received by a receiving unit or a receiving module. A signal can be processed by a processing unit or a processing module. Other steps can be performed by an analysis unit / module, a rendering unit / module, an input unit / module, an output unit / module, a display unit / module, a control unit / module, a sensing unit / module, an acquisition unit / module, an adaptation unit / module and / or a networking unit / module. The corresponding unit / module can be hardware, software or a combination thereof. For example, one or more units / modules can be integrated circuits, such as field programmable gate arrays (FPGA) or application-specific integrated circuits (ASIC).

[0111] Although described in detail, it should be understood that various changes, substitutions and modifications may be made without departing from the spirit and scope of the present invention as defined by the appended claims. In addition, the scope of the present invention is not limited to the specific embodiments described herein, as one of ordinary skill in the art will readily appreciate from the present invention, processes, machines, manufactures, compositions of matter, components, methods or steps (currently existing or to be developed later) may perform substantially the same functions or obtain substantially the same results as the corresponding embodiments described herein. Therefore, the appended claims are intended to include such processes, machines, manufactures, compositions of matter, components, methods or steps within their scope.

Claims

1. A method, characterized in that include: Acquiring an operating condition of a wearable device, wherein the operating condition comprises a visual operating condition, an auditory operating condition, a position operating condition or an environmental operating condition, and the wearable device comprises a memory and a three-dimensional display, and the three-dimensional display is optically see-through; acquiring, from the memory of the wearable device, an operation preference of the wearable device, wherein the operation preference comprises a radius of curvature of an eye of a user of the wearable device; An exit angle and an intensity of light to be emitted by the three-dimensional display are calculated, wherein the exit angle is calculated based on the operating preference of the wearable device, and the intensity is calculated based on the operating condition of the wearable device.

2. The method according to claim 1, characterized in that: Calculating the exit angle and the intensity of the light is performed on the wearable device.

3. The method according to claim 1, characterized in that Calculating the exit angle and the intensity of the light is performed on an external device.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: The operational preference is received from the user of the wearable device.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: A three-dimensional image is displayed through the three-dimensional display.

6. The method according to any one of claims 1 to 5, characterized in that The operating preferences also include an indication of a preferred user interface, the preferred user interface corresponding to the operating condition, the method further comprising: Adapting a user interface of the wearable device according to the operating condition and the operating preference of the wearable device.

7. The method according to claim 6, characterized in that The user interface adapted to the wearable device includes: when the operating condition indicates that the user of the wearable device is driving, using a voice-based user interface; When the operating condition indicates that the user of the wearable device is walking, a touchpad-based user interface is used.

8. The method according to claim 6, characterized in that The user interface adapted to the wearable device includes: When the operating condition indicates that the user of the wearable device is in a crowded area, using a touchpad-based user interface; When the operating condition indicates that the user of the wearable device is in a quiet area, a voice-based user interface is used.

9. A method, characterized in that include: Obtain the operating conditions of the wearable device; Acquire the operation preference of the wearable device; Obtaining application requirements of the wearable device; The operation of the wearable device is adapted according to the operating condition, the operating preference and the application requirement of the wearable device.

10. The method according to claim 9, characterized in that Acquiring the operating condition of the wearable device includes: An input signal is received from a visual sensor of the wearable device.

11. The method according to claim 9, characterized in that Acquiring the operating condition of the wearable device includes: An input signal is received from an audio sensor of the wearable device.

12. The method according to claim 9, characterized in that Acquiring the operating condition of the wearable device includes: An input signal is received from a position sensor of the wearable device.

13. The method according to claim 9, characterized in that Acquiring the operating condition of the wearable device includes: An input signal is received from an environmental sensor of the wearable device.

14. The method according to any one of claims 9 to 13, characterized in that The operation of adapting the wearable device includes: A processing system adapted to the wearable device.

15. The method according to any one of claims 9 to 13, characterized in that The operation of adapting the wearable device includes: Adapting the operation of the three-dimensional display of the wearable device.

16. The method according to claim 15, characterized in that The three-dimensional display includes a light emitting unit, and the operation of adapting the three-dimensional display includes: changing an emission angle of light generated by the light emitting unit.

17. The method according to any one of claims 9 to 13, characterized in that The operation of adapting the wearable device includes: Adapting the user interface of the wearable device.

18. The method according to any one of claims 9 to 13, characterized in that The operation of adapting the wearable device includes: Adapting the wearable device to render augmented reality content.

19. A wearable device, characterized in that: include: processor; A non-transitory computer-readable storage medium for storing a program executed by the processor, the program comprising the following instructions: Obtaining an operating condition of the wearable device; Acquire the operation preference of the wearable device; Obtaining application requirements of the wearable device; The operation of the wearable device is adapted according to the operating condition, the operating preference and the application requirement of the wearable device.

20. The wearable device according to claim 19, characterized in that: Also comprising a visual sensor, the instructions for obtaining the operating condition of the wearable device include the following instructions: An input signal is received from the vision sensor.

21. The wearable device according to claim 19, characterized in that: Also including an audio sensor, the instructions for obtaining the operating condition of the wearable device include the following instructions: An input signal is received from the audio sensor.

22. The wearable device according to claim 19, characterized in that: A position sensor is also included, and the instructions for obtaining the operating condition of the wearable device include the following instructions: An input signal is received from the position sensor.

23. The wearable device according to claim 19, characterized in that: Also including an environmental sensor, the instructions for obtaining the operating condition of the wearable device include the following instructions: An input signal is received from the environmental sensor.

24. The wearable device according to any one of claims 19 to 23, characterized in that: Also included is a processing system, wherein the instructions for adapting the operation of the wearable device include the following instructions: Adapting the processing system.

25. The wearable device according to any one of claims 19 to 23, characterized in that: Also including a three-dimensional display, the instructions for adapting the operation of the wearable device include the following instructions: Operation of the three-dimensional display is adapted.

26. The wearable device according to claim 25, characterized in that The three-dimensional display includes a light emitting unit, and the operation of adapting the three-dimensional display includes: changing an emission angle of light generated by the light emitting unit.

27. The wearable device according to any one of claims 19 to 23, characterized in that: The instructions for adapting the operation of the wearable device include the following instructions: Adapting the user interface of the wearable device.

28. The wearable device according to any one of claims 19 to 23, characterized in that: The instructions for adapting the operation of the wearable device include the following instructions: Adapting the wearable device to render augmented reality content.

Images