Display of three-dimensional images

Abstract

The invention relates to display of three-dimensional images. Apparatus and methods for displaying a 3D representation of an object are described. An apparatus may include a rotatable structure, a motor, and a plurality of light field sub-displays disposed on the rotatable structure. A device may store light field images to be displayed that provide a plurality of different views of an object in different viewing directions. The processor may drive the motor to rotate the rotatable structure and map the light field image to each of the light field sub-displays based at least in part on the rotation angle, and illuminate the light field sub-displays based in part on the mapped light field image. In one embodiment, an apparatus may include a display panel configured to be viewed from a reference viewing direction, where the display panel is curved outwardly from a plane perpendicular to the reference viewing direction, and a plurality of light field sub-displays disposed on the display panel.

Description

[0001] This application is a divisional application of an application with a filing date of January 19, 2017, an application number of 202111360771.8, and an invention name of “Display of Three-Dimensional Images”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 62 / 288,680, entitled “HOLOGRAPHICPROPELLER,” filed on January 29, 2016, U.S. Provisional Patent Application No. 62 / 343,722, entitled “DISPLAY FOR THREE-DIMENSIONAL IMAGE,” filed on May 31, 2016, and U.S. Provisional Patent Application No. 62 / 343,767, entitled “CURVEDDISPLAY FOR THREE-DIMENSIONAL IMAGE,” filed on May 31, 2016. The disclosures of all of these prior applications are hereby incorporated by reference herein in part or in their entirety. Technical Field

[0004] The present disclosure relates to apparatus and methods for displaying a three-dimensional representation of an object, and more particularly to displaying a light field of an object to depict the three-dimensional representation of the object. Background Art

[0005] Light from natural objects has a specific content in terms of rays of light having magnitude and direction at each point in space when it encounters the human eye. This structure is called the light field. Conventional two-dimensional (2D) displays (paintings, photographs, computer monitors, televisions, etc.) emit light isotropically (e.g., light is emitted uniformly from the display). Therefore, these 2D displays can only approximate the light field of the objects they represent. Summary of the invention

[0006] Therefore, there is a need to build displays that reproduce or attempt to reproduce the exact or approximate light fields produced by natural objects. Such displays produce more compelling images that appear three-dimensional (3D) and can be mistaken for natural objects. Traditional 2D displays cannot achieve these functions.

[0007] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include a rotatable structure; a motor configured to rotate the rotatable structure; a plurality of light field sub-displays disposed on the rotatable structure; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory, the motor, and the light field sub-display. The processor is programmed with executable instructions to: drive the motor to rotate the rotatable structure about a rotation axis, the rotatable structure being positioned at a rotation angle that varies with time; access the light field image; map the light field image to each of the plurality of light field sub-displays based at least in part on the rotation angle; and illuminate the plurality of light field sub-displays based at least in part on the mapped light field image.

[0008] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. The method may include driving a motor to rotate a rotatable structure including a plurality of light field sub-displays about a rotation axis, the rotatable structure being positioned at a rotation angle that varies over time. The method may also include selecting a light field image to be displayed by the display device, the light field image providing different views of the object in different viewing directions; mapping the light field image to each of the plurality of light field sub-displays based at least in part on the rotation angle; and illuminating the light field sub-display based at least in part on the mapped light field image.

[0009] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include a light field sub-display configured to rotate, the light field sub-display having multiple display positions; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory and the light field sub-display. The processor may be programmed with executable instructions to rotate the light field sub-display about a rotation axis, wherein the display position is based on a rotation angle that varies over time; select the light field image; map the light field image to the display position based at least in part on the rotation angle; and illuminate the light field sub-display based at least in part on the mapped light field image.

[0010] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include one or more light field sub-displays, each of the one or more light field sub-displays having multiple display positions, wherein the one or more light field sub-displays are configured to rotate about one or more rotation axes; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory and the one or more light field sub-displays. The processor may be programmed with executable instructions to drive the one or more light field sub-displays to rotate about at least one of the rotation axes, wherein the multiple display positions are based on a rotation angle that varies over time; and illuminate the one or more light field sub-displays based at least in part on the light field image and the multiple display positions.

[0011] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include a display panel viewed from a reference viewing direction, wherein the display panel is curved outward from a plane perpendicular to the reference viewing direction, and a plurality of light field sub-displays are disposed on the display panel. The display device may also include a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory and the light field sub-display. The processor may be programmed with executable instructions to select the light field image; map the light field image to each of the light field sub-displays based at least in part on the position of the light field sub-display on the display panel; and illuminate the light field sub-display based at least in part on the mapped light field image.

[0012] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. The method may include selecting a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions. The method may also include mapping the light field image to each of the plurality of light field sub-displays based at least in part on the positions of the plurality of light field sub-displays on a display panel. The method may also include illuminating the plurality of light field sub-displays based at least in part on the mapped light field image.

[0013] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include a display panel configured to be viewed from a reference viewing direction, wherein the display panel is bent outward from a plane perpendicular to the reference viewing direction. The display device may also include one or more light field sub-displays, each of the one or more light field sub-displays having a position on the display panel. The display device may also include a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing multiple different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory and the light field sub-display. The processor may be programmed with executable instructions to select the light field image; and illuminate the one or more light field sub-displays based at least in part on the light field image and the position of the one or more light field sub-displays on the display panel.

[0014] In some embodiments, a display device and method for displaying a 3D representation of an object are disclosed. In one embodiment, the display device may include a curved panel including a plurality of light field sub-displays.

[0015] Details of one or more embodiments of the subject matter described in this specification are given in the following drawings and description. Other features, aspects and advantages will become apparent from the description, drawings and claims. Neither the present disclosure nor the following detailed description is intended to define or limit the scope of the subject matter of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An exemplary display device is schematically shown.

[0017] Figure 2A and 2B is a perspective view schematically showing an example of a light field sub-display for outputting light field image information ( Figure 2A ) and top view ( Figure 2B ).

[0018] Figures 3A to 3C It is schematically shown Figure 2A and 2B A cross-sectional side view of a portion of an embodiment of a lightfield sub-display.

[0019] Figure 4A and 4B An example of a waveguide stack for outputting light field image information to a user is schematically shown.

[0020] Figures 5A to 5G Various examples of display devices are schematically shown.

[0021] Fig. 6A and 6B is a perspective view schematically illustrating an exemplary display device displaying a 3D representation of an image (a dog in this example) being viewed by multiple viewers.

[0022] Figure 7 is a perspective view schematically illustrating another exemplary display device that is displaying a 3D representation of an image viewed by multiple viewers.

[0023] Figure 8 is a process flow diagram of an example of a method of displaying a 3D representation of an object using a display device.

[0024] Fig. 9 is a process flow diagram of an example of a method of mapping light field image information to a light field sub-display of a display device.

[0025] Fig.10 is a process flow diagram of an example of a method of illuminating a light field sub-display of a display device.

[0026] Fig.11 is a perspective view schematically illustrating an exemplary display device.

[0027] Fig. 12A and 12B is a diagram schematically showing a 3D representation of an image (a dog in this example) being viewed by multiple viewers. Fig.11 A top view of an exemplary display device.

[0028] Fig.13A is a perspective view schematically illustrating another exemplary display device.

[0029] Fig. 13B is a diagram schematically showing a 3D representation of an image being viewed by a plurality of viewers. Fig.13A A top view of a display device.

[0030] Figures 14A to 14E are perspective views schematically showing various examples of display devices.

[0031] Fig.15 is a process flow diagram of an example of a method of displaying a 3D representation of an object using a display device.

[0032] Fig.16 is a process flow diagram of an example of a method of mapping light field image information to a light field sub-display of a display device.

[0033] Fig.17 is a process flow diagram of an example of a method of illuminating a light field sub-display of a display device.

[0034] Reference numerals may be repeated throughout the drawings to indicate corresponding relationships between referenced elements. The drawings are provided to illustrate exemplary embodiments described herein and are not intended to limit the scope of the present disclosure. DETAILED DESCRIPTION

[0035] Overview

[0036] Currently, many types of light field displays are expensive and therefore not suitable for many applications (e.g., commercials, viewing at home, etc.). Current implementations of light field displays (e.g., flat panel displays) use a large number of pixels and waveguides to simulate a 3D representation of an object. At any single point in time, this representation requires displaying several images, each of which presents a different direction of viewing the object and a varying depth of focus, so that the object appears to be three-dimensional. In some embodiments, the larger the angle at which the viewer is located from the normal to the flat display panel, the more limited the 3D representation field of view can be provided to the viewer using the flat display panel. The present disclosure describes examples of light field displays that are not overly expensive because the implemented light field sub-display technology can display multiple viewing angles or depths of focus at any single instance, and can be controlled to switch between multiple different views of an object being displayed in a three-dimensional representation. The present disclosure describes some examples that can be configured to provide a larger field of view of an object being displayed in a 3D representation. Such displays can be used for indoor or outdoor display applications, such as advertising, home viewing, indoor or outdoor decoration, art, etc. For example, a store front or other commercial establishment may wish to attract customers by displaying objects in three dimensions as opposed to traditional two-dimensional displays. A three-dimensional representation may be more eye-catching, or more likely to be noticed, by passers-by as opposed to a flat two-dimensional representation.

[0037] The present disclosure describes an example of a display device including a rotatable structure (e.g., a propeller) that combines multiple light field sub-displays, where each light field sub-display is strobed with a different image depending on the current rotational state of the rotatable structure and the overall image projected by the display. The strobing (e.g., switching displayed content) rate can be a frequency that is imperceptible to the eyes of a person viewing the object. The rotational motion of the rotatable structure causes the light field sub-displays to sweep across a particular area, so a lower cost implementation of a display that provides a 3D image to a viewer is possible.

[0038] The present disclosure also describes an example of a display device including a curved display panel that combines multiple lightfield sub-displays, where each lightfield sub-display is illuminated with a different image representing a different viewing direction, depending on the position of the lightfield sub-display on the display panel and the overall image projected by the display device. The curvature of the display panel can enable the lightfield sub-displays to display 3D representations of objects that are easier for a viewer to perceive at larger angles from the normal to the display device.

[0039] Display device examples

[0040] Figure 1 An example of a display device 100 is shown, which is configured to display an image viewable as a 3D representation of an object. The display device 100 includes a rotatable structure 105, a motor 104, and a control system 110. The rotatable structure 105 can be coupled to the motor 104, which is configured to drive the rotatable structure 105 along a path 103 around a rotation axis 120 based on input from a local data processing module of the control system 110. The control system 110 can be operably coupled to the display device 100, which can be mounted in various configurations, such as fixedly attached to the display device 100 or located at other locations related to the display device 100 (e.g., located in a separate part of a room or in a central control room). The rotatable structure 105 may include an array of light field sub-displays 101 arranged along one or more elongated elements 102. The light field sub-displays 101 can be controlled by the control system 110 to generate and display a 3D representation of an object.

[0041] In some embodiments, the movement of the rotatable structure 105 causes the light field sub-display 101 to move around the path 103, and when driven by the control system 110 to illuminate the light field sub-display 101, the rotatable structure displays an image that can be viewed by a bystander as a 3D representation of the object to be displayed. For example, the display device 100 can be placed in front of a store or in a viewing area, where a person located at a viewing distance from the display device 100 can view the image displayed by the display device 100 by looking at the rotatable structure 105. In some embodiments, when the light field sub-display 101 rotates around the path 103 due to the rotational movement imparted to the rotatable structure 105 by the motor 104, an extended 3D representation of the object is created. In some embodiments, the plurality of light field sub-displays 101 can each include one or more pixels, as described below, which can be illuminated according to the light field image data stored in the digital memory 112 (e.g., a non-transitory data memory) to display the 3D representation of the object. In some embodiments, the speaker 118 can be coupled to the display device 100 to provide audio output.

[0042] Reference again Figure 1, the rotatable structure 105 can be arranged in a manner similar to a propeller rotating about an axis 120. Figure 1 As shown, a rotatable structure 105 having a propeller arrangement may include a plurality of elongated elements 102. The elongated elements 102 may also be configured as a plurality of arms or blades of a propeller. Figure 1 The display device 100 is shown as having four elongated elements 102, but the number, arrangement, length, width or shape of the elongated elements 102 may be different (see, for example Figures 5A to 5G For example, the number of elongated elements 102 may be 1, 2, 3, 4, 5, 6 or more (e.g., Figure 5A and 5B The elongated element 102 may be straight (e.g., Figure 1 , 5A and 5B), such as Figure 5C The propeller is shown as being curved, or curved inwardly or outwardly from a plane perpendicular to the propeller's axis of rotation 120 (e.g. Figure 7 ).

[0043] Continue to refer Figure 1 , each elongated element 102 comprises an array of light field sub-displays 101 arranged along the length of the elongated element 102 . Figure 1 While five light field sub-displays 101 are shown disposed on each elongated element 102 (as well as an additional optional sub-display at the center of the display where the elongated elements intersect), other embodiments are possible. For example, the number of light field sub-displays 101 on each elongated element 102 may be 1, 2, 3, 4, 5, 6, or more. In another embodiment, the rotatable structure may include a single light field sub-display disposed thereon. The light field sub-display 101 may be any display configured to generate a light field. In some embodiments, the light field sub-display 101 may include one or more pixels configured to emit anisotropic light (e.g., directional emission). For example, as will be combined with Figures 2A to 3C Described in more detail, the light field sub-display 101 may include a microlens array disposed adjacent to a pixel array that emits light isotropically toward the microlens array. The microlens array redirects light from the pixel array into an array of light beams propagating at different exit angles to produce a light field image. In some embodiments, each microlens in the microlens array may be configured as a pixel of the light field sub-display 101. In another embodiment, the light field sub-display 101 may include a waveguide stack assembly that generates light, such as in combination with Figure 4A and 4B described.

[0044] The display device also includes a motor 104 configured to drive a rotatable structure 105. For example, the motor 104 can rotate the rotatable structure 105 in a circular motion around the rotation axis 120, as shown in the rotation path 103. When the rotatable structure 105 is driven by the motor 104, the light field sub-display 101 also rotates around the rotation path 103. The control system 110 can be configured to control the rotation speed applied to the rotatable structure 105 by the motor 104 at a desired frequency. The rotation frequency can be selected so that the rotatable structure 102 is not perceptible to the viewer, and the viewer mainly perceives the 3D image due to the visual persistence of the human visual system. Such a display is sometimes commonly referred to as a visual persistence (POV) display. Other rotation frequencies are also possible. The combination of rotating the light field sub-display 101 and the illumination of each light field sub-display 101 projects a representation of an image that can be viewed by a viewer. The image can include an object, a graphic, text, etc. The image can be part of a series of image frames that project an object or object that appears to be moving or changing as in a video. The representation may appear to be 3D and may be mistaken by the viewer as a natural object rather than a projection. The motor 104 and the control system 110 may be arranged to be inconspicuous to the viewer (e.g., below the propeller and connected to the propeller via a suitable transmission). Since the propeller arm is not visible (when the propeller rotates fast enough), the image may appear to be suspended in mid-air, thereby attracting the attention of passers-by. Therefore, the display device 100 can be advantageously used for advertising, marketing or sales, for demonstrations, or to otherwise generate interest or convey information to the viewer.

[0045] The local data processing module of the computer control system 110 may include a hardware processor 112 and a digital memory 114. In some embodiments, the digital memory 114 may be a non-volatile memory (e.g., flash memory) or any non-transitory computer-readable medium. The digital memory 114 may be configured to store data defining instructions for the hardware processor 112. These instructions configure the hardware processor 112 to perform the functions of the display device 100. For example, both the hardware processor 112 and the digital memory 114 may be used to assist in processing, caching, and storing light field data. The data may include data related to: a) a light field image of an object to be displayed, b) a light field sub-display position that changes over time, or c) a mapping of a light field image to a light field sub-display position. In some embodiments, the light field image includes a plurality of rendered frames of an object, wherein each rendered frame is a 2D representation of the object in a viewing direction (e.g., a direction in which a viewer may be relative to the display device 100). Each rendered frame may include a plurality of pixels, hereinafter referred to as rendered pixels, which may be combined to represent an image of an object to be displayed. Each rendered pixel may be associated with a position on the rendered frame (e.g., a rendered pixel position). The plurality of rendered frames and rendered pixel positions may be stored in digital memory 114 for selection and use by control system 110. The light field image may include imaging parameters (e.g., color and intensity of light used to display the rendered frame), where the imaging parameters are associated with a viewing direction of the rendered frame. In some embodiments, the light field sub-display position is defined by the position of light field sub-display 101 along elongated element 102, which varies with time and with an angle of rotation based on the rotation speed of rotatable structure 105. The light field sub-display position may also include the time-varying position of components of each light field sub-display (e.g., microlenses described below).

[0046] In some embodiments, the hardware processor 112 may be operably coupled to the digital memory 114 and configured to analyze and process data in the digital memory 114. The hardware processor 112 may also be operably coupled to the motor 104 and configured to drive the motor 104 at a rotational speed. In some embodiments, the rotational speed may be preselected based on the light field image, the number of light field sub-displays 101, or the number of elongated elements 102. The hardware processor 112 may also be operably coupled to each light field sub-display 101 and configured to drive each light field sub-display 101 (e.g., pixels of each light field sub-display 101 as described below) based on the light field image stored in the digital memory 114. For example, when the rotatable structure 105 is rotated based on instructions executed by the hardware processor 112, rotation is imparted to the light field sub-displays 101, causing them to sweep out a series of concentric arcs about the rotation axis 120 along the rotation path 103. The hardware processor 112 may also drive each lightfield sub-display 101 (e.g., a pixel described below) to emit light when the lightfield sub-display 101 (or a pixel therein) reaches a position associated with a rendered pixel position and image parameters stored in the digital memory 112. The rotation speed of the rotatable structure 105 may be high enough so that a viewer does not perceive the elongated elements 102 of the rotatable structure 105 as the rotatable structure 105 rotates (e.g., the rotatable structure 105 actually appears transparent), but instead sees illumination from the lightfield sub-display 101, thereby displaying a 3D representation of an object.

[0047] One possible way to enable a 3D representation of a display object is to pre-render multiple viewpoints by the control system 110 or another rendering engine. For any given orientation (e.g., rotation angle) of the rotatable structure 105, a mapping can be generated or retrieved that maps the position (z) of a pixel of the light field sub-display 101 (e.g., based on the rotation of the rotatable structure 105) at time (t) to a rendered pixel (u) of a rendered frame (k). The mapping can be accomplished by a processor 112, which can include a microprocessor or microcontroller, a graphics processing unit (GPU), or dedicated hardware (e.g., a floating point gate array (FPGA) or an application specific integrated circuit (ASIC)).

[0048] In one embodiment, the control system 110 may be configured to map the rendered pixels of the rendered frames. For example, the rendered frame k may be associated with the viewing direction of the object to be displayed, and the rendered pixel (u) may have a position (e.g., represented by coordinates, such as X and Y coordinates or position coordinates) within the rendered frame (k). The mapping may be constant and independent of the object to be displayed, and thus may be pre-calculated and stored (e.g., in the digital memory 114) in a data structure (e.g., in a lookup table (LUT)).

[0049] In one embodiment, the control system 110 may also be configured to map the rendered pixel positions to the positions of the light field sub-display 101. For example, each pixel of the light field sub-display 101 may be at different positions at different times based on the rotation speed of the rotatable structure 105. The rotation speed may, but need not be, constant in time. Additionally, because the light field sub-display 101 rotates over time, the rendered pixel positions of the light emitted by the pixels of the light field sub-display 101 may be translated for the overall rotation. Thus, as the pixels sweep out along the path 103, each rendered pixel position (u) of the rendered frame (k) may be associated with a given position of the pixel of the light field sub-display 101 based on the position (z) of the pixel along the elongated element 102 as a function of time (t). Thus, the corresponding rendered pixels of each rendered frame may be grouped together and mapped to the pixels of the light field sub-display 101. The mapping is configured such that the rendered pixel positions are translated to the pixels of the light field sub-display 101 so that the light emitted from the light field sub-display 101 is anisotropically directed based on the viewing direction of the rendered frame. This may also be pre-computed and stored (e.g., in digital memory 114) in a data structure (e.g., in a lookup table (LUT)), which may be the same as or different from the data structure described above. In some embodiments, as rotatable structure 105 rotates, pixels of light field sub-display 101 may be strobed (e.g., alternating or switching between different rendered frames of the light field image) based on the mapped transformed image parameters of the rendered frames.

[0050] In some embodiments, because some light field sub-displays 101 are farther from the rotation axis 120, these light field sub-displays 101 sweep out a larger circular area than light field sub-displays 101 that are closer to or located on the rotation axis 120. In some cases, the apparent intensity of light from light field sub-displays 101 that are farther from the rotation axis 120 as viewed by a viewer of the display object may tend to be lower than the intensity of light emitted from light field sub-displays 101 that are closer to the rotation axis 120 because the amount of illumination per unit area is reduced for light field sub-displays 101 that are farther from the rotation axis 120. Therefore, in some embodiments, in order to keep the apparent intensity of the image across the rotatable structure 105 relatively constant, the illumination brightness and / or strobe duration can be linearly scaled with the radius of a particular light field sub-display 101 based on the distance to the rotation axis 120. In other embodiments, light field sub-displays 101 at larger radii have an increased size and / or an increased number of pixels (compared to light field sub-displays 101 that are closer to the rotation axis). In other embodiments, more lightfield sub-displays 101 may be used at larger radii, for example, by reducing the spacing between adjacent lightfield sub-displays 101 or by branching the elongated element 102 into sub-elements as the distance from the axis of rotation increases.

[0051] The control system 110 may include a connection to a network, for example, to receive an image or image display instruction to be displayed by the display device 100. The display device 100 may include an audio function. For example, the display device 100 may include or be connected to a speaker system 118 to project audio in combination with the projected image. In some embodiments, the display device 100 may include a microphone 119 and voice recognition technology to enable the display device 100 to receive and process audio commands or comments from viewers. For example, the display device 100 may be configured to recognize comments from interested viewers and take actions to modify the display device 100 in response to the comments (e.g., by changing the color of the projected image, changing the projected image, outputting an audio response to the comment, etc.). As an example, in a retail store environment, a display may show an image of a product for sale, and in response to a question about the price of the product, the display may output the price in an audible manner (e.g., "This product is priced at two dollars today") or by a change in the displayed image (e.g., text or graphics showing the price).

[0052] The display device 100 may include a proximity sensor 116 to detect if an object is nearby, and the control system 110 may take appropriate action, such as displaying an audible or visual warning or shutting off or slowing the rotation of the propellers. These embodiments may provide a safety advantage if a viewer attempts to touch a 3D visible object without being aware of the rapidly rotating propeller arms.

[0053] Although examples of devices for generating light fields are described herein, it should be understood that it is not necessary to use a single light field sub-display type to display a 3D representation of an object in a display device. Other light field displays can be envisioned so that multiple light field sub-displays are arranged on a rotatable structure to generate a 3D representation of an object. For example, any of the light field sub-displays, assemblies, or arrangements described in U.S. Patent Application No. 62 / 288,680, entitled "Holographic Propeller," filed on January 29, 2016, can be implemented for displaying a 3D representation of an object, wherein the entire contents of the patent application are incorporated herein by reference. One non-limiting advantage of some embodiments disclosed herein is that by attaching an array of light field sub-displays along a rotating elongated element, a display device can display a 3D representation using a reduced number of light field sub-displays compared to a single non-rotating display covered by pixels. Another non-limiting advantage of embodiments of the present invention is that fewer display elements or light field sub-displays need to be illuminated at any one time compared to a single display that illuminates the entire display to generate an image. In some embodiments, control system 110 may be configured to control actuation of each lightfield sub-display 101 (eg, the timing, intensity, and color of illumination of each lightfield sub-display) based on a desired image to be projected by display device 100 .

[0054] Example of a light field sub-display including a microlens array assembly

[0055] Figures 2A to 2B It shows that you can follow Figure 1 An example of a light field sub-display 101 configured with a rotatable structure 105. In some embodiments, the light field sub-display 1010 may be configured as follows: Fig.13A Or on the display panel 1305 shown in 14A to 14E. Figure 2A is an exploded perspective view of a portion of a lightfield sub-display 101 having a microlens array 210 spaced apart from a pixel array 220 including a plurality of pixels 205. The microlens array 210 includes a plurality of microlenses 215. Figure 2B yes Figure 2AA top view of a portion of light field display 101 is shown. Pixels 205 of pixel array 220 may be liquid crystals (LC), light emitting diodes (LEDs), organic LEDs (OLEDs), or any other type of pixel structure configured to emit light to render an image. Typically, pixels 205 of pixel array 220 emit light substantially isotropically in a direction at least above pixel array 220 and toward microlens array 210. Figures 2A to 2B And other figures shown herein may not be drawn to scale, but are for illustration purposes only. In addition, these figures schematically show a portion of the light field sub-display 101 , which may include more than these four micro-lenses 215 and more than 100 pixels 205 .

[0056] Figure 2A and 2B The light field sub-display 101 is shown to include a microlens array 220 having a plurality of microlenses 215 . Figure 2A and 2B The illustrated microlens array 210 includes a 2×2 array of microlenses 215. Each microlens 215 is associated with a subset of pixels 205 of the pixel array 220. For example, microlens 215a is used to redirect light from a subset 225 of pixels 205 of the pixel array 220 disposed below microlens 215a to various angular directions. Figures 3A to 3C The redirection of light by the microlens 215a is described.

[0057] use Figures 2A to 2B The resolution of the display device 100 of the light field sub-display 101 may depend on, for example, the number of microlenses 215 included in the microlens array 210 and the number of pixels in the subset 225 associated with each microlens. In some embodiments, each microlens 215 may be configured as a pixel of the light field sub-display 101. For example, Figure 2A The pixel array 220 shown includes an array of 10×10 pixels (shown in dashed lines). Each microlens 215 can be associated with a subset 225 of pixels 205, such as Figure 2A and 2B As shown, microlenses 215a are associated with a 5×5 subset 225 (shown in solid lines) of pixels 205. Microlens array 210 and pixel array 220 are intended to be exemplary, and in other embodiments, the arrangement, number, shape, etc. of microlenses and pixels may differ from that shown. For example, pixel array 220 may include 100×100 pixels covered by microlens array 210 such that each microlens 215 covers a 10×10 array of pixels on pixel array 220.

[0058] exist Figures 2A to 2BIn the example shown, the cross-sectional shape of the microlenses 215 is illustrated as circular, but they can be rectangular or any other shape. In some embodiments, the shape or spacing of the individual microlenses 215 can vary across the microlens array 210. In addition, although Figure 2A and 2B A 2×2 microlens array is shown arranged on top of a 10×10 pixel array, but it should be understood that this is for illustrative purposes and any other number or size n×m (n, m=1, 2, 3, 4, 5, 10, 20, 30, 64, 100, 512, 768, 1024, 1280, 1920, 3840 or any other integer) may be used for the microlens array 210 or the pixel array 220.

[0059] One non-limiting advantage of utilizing a microlens array 210 is that each microlens array 210 of a single light field sub-display 101 can be configured as a light field display capable of providing a light field to a viewer of the display device. A light field display is capable of controlling the direction of the emitted light as well as the color and intensity. In contrast, conventional displays emit light isotropically in all directions. For example, a microlens 215a can be associated with a subset 225 of pixels 205. The subset 225 of pixels 205 can emit isotropic light, but when the light passes through the microlens 215a, the light is directed to the viewer to imitate or simulate light originating from a spatial point at the focal plane on which the viewer is focusing.

[0060] Figures 3A to 3C is a partial side view of lightfield sub-display 101 including illustrative representations of ray traces for multiple arrangements of pixel array 220 and microlens array 210 . Figure 3A A partial cross-sectional side view of the lightfield sub-display 101 is shown including light rays emitted from a subset 225 of pixels 205 of the pixel array 220. The pixels 205 of the pixel array 220 are located at a distance a from the microlens array 210. In some embodiments, the hardware processor is configured to drive each pixel 205 of the pixel array 220 to emit light based on image data stored in the digital memory 114. The light emitted from each of the individual pixels 205 interacts with the microlens array 210 such that the spatial extent of light emitted from the subset 225 of pixels 205 beneath the associated microlens 215a produces an array of light beams 305a propagating at different exit angles. Figure 3AIn the illustrated embodiment, the distance a between the microlens array 210 and each pixel 205 is approximately equal to the focal length (f) of the microlens 215 in the microlens array 210. When the distance a is equal to the focal length (f), the light emitted from each pixel 205 of the pixel array 220 interacts with the microlens array 210, so that the spatial range of the light emitted from the subset 225 of the pixels 205 produces a substantially collimated light beam array 305a at different exit angles. The different line types of light (e.g., solid lines, dotted lines, etc.) do not refer to the color or intensity of the light, but are merely illustrative to depict the geometric shape of the light emitted by different pixels.

[0061] In some embodiments, the number of pixels in the subset 225 of pixels 205 disposed under each microlens in the microlens array 210 may be selected based on the number of light beams 305a designed to be emitted from each microlens in the microlens array 210. For example, an n×m subset 225 of pixels 205 under a microlens 215a may produce an array of n×m light beams perceptible to a viewer, thereby representing n×m different viewing directions of an object represented by the display device 100. In various embodiments, n and m (which may be different from each other and from each subset 225 of pixels 205) may be integers, such as 1, 2, 3, 4, 5, 10, 16, 32, 64, 100, 256, or more. For example, a 5×5 subset 225 of pixels 205 may be disposed under a microlens 215a. Figure 2A The microlenses 215a can emit light in 25 different directions. Each direction can be associated with a viewing direction of an image to be displayed by the display device 100.

[0062] exist Figure 3AIn the illustrated embodiment, each pixel 205 is located at a focal length (f) of the microlens array 210 such that light emitted from each pixel 205 will be fully or partially collimated and redirected to an exit angle by the microlens 215, such that a subset 225 of pixels 205 below the microlens 215 effectively produces a plurality of light beams 305a, each corresponding to a particular angle of the overall light field produced by the display. In some embodiments, if relatively few pixels are located in the subset 225 of pixels 205, gaps 310a may exist between the collimated light beams 305a. The gaps 310a may be perceived by a viewer viewing the image at an angle associated with the gaps 310a, and may distract from the appearance of the image if the angular extent of the gaps 310a is too great. The gaps 310a may be observed as a falloff in the intensity of the light 305a directed to the viewer at that angle. If the angular extent of the gaps 310a is too great, the viewer may perceive the brightness of the displayed image as a modulation when the viewer moves her head or eyes or slightly changes her position relative to the display, which may be distracting. In one embodiment, the gap 310a may be reduced by increasing the number of pixels in the subset 225 of pixels 205 so that the angular extent of the gap 310a is sufficiently small. Ray tracing software may be used to model the distribution of light from the light field sub-display 101 and determine the number, spacing, spatial distribution, etc. of pixels and microlenses based on factors such as the typical distance at which a viewer views the display, the amount of modulation that is acceptable, and the like.

[0063] In another embodiment, as an alternative to or in combination with the embodiments described herein, pixels in subset 225 of pixels 205 may be placed at a distance a from microlens array 210 that is slightly greater than or slightly less than focal plane 230 of microlens 215 in the microlens array (see, e.g., Figure 3B and 3C ). This may result in a certain degree of divergence of the individual light beams, so that there are fewer, reduced, or no gaps in the light field at the far field of the light field sub-field display 101. For example, Figure 3B A situation is shown where the distance a is less than the focal length f, whereby the light beam 305b diverges outwardly, thereby reducing the angular extent of the gap 310b. Figure 3C A situation is shown where the distance a is greater than the focal length f, so that the beams may diverge towards the central beam, which may result in a larger gap 310c in some embodiments.

[0064] Light field sub-display including waveguide stack assembly

[0065] although Figures 2A to 3CAn example light field sub-display 101 including a microlens array 210 for use in the display device 100 is shown, but this is for illustration only and not limitation. It should be understood that the various advantages of the embodiments disclosed herein may be achieved by any variation and type of display capable of generating a light field used as one or more light field sub-displays 101. For example, any of the light field displays, stacked waveguide assemblies, or other optical emitters described in U.S. Patent Application No. 14 / 555,585, entitled “Virtual and Augmented Reality Systems and Methods,” filed on November 27, 2014, and published as U.S. Patent Application Publication No. 2015 / 0205126, may be implemented as Figure 1 One or more of the light field sub-displays 101 of the display 100 may be implemented as described below. Fig.11 One or more of the light field sub-displays 1101 of the display 1100, wherein the entire contents of the above-mentioned U.S. patent application are incorporated herein by reference. In addition, the stacked waveguide assembly can be used as a Figure 2A and 2B The light field sub-display can be replaced by or combined with the microlens array.

[0066] Figure 4A and 4B One such embodiment of a stacked waveguide assembly 178 that can be implemented as a light field sub-display 101 is shown. For example, Figure 4A and 4B Aspects of a method of simulating a three-dimensional image using multiple depth planes are shown. Figure 4A and 4B The optics shown correspond to a stacked waveguide assembly of transmissive beamsplitter substrates, each substrate being configured to project light at a different focal plane.

[0067] refer to Figure 4A, objects at different distances from an eye 404 (which may be a single eye or binocular) may be accommodated by the eye 404 so that the objects are in focus. Thus, it can be said that a particular accommodation state is associated with a particular depth plane having an associated focal length, such that when the eye is in an accommodation state for a particular depth plane, objects or portions of objects in that depth plane are in focus. In some embodiments, a three-dimensional image may be simulated by providing a different presentation of an image (e.g., a different rendering frame) for each eye 404, and also by providing a different presentation of the image corresponding to each depth plane or different viewing angles. Without being limited by theory, it is believed that the human eye can generally interpret a limited number of depth planes to provide depth perception. Therefore, by providing a different presentation of an image corresponding to each of these limited number of depth planes to the eye, a highly credible simulation of perceived depth may be achieved.

[0068] Figure 4A An example of a stacked waveguide assembly 178 for outputting image information to a user is shown. A stacked waveguide assembly or waveguide stack 178 may be used to provide a three-dimensional perception to the eye / brain using multiple waveguides 182, 184, 186, 188, 190. In some embodiments, the waveguide assembly 178 may correspond to Figure 1 The light field sub-display 101 is provided.

[0069] Continue to refer Figure 4A , the stacked waveguide assembly 178 may also include a plurality of features 198, 196, 194, 192 located between the waveguides. In some embodiments, the features 198, 196, 194, 192 may be lenses. The waveguides 182, 184, 186, 188, 190 or a plurality of lenses 198, 196, 194, 192 may be configured to send image information to the eye at various levels of wavefront curvature or light divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. Image injection devices 410, 420, 430, 440, 450 may be used to inject rendered frame image information (as described above) into the waveguides 182, 184, 186, 188, 190, and each of the waveguides 182, 184, 186, 188, 190 may be configured to distribute incident light across each corresponding waveguide for output toward the eye 404. In some embodiments, a single beam (e.g., a collimated beam) can be injected into each waveguide to output an entire field of cloned collimated beams directed toward the eye 404 at a specific angle (and amount of divergence) corresponding to the depth plane of the rendered frame and associated with the specific waveguide.

[0070] The waveguides 182, 184, 186, 188, 190 can be configured to propagate light within each respective waveguide by total internal reflection (TIR). The waveguides 182, 184, 186, 188, 190 can each be planar or have another shape (e.g., curved) with top and bottom major surfaces and edges extending between these top and bottom major surfaces. In the illustrated configuration, the waveguides 182, 184, 186, 188, 190 can each include light extraction optical elements 282, 284, 286, 288, 290 that are configured to extract light from the waveguides by redirecting light propagating within each respective waveguide out of the waveguides, thereby outputting image information to the eye 404. The light propagating in the waveguides impinges on the location of the light redirecting element, and the extracted light beam is output by the waveguide. The light extraction optical elements 282, 284, 286, 288, 290 can be, for example, reflective or diffractive optical features. Although shown as disposed at the bottom major surface of the waveguide 182, 184, 186, 188, 190 for ease of description and clarity of depiction, in some embodiments, the light extraction optical elements 282, 284, 286, 288, 290 may be disposed at the top or bottom major surface, or may be disposed directly in the body of the waveguide 182, 184, 186, 188, 190. In some embodiments, the light extraction optical elements 282, 284, 286, 288, 290 may be formed in a layer of material attached to a transparent substrate to form the waveguide 182, 184, 186, 188, 190. In some other embodiments, the waveguide 182, 184, 186, 188, 190 may be a single piece of material and the light extraction optical elements 282, 284, 286, 288, 290 may be formed on the surface of or within the interior of the piece of material.

[0071] Continue to refer Figure 4AAs discussed herein, each waveguide 182, 184, 186, 188, 190 is configured to output light based on a particular depth plane or viewing direction to form a rendered frame or presentation. For example, the waveguide 182 closest to the eye can be configured to pass collimated light injected into the waveguide 182 to the eye 404. The collimated light can represent an optical infinity focal plane. The next upstream waveguide 184 can be configured to emit collimated light that passes through a first lens 192 (e.g., a negative lens) before reaching the eye 404. The first lens 192 can be configured to produce a slightly convex wavefront curvature so that the eye / brain interprets the light from the next upstream waveguide 184 as coming from a first focal plane or viewing direction from optical infinity inward toward the eye 404. Similarly, the third upstream waveguide 186 passes its output light through the first lens 192 and the second lens 194 before reaching the eye 404. The combined optical power of the first and second lenses 192 and 194 can be configured to produce another incremental wavefront curvature such that the eye / brain interprets light from the third waveguide 186 as coming from a second focal plane or viewing direction that is closer to the person inward from optical infinity than light from the next waveguide 184. Thus, one or more waveguides of the waveguide stack can be configured, alone or in combination with other waveguides, as one or more pixels of a lightfield sub-display.

[0072] The other waveguide layers (e.g., waveguides 188, 190) and lenses (e.g., lenses 196, 198) are similarly configured, with the highest waveguide 190 in the stack sending its output through all lenses between it and the eye to obtain a total focal power that represents the focal plane closest to the person. In order to compensate for the lens stack 198, 196, 194, 192 when viewing / interpreting light from the world 144 on the other side of the stacked waveguide assembly 178, a compensating lens layer 180 can be provided at the top of the stack to compensate for the total focal power of the lens stack 198, 196, 194, 192 below. This configuration provides as many perceived focal planes as there are available waveguide / lens pairs. Both the light extraction optics of the waveguide and the focusing aspects of the lens can be static (e.g., not dynamic or electro-active). In some alternative embodiments, one or both of them can be dynamic through the use of electro-active features.

[0073] Continue to refer Figure 4A, the light extraction optical elements 282, 284, 286, 288, 290 can be configured to redirect light out of their respective waveguides and output the light with an appropriate amount of divergence or collimation for a specific depth plane (or viewing direction) associated with the waveguide. Therefore, waveguides with different associated depth planes (or viewing directions) can have different configurations of light extraction optical elements that output light with different amounts of divergence depending on the associated depth plane (or viewing direction). In some embodiments, as discussed herein, the light extraction optical elements 282, 284, 286, 288, 290 can be volume features or surface features that can be configured to output light at specific angles. For example, the light extraction optical elements 282, 284, 286, 288, 290 can be volume holograms, surface holograms, or diffraction gratings. In other embodiments, they can simply be spacers (e.g., cladding or structures for forming air gaps).

[0074] Figure 4B An example of an exit beam output by a waveguide is shown. One waveguide is shown, but it should be understood that other waveguides in the waveguide assembly 178 can play a similar role, where the waveguide assembly 178 includes multiple waveguides. Light 400 is injected into the waveguide 182 at the input edge 382 of the waveguide 182 and propagates within the waveguide 182 by TIR. At the point where the light 400 strikes the light extraction optical element 282, a portion of the light exits the waveguide as an exit beam 402. The exit beams 402 are shown as being substantially parallel, but they can also be redirected to propagate to the eye 404 at an angle (e.g., forming diverging exit beams) that depends on the depth plane or viewing angle associated with the waveguide 182. It should be understood that substantially parallel exit beams can be indicative of a waveguide having light extraction optical elements that extract light to form an image that appears to be set on a depth plane that is a large distance away from the eye 404 (e.g., optical infinity). Other waveguides or other collections of light extraction optical elements may output more divergent exit beam patterns that would require the eye 404 to adapt to a closer distance in order to focus them on the retina, and these exit beam patterns may be interpreted by the brain as light coming from a distance closer to the eye 404 than optical infinity.

[0075] Alternative embodiments for displaying 3D representations of objects

[0076] although Figure 1An example of a display device 100 including a rotatable structure 105 having four elongated elements 102 with light field sub-displays 101 disposed thereon is shown, but in other embodiments, the display device 100 may be configured differently. For example, the rotatable structure may include any number of elongated elements having any shape or size. Furthermore, the rotatable structure may be a single structure having one or more arrays of light field sub-displays. Figures 5A to 5G Some embodiments of display devices 100 according to the disclosure herein are shown, however, other configurations are possible.

[0077] Figure 5A and 5B A display device 100 is shown with a different rotatable structure 105 configured as a propeller, wherein the number and arrangement of the elongated elements 102 are different from Figure 1 The difference shown (the motor 104 and the control system 110 are not shown). For example, Figure 5A A rotatable structure 105a comprising three elongated elements 102a is shown. Figure 1 Each elongated element 102a includes a plurality of light field sub-displays 101. Figure 5A An arrangement of three equally spaced elongate elements 102a is shown, but the elongate elements 102a need not be equally spaced, but may have any spacing therebetween. Figure 5B Another example of a rotatable structure 105b is shown, wherein the rotatable structure 105b includes six elongated elements 102b. The length or width of the elongated elements need not be equal. Figure 5A and 5B As shown, the number of light field sub-displays 101 on each elongated element (102a, 102b) is the same, which need not be the case for all rotatable structural designs. The number of light field sub-displays 101 may vary as required for a particular application of the display device 100.

[0078] In some embodiments, the elongated elements need not be straight, but can have any non-straight shape (e.g., curved, arched, segmented, etc.). Figure 5C Another rotatable structure 105c is shown whose elongated element 102c has an arc shape, wherein the arc is along the same plane on which the light field sub-display 101 is arranged. For example, the elongated element 102c is curved along a plane perpendicular to the rotation axis 120 of the rotatable structure 105c.

[0079] In some embodiments, the elongated elements need not have a square or rectangular cross-section. For example, each elongated element may have a circular or oval cross-section. In other embodiments, the elongated elements may have a cross-section of any polygonal shape (e.g., a triangular, pentagonal, hexagonal, etc. cross-sectional shape). Although Figure 1 and 5A The embodiments shown in 5G show multiple light field sub-displays 101 arranged along a single planar surface perpendicular to the rotation axis 120, but this is not necessarily the case. Figure 5A , a light field sub-display 101a (shown in dashed lines) may optionally be disposed on other surfaces of the elongated element.

[0080] Similarly, each elongated element can rotate about a second rotation axis that is different from the rotation axis 120 of the rotatable structure. Figure 5A , each elongate element 102a may have an axis 530 extending along the elongate element. The display device 100 may then be configured to rotate one or more of the elongate elements 105a about its own axis 530, either individually or in combination.

[0081] In some embodiments, the display device 100 may include a plurality of rotatable structures. For example, Figure 5D A plurality of rotatable structures 105d and 105e are shown which rotate about an axis of rotation 120 independently of one another. Figure 5D Two rotatable structures (105d, 105e) are shown, but 3, 4, 5 or more rotatable structures may be used. Figure 5D As shown, the number of elongated elements 102d and 102e need not be the same on each rotatable structure, however, they can have the same number, shape and arrangement on both rotatable structures. In some embodiments, the rotation speed or rotation direction of the rotatable structure 105d is the same as the rotation speed or rotation direction of the rotatable structure 105e. In another embodiment, the rotation speed or rotation direction is different for different rotatable structures, for example, the rotatable structures rotate in opposite directions. In addition, the number of light field sub-displays 101 provided on each rotatable structure does not need to be the same or need not be arranged in the same manner.

[0082] In some embodiments, in addition to or instead of using a plurality of elongated elements, the rotatable structure 105 of the display device 100 may include a transparent element that can be rotated by the motor 104. The transparent element may be a Perspex disk or a thin two-dimensional polymer, thermoplastic, or acrylic element. For example, Figure 5E and 5F An example of such an arrangement is shown. Figure 5E is a perspective view of an exemplary rotatable structure 105f including a transparent element 510. Fig. 5F It is along Figure 5E AA is a cross-sectional view of the display device 100 shown in FIG. As described above, the light field sub-display 101 can be attached to the transparent element 510 in any suitable arrangement and illuminated by the control system 110. Figure 5E and 5F As shown, the light field sub-display 101 can be arranged on the surface of the transparent element 510 along the elongated direction 502f, so that the arrangement of the light field sub-display 101 is similar to Figure 1 and 5A -5C is shown along the arrangement of the elongated element 102. Although Fig. 5F The light field sub-display 101 is shown located on the upper surface of the transparent element 510, however, the light field sub-display 101 can be attached to the lower surface of the transparent element 510 or disposed within the transparent element 510. For example, the light field sub-display 101 can be attached to the surface of a first transparent disk, followed by a second transparent disk disposed on top of the first disk. These embodiments can advantageously protect the sub-displays from being touched by a viewer or from environmental damage.

[0083] The material of transparent element 510 may be selected to have no or minimal effect on the optical properties of light transmitted from each lightfield sub-display 101 (e.g., the material is substantially transparent in visible light). In other embodiments, transparent element 510 may include color filtering, polarization correction, or other optical properties to be imparted to light emitted from lightfield sub-display 101. Figure 5E and 5F One non-limiting advantage of the display device is that the light field sub-display 101 is attached to or contained in a rotating disk that allows a foreign object (e.g., a hand of a person viewing the image) to be inserted into the rotating disk. Figure 1 and 5A The risk between each arm of the propeller embodiment shown in 5C is minimized, thereby reducing the possibility of damaging the display device 100 or injuring foreign objects.

[0084] Figure 5G An embodiment of a stationary display device is shown. The display device 500 comprises an array of light field sub-displays 101 arranged on a transparent substrate 550 . Figure 5GAn 11×11 array of light field sub-displays 101 is schematically shown, however, any size n×m array of light field sub-displays may be implemented. A subset of the array of light field sub-displays 101 may be formed into elongated features 502g by being illuminated by the control system 110 to produce any number or arrangement of elongated elements 502g. The illuminated subset array of light field sub-displays 101 may be varied at a rotational speed such that the elongated features 502g are electrically rotated about the display device 500. In effect, by sequentially illuminating the elongated features 502g of the light field sub-displays 101, the control system 110 may electronically simulate the physical rotation of a propeller arm.

[0085] At each instant, as elongated feature 502g rotates, the subset array of light field sub-displays 101 that make up elongated feature 502g changes. Thus, by switching light field sub-displays 101 on or off or strobing them, elongated feature 502g may appear to rotate about path 503g. As elongated feature 502g "rotates," light field sub-displays 101 in the subset array of light field sub-displays 101 are controlled by controller 110 to display a 3D representation of an image. Figure 5G One non-limiting advantage of the embodiment shown in is that the display device 500 has no mechanical rotating parts, and the rotation is imparted to the light field sub-display 101 through the processing of the controller. Therefore, there is no rotatable structure that could cause damage or harm to the surrounding area. Figure 5G In the illustrated embodiment, no motor is used because the display device 500 is stationary. However, in other embodiments, a motor may be used to rotate the substrate 550 so that a combination of physical rotation of the substrate 500 and electronic "spin" of the illuminated light field sub-display 101 provides a light field image.

[0086] Examples of non-planar light field displays

[0087] Fig. 6A and 6B is a perspective view of an example of a display device 100 and a plurality of viewers 620a, 620b viewing an example image 610 (of a dog) displayed by the display device 100 in different viewing directions. Fig. 6A and 6B The display device 100 shown may be substantially similar to Figure 1 and 5A A display device 100 for 5G.

[0088] Fig. 6AA viewer 620a is shown positioned generally in front of the display device 100, e.g., at a small angle relative to the direction of the rotation axis 120. The field of view of the display device 100 for the viewer 620a is shown as dashed line 615a. The field of view 615a is wide enough for the viewer 620a to fully view the image displayed by the display device 100.

[0089] In comparison, Figure 6B Viewer 620b is shown positioned so that viewer 620b is viewing image 610 projected by display device 100 at an angle offset from rotation axis 120. As viewer 620b views image 610 at increasing angles relative to rotation axis 120, field of view 615b may become increasingly narrow. This narrow field of view 615b may result in a distorted image, a flat image, or even an invisible image. In some embodiments, this may be because lightfield sub-display 101 is viewed from increasing oblique angles, and lightfield sub-display 101 is unable to direct light at increasing angles relative to rotation axis 120. Due to the 3D lightfield properties of the light projected from display device 100, an off-axis viewer (e.g., viewer 620b) will perceive a different perspective of image 610 projected from the display.

[0090] therefore, Figure 7 An embodiment of a display device 100 is shown that is configured to display a 3D representation of an object at a greater angle relative to the rotation axis 120 . Figure 7 A perspective view of an example of a display device 100 is shown in which the rotatable structure 105 is bent so as to bulge toward a viewer 720a, 720b.

[0091] exist Figure 7 In the illustrated embodiment, the elongated element 102 of the rotatable structure 105 is bent outwardly from a plane perpendicular to the rotation axis 120 to achieve a convexity. An advantage of the display device 100 having a convex rotatable structure 105 is that a viewer (e.g., viewer 720b) ​​who is not directly in front of the display device (e.g., like viewer 720a) can see a substantial field of view 715b of the display device 100 (e.g., in contrast to the field of view 715b of the display device 100). Fig. 6A and 6B The flat rotatable structure has an increased field of view compared to the conventional structure).

[0092] The curvature of the elongated element 102 may be selected to provide a desired field of view for the display device 100. The curvature need not be constant along the elongated element 102, or the same for each elongated element 102. For example, each elongated element 102 may have a different radius of curvature, or a single elongated element 102 may have a radius of curvature that depends on the distance from the axis of rotation or along the elongated element 102.

[0093] In addition, despite Figure 7 The display device 100 has a similar Figure 1 The display device 100 may include a rotatable structure 105 of the display device 100 , but in other embodiments, the display device 100 may include any rotatable structure described herein.

[0094] Example of routines for displaying 3D representations of objects

[0095] Figure 8 800 is a flow chart of an exemplary routine for displaying a 3D representation of an object using a display device as described herein. Routine 800 is an example process for processing light field image data and illuminating a light field sub-display to display a 3D representation of an object or image. Routine 800 may be performed by the control system 110 of an embodiment of the display device 100.

[0096] The routine 800 starts at block 810 and then moves to block 820, where the control system drives the rotatable structure via the motor 104 so that the rotatable structure 105 rotates along the path 103 about the rotation axis 120 at a rotation speed. As the motor 104 drives the rotatable structure 105, the light field sub-display 101 of the rotatable structure 105 is associated with a position based on a rotation angle that varies over time. For a constant rotation speed, the rotation angle is the rotation speed multiplied by time plus an initial rotation angle (at time=0). In some embodiments, the rotation speed may be based in part on the arrangement of the rotatable structure 105 (e.g., the number or spatial arrangement of the elongated elements or sub-displays disposed on the rotatable structure). The rotation speed may also be based in part on the number of rendered frames of the object to be displayed and the object to be represented by the display device 100. As described above, the rotation speed may be fast enough so that the human visual system cannot perceive the elongated elements.

[0097] The routine 800 continues to block 830 where a light field image is accessed, for example from the memory 114 or another separate or remote storage unit. In some embodiments, the image is a light field representation of an object to be displayed. The light field image may be composed of a plurality of rendered frames. Each rendered frame may be a representation of an object to be displayed in a different viewing direction. In this manner, each of the plurality of rendered frames is associated with a viewing direction of the object. In other embodiments, the object images may be ordered so that the object appears to move in space. In this case, the accessed light field image may include a plurality of light field images, where each light field image is a single frame of the video.

[0098] The routine 800 continues to block 840 where the light field image is mapped to the light field sub-display. Figure 1The control system 110 may execute instructions to generate an association or mapping of the selected light field image to each light field sub-display 101 based in part on the rotation angle of the display device. In some embodiments, each rendered frame of the light field image may be mapped to a pixel of the light field sub-display 101 (e.g., Figure 2A and 2B The mapping may be based in part on a rotational speed or rotational angle of the rotatable structure that varies over time. The mapping of the light field image may also include determining which pixels to map to be displayed by the light field sub-display (e.g., Figure 2A and 2B In one embodiment, the mapping of the light field image to the light field sub-display can be performed in accordance with the following method in combination with the light field image. Fig. 9 Detailed description of the routine to perform.

[0099] The routine 800 continues to block 850, where the lightfield sub-display is illuminated. For example, the lightfield sub-display can be illuminated based at least in part on the mapped lightfield image. Figure 1 The control system 110 may execute instructions to cause the light field sub-display 101 to be illuminated based in part on the mapped light field image and the time-varying rotation angle of the rotatable structure 105. In one embodiment, the light field sub-display 101 may be modulated (e.g., turned on and off) based in part on the rendering frame and as a function of time. For example, when the position of the light field sub-display 101 moves due to the rotation of the rotatable structure 105, the rendering frame to be represented may be changed, and the light field sub-display 101 may be switched (e.g., strobed) between multiple rendering frames. In one embodiment, the illumination of the light field sub-display 101 may be based on the following in combination: Fig.10 Detailed description of the routine to perform. Thereafter, at block 860, routine 800 ends.

[0100] In various embodiments, routine 800 may be performed by Figure 1 The hardware processor of the display device 100 (for example, Figure 1 In other embodiments, a remote computing device (in network communication with the display device) having computer executable instructions can cause the display device to perform aspects of routine 800.

[0101] Example of routines for mapping light field images to light field sub-displays

[0102] Fig. 9 is a flow chart of an exemplary routine for mapping a light field image to a light field sub-display. Routine 900 may be Figure 1An example of one method in which the hardware processor 112 of the control system 110 or a remote computing device may map a light field image to each light field sub-display 101 based at least in part on the rotation angle of the rotatable structure 105 .

[0103] The routine 900 starts at block 910 and then moves to block 920, where one or more rendered frames of a light field image are retrieved. For example, at block 920 of the routine 900, a light field image is selected from the digital memory 114 of the control system 110, where the light field image may include a plurality of rendered frames. Each rendered frame may indicate a different view of a plurality of different views of an object. In addition, the rendered frame may include a plurality of rendered pixels that may be combined to represent an image of the object to be displayed. For each rendered pixel of the rendered frame, the routine continues to subroutine 930.

[0104] For each rendered pixel, subroutine 930 continues to block 940 where the position of the given rendered pixel is retrieved. Each rendered pixel may have a position within a rendered frame. For example, a rendered frame may be a 2D representation of an object for a given viewing direction, and each rendered pixel may have a coordinate (e.g., X and Y coordinate) position within the rendered frame. In some embodiments, each rendered frame of the light field image may include the same number of rendered pixels, such that the positions of the rendered pixels are constant between rendered frames.

[0105] At block 950, a time-varying light field sub-display position is determined based at least in part on a rotational speed (which varies over time) of the rotatable structure.

[0106] At block 960, each rendered pixel position of a given rendered pixel is associated with a light field sub-display position. In some embodiments, as described above, the position of a rendered pixel (u) may be associated with a light field sub-display position on the rotatable structure 105 that varies (z) over time (t), where the position of each light field sub-display 101 is based on a rotation angle that varies over time. In some embodiments where the number and position of rendered pixels do not change between rendered frames, the association may be constant for any rendered frame of the light field image. At block 970, the routine 900 may generate (and store) a data structure (e.g., a lookup table (LUT)) that associates rendered pixels with light field sub-display positions. Multiple display devices can select the same lookup table to synchronize images displayed by multiple display devices that are separate or physically separated from each other. At block 980, the routine ends.

[0107] Example of routines for illuminating a light field sub-display

[0108] Fig.10 is used to illuminate a display device (e.g. Figure 1Flow chart of an exemplary routine for a light field sub-display of a display device 100). The routine 1000 may be Figure 1 An example of a method in which a hardware processor 112 of a control system 110 or a remote computing device may be used to illuminate a lightfield sub-display 101 based at least in part on mapped lightfield data.

[0109] The routine 1000 starts at block 1010 and then moves to block 1020 where a light field image is retrieved. As described above, the light field image may include a plurality of rendered frames representing different viewing directions. The plurality of rendered frames may include color and intensity (e.g., image parameters) and other optical properties for a rendered image associated with each rendered pixel of the rendered frame to depict an object in the viewing direction associated with the rendered frame. For each rendered frame, the routine 1000 continues to subroutine 1030.

[0110] For each rendered frame, subroutine 1030 continues to block 1040 where a transformed rendered pixel position is determined. The transformed rendered pixel position may involve a rendered pixel position transformed to an associated light field sub-display position (e.g., as in Fig. 9 In some embodiments, the converted rendering pixel position may be determined by selecting a data structure (e.g., in Fig. 9 960) to execute.

[0111] At block 1050, a color and intensity of light emitted by the lightfield sub-display is determined based at least in part on the rendered frame to be displayed. In one embodiment, the color and intensity may be defined by the rendered pixels to be displayed by the lightfield sub-display 101. For example, referring to Figure 2A and 2B , each rendered frame is associated with a viewing direction. Each pixel (e.g., pixel 205) in the pixel array 225 of the light field sub-display 101 can be associated with a direction of emitted light based on an association with a microlens 215a, which can be mapped to a given rendered pixel. Therefore, each pixel 205 of the pixel array 225 can be associated with a given viewing direction at any time. Based on this association, it can be determined which rendered pixel of the rendered frame will be associated with a given pixel 205 of the pixel array 225. With this association, the subroutine 1030 can retrieve the color and intensity of the rendered pixel to determine the color and intensity of light that a given pixel of the light field sub-display 101 will emit based on the viewing direction of the rendered frame.

[0112] The subroutine 1030 continues to block 1060 where each light field sub-display is illuminated based on the determined color and intensity and the rotation angle of the rotatable structure. For example, as the light field sub-display 101 rotates through the rotation path 103, the rendered frame displayed by the light field sub-display 101 can change based on the change in position. Thus, as the light field sub-display 101 rotates, the pixels 205 of the light field sub-display 1010 can be illuminated or strobed (e.g., alternating or switching between different rendered frames of the light field image) based on the rendered frame to be displayed by the light field sub-display 101. Thereafter, at block 1080, the routine 1000 ends.

[0113] Example of a flat light field display device

[0114] Fig.11 An example of a display device 1100 (eg, a flat screen or flat-panel television in this example) is shown that is configured to display an image viewable as a 3D representation of an object. The display device 1100 includes a display panel 1105 and a control system 1110. Fig.11 In the illustrated embodiment, the display device 1100 may also include a frame 1115 and a bracket 1130 (or other means of securing the display device to a vertical or horizontal surface). The display panel 1105 may include an array of light field sub-displays 1101 disposed on a viewing surface of the display panel and configured to be viewed in a reference viewing direction 1120. The reference viewing direction 1120 may be perpendicular to the plane of the display panel 1105. Thus, the reference viewing direction 1120 points in the direction of a viewer positioned directly in front of the display. The light field sub-displays 1101 may be controlled by a control system 1110 to generate and display a 3D representation of an object. One possible way in which a 3D representation of a display object may be achieved is that a plurality of light field sub-displays 1101 may anisotropically direct light into an array of light beams propagating at different exit angles to produce a light field image. For example, the light field sub-displays 1101 may be substantially similar to a combination of Figures 2A to 3C A light field sub-display 101 is depicted. However, other configurations are possible.

[0115] Fig. 12A and 12B 1 is a top view of an example of a display device 1100 and a plurality of viewers 1220a, 1220b viewing an exemplary image 1210 (an image of a dog in this example) displayed by the display device 1100 in different viewing directions. Fig. 12A and 12B The display device 1100 shown may be substantially similar to Fig.11 Display device 1100.

[0116] Fig. 12A Viewer 1220a is shown positioned generally in front of display device 1100, e.g., at a small angle relative to reference viewing direction 1120. The field of view of display device 1100 for viewer 1220a is shown as dashed line 1215a. Field of view 1215a is wide enough for viewer 1220a to fully view the image displayed by display device 1100.

[0117] In comparison, Fig. 12B Viewer 1220b is shown positioned such that viewer 1220b views image 1210 projected by display device 1100 at an angle that is offset from reference viewing direction 1120. As viewer 1220b views image 1210 at increasing angles relative to reference viewing direction 1120, field of view 1215b may become increasingly narrow. A narrow field of view 1215b may result in a distorted image, a flat image, or even an invisible image. In some embodiments, this may be because the lightfield sub-display 1101 is viewed from increasing oblique angles, and the lightfield sub-display 1101 is unable to direct light at increasing angles relative to reference viewing direction 1120. Due to the 3D lightfield properties of the light projected from display device 1100, an off-axis viewer (e.g., viewer 1220b) will perceive a different perspective of the image 1210 projected from the display.

[0118] Examples of non-planar light field displays

[0119] Fig.13A An example of a display device 1100 is shown that is configured to display an image viewable as a 3D representation of an object. The display device 1100 includes a display panel 1305 and a control system 1110. The control system 1110 can be operably coupled to the display device 1100, which can be mounted in various configurations, such as being fixedly attached to the display device 1100 or located at other locations related to the display device 1100 (e.g., located in a separate part of a room or a central control room). The display panel 1305 is configured to be viewed from a viewing direction and can include an array of light field sub-displays 1101 disposed on a viewing surface. Fig.13A An example of a display device 1100 having a curved display panel 1305 is shown, which is configured to be viewed at a greater angle (e.g., with respect to a reference viewing direction 1120) than a reference viewing direction 1120. Fig.11 Compared to the flat panel display 1100, the 3D representation of the object is displayed at a larger angle relative to the viewing direction. Fig.11 As described above, the reference viewing direction may be perpendicular to a plane tangent to the center of the display (see, for example, Fig.13A). Reference viewing direction 1120 generally points in the direction of a viewer located directly in front of display device 1100. In some embodiments, control system 1110 may be configured to control actuation of each lightfield sub-display 1101 (e.g., the timing, intensity, and color of illumination of each lightfield sub-display 1101) based on a desired image to be projected by display device 1100.

[0120] exist Fig.13A In the illustrated embodiment, the display device 1100 is shown as a television, which can be operated in a manner similar to a liquid crystal display (LCD) television, a light emitting diode (LED) television, or other flat-panel televisions. Such a configuration may include a frame 1115 and a bracket 1130. The bracket 1130 may be configured to support the display device 1100 on a horizontal surface (e.g., a table or a shelf). In another embodiment, the bracket 1130 may be configured as a suspension device that is configured to attach the display device to a vertical surface (e.g., a wall) or to suspend the display device 1100 from an accessory above the display device 1100. The frame 1115 may include a control system 1110 and other electronic and drive circuits for operating the display device 100.

[0121] like Fig.13A As shown, the display panel 1305 has a curvature applied thereto. For example, the display panel 1305 may be configured to be viewed from the reference viewing direction 1120 and to bend outward from a plane perpendicular to the reference viewing direction 1120. The radius of curvature may be configured to provide a curvature with respect to the reference viewing direction 1120. Fig.11 The flat display 1100 has any desired curvature at a greater angle than relative to the viewing direction, as shown below in conjunction with Fig. 13B described. Fig.13A An example of a display device 1100 is shown in which a display panel 1305 is bent outward from a plane perpendicular to a reference viewing direction 1120 so as to bulge relative to the reference viewing direction 1120. For example, Fig.13A Depicted are imaginary axes shown as the X, Y, and Z axes. These axes are for illustration purposes only. Fig.13A In an embodiment, the Z axis may be parallel to the reference viewing direction 1120. The X axis and the Y axis may form a plane perpendicular to the X axis and, for example, the reference viewing direction 120. The X axis and the Y axis may also be perpendicular to each other, where the X axis is a horizontal axis and the Y axis is a vertical axis. Fig.13A As shown, the display panel 1305 can be bent outward from a plane perpendicular to the viewing direction and convexly bend around a vertical (e.g., Y) axis. The display panel 1305 can have a cylindrical shape, for example, the display panel 1305 can resemble a portion of a cylinder standing upright at one end. Fig.13A One example of a display panel 1305 is shown, however, other configurations are possible, for example Figures 14A to 14E shown.

[0122] Continue to refer Fig.13A , the display panel 1305 may include an array of light field sub-displays 101 disposed thereon. Fig.13A 121 light field sub-displays 1101 are shown disposed on the display panel, but other embodiments are possible. For example, the number of light field sub-displays 1101 on the display panel 1305 can be as few as 1, 2, 3, 4, 5, 6, or more, or as many as are required to provide a desired image resolution, as described below. In some embodiments, the display panel 1305 may include a single light field sub-display 1101 disposed thereon. The light field sub-display 1101 can be any display configured to generate a light field. In some embodiments, the light field sub-display 1101 may include one or more pixels configured to emit anisotropic light (e.g., directional emission). For example, as combined Figures 2A to 3C Described in more detail, the light field sub-display 1101 may include a microlens array disposed adjacent to a pixel array that emits light isotropically toward the microlens array. The microlens array redirects light from the pixel array to an array of light beams propagating at different exit angles to produce a light field image. In some embodiments, each microlens in the microlens array may be configured as a pixel of the light field sub-display 1101. In another embodiment, the light field sub-display 1101 may include a waveguide stack assembly that generates light. As described below in conjunction with Figure 4A and 4B described.

[0123] In some embodiments, the light field sub-display 1101 may be controlled by the control system 1110 to generate and display a 3D representation of an object. For example, the control system 1110 may be configured to drive the illumination of the light field sub-display 1101 to display an image viewable by a bystander as a 3D representation of the object to be displayed. In some embodiments, each of the plurality of light field sub-displays 1101 may include one or more pixels, as described below, which may be illuminated according to light field image data stored in a digital memory 1112 (e.g., a non-transitory data memory) of the controller 1110 to display a 3D representation of the object. The illumination of each light field sub-display 1101 may project a representation of an image that can be viewed by a viewer. The image may include an object, graphics, text, etc. The image may be part of a series of image frames projecting an object or object that appears to be moving or changing in a video. The representation may appear to be 3D and may be mistaken by the viewer as a natural object rather than a projection. Because the light is directionally emitted from the light field sub-display 1101, the image may appear to be suspended in mid-air, thereby attracting the attention of passers-by. Thus, the display device 1100 can be advantageously used for advertising, marketing or sales, for demonstrations, or to otherwise generate interest or convey information to viewers. The display device 1100 can be placed in a store front or viewing area where people at a viewing distance from the display device 1100 can view the image displayed by the display device 1100 by looking at the display panel 1305.

[0124] The local data processing module of the computer control system 1110 may include a hardware processor 1112 and a digital memory 1114. In some embodiments, the digital memory 1114 may be a non-volatile memory (e.g., flash memory) or any non-transitory computer-readable medium. The digital memory 1114 may be configured to store data defining instructions for the hardware processor 1112. These instructions may configure the hardware processor 1112 to perform the functions of the display device 1100. For example, both the hardware processor 1112 and the digital memory 1114 may be used to assist in processing, caching, and storing light field data. The data may include data related to: a) a light field image of an object to be displayed, b) a light field sub-display position on the display panel 1305, or c) a mapping of a light field image to a light field sub-display position.

[0125] In some embodiments, the light field image includes multiple rendered frames of the object, where each rendered frame is a 2D representation of the object in a viewing direction (e.g., the direction in which the viewer may be relative to the display device 1100). Each rendered frame may include multiple pixels, hereinafter referred to as rendered pixels, which may be combined to render an image of the object to be displayed. Each rendered pixel may be associated with a position on the rendered frame (e.g., a rendered pixel position). The multiple rendered frames and rendered pixel positions may be stored in the digital memory 1114 for selection and use by the control system 1110. The light field image may include imaging parameters (e.g., the color and intensity of the light used to display the rendered frame), where the imaging parameters are associated with the viewing direction of the rendered frame.

[0126] In some embodiments, the light field sub-display location may be the location of the light field sub-display 1101 on the display panel 1305. In some embodiments, the light field sub-display 1101 may be arranged in an array or grid (eg, a pattern), such as Fig.13A Other configurations are possible. For example, the light field sub-displays 1101 may be arranged in a spiral arrangement extending radially from a center point on the display panel 1305. Alternatively, the light field sub-displays 1101 may be arranged in a plurality of linear arrangements extending outward from a center point (similar to the spokes of a bicycle wheel). The light field sub-display positions may also include the positions of components (e.g., microlenses described below) of each light field sub-display 1101 that vary over time.

[0127] In some embodiments, the hardware processor 1112 may be operably coupled to the digital memory 1114 and configured to analyze and process the data in the digital memory 1114. The hardware processor 1112 may also be operably coupled to each light field sub-display 101 and configured to drive the pixels of each light field sub-display 1101 based on the light field image stored in the digital memory 1114. For example, the hardware processor 1112 may drive each light field sub-display 1101 (e.g., the pixels of the light field sub-display 1101) to emit light associated with the rendered pixel position and image parameters stored in the digital memory 1112. In some embodiments, the image parameters may be configured as characteristics of the rendered pixels that render an image when combined with other rendered pixels of the rendered frame. As non-limiting examples, the image parameters may be configured as color, intensity, shape, brightness, or any other optical characteristics for the rendered image. The directional aspects of the rendered frame may cause a viewer to perceive the illumination from the light field sub-display 1101 as a 3D representation of a displayed object.

[0128] One possible way to enable 3D representation of display objects is that multiple viewpoints can be pre-rendered by the control system 1110 or another rendering engine. For any given arrangement of the light field sub-display 1101 on the display panel 1305, a mapping can be generated or retrieved that maps the pixel position (z) of the light field sub-display 1101 to the rendered pixel (u) of the rendered frame (k) to be displayed. The mapping can be done by the processor 112, which can include a microprocessor or microcontroller, a graphics processing unit (GPU), or dedicated hardware (e.g., a floating point gate array (FPGA) or an application specific integrated circuit (ASIC)).

[0129] In one embodiment, the control system 1110 can be configured to map the rendered pixels of the rendered frame. For example, the rendered frame (k) can be associated with the viewing direction of the object to be displayed, and the rendered pixel (u) can have a position (e.g., represented by coordinates, such as X and Y coordinates or position coordinates) within the rendered frame (k). The mapping can be constant and independent of the object to be displayed, and thus can be pre-calculated and stored (e.g., in the digital memory 1114) in a data structure (e.g., in a lookup table (LUT)).

[0130] In one embodiment, the control system 1110 may also be configured to map the rendered pixel positions to the positions of the lightfield sub-display 1101. For example, each pixel of the lightfield sub-display 1101 may be located at a different position on the display panel 1305. The rendered pixel positions of the light emitted by the pixels of the lightfield sub-display 101 may be converted for the position of the lightfield sub-display 1101. Thus, each rendered pixel position (u) of a rendered frame (k) may be associated with a given position of a pixel of the lightfield sub-display 1101 based on the position (z) of the pixel on the display panel 1105. Thus, the corresponding rendered pixels of each rendered frame may be grouped together and mapped to the pixels of the lightfield sub-display 1101. The mapping is configured such that the rendered pixel positions are converted to the pixels of the lightfield sub-display 1101 so that the light emitted from the lightfield sub-display 1101 is anisotropically directed based on the viewing direction of the rendered frame. This may also be pre-computed and stored (e.g., in digital memory 1114) in a data structure (e.g., in a lookup table (LUT)), which may be the same as or different than the data structure described above. In some embodiments, when displaying different image frames (e.g., the image may be part of a series of image frames projecting an object or subject that appears to be moving or changing in a video), the pixels of the light field sub-display 1101 may be strobed (e.g., alternating or switching between different rendered frames of the light field image) based on the transformed rendered pixel positions of the rendered frame (including image parameters for each rendered pixel).

[0131] In some embodiments, image parameters may be based on the shape of the display panel 305 or the location of the light field sub-display 1101 (or its pixels). For example, due to the curvature of the display panel 1305, some light field sub-displays 1101 are farther from the viewer than other light field sub-displays 1101. Light field sub-displays 1101 disposed near the center of the display panel 1305 may be physically closer to the viewer, while light field sub-displays 101 disposed near the edge of the display panel 1305 (e.g., light field sub-displays on the right or left side of the display panel 1305) are farther from the viewer. In some cases, the apparent intensity of light viewed by a viewer of a displayed object from a light field sub-display 1101 that is far from the viewer tends to be lower than the intensity of light emitted from a light field sub-display 1101 that is closer to the viewer, because the amount of illumination per unit distance is reduced for light field sub-displays 1101 that are farther from the viewer. Therefore, in some embodiments, in order to keep the apparent intensity of the image relatively constant across the display panel 1305, the illumination brightness may be scaled based on the distance from the viewer. For example, illumination can be scaled based on the difference in distance between the nearest lightfield sub-display 1101 and a particular lightfield sub-display 1101. In some embodiments, the scaling can be based on the shape of the display panel 1305. In some embodiments, the scaling can be linear or curved relative to the shape of the display panel 1305. In other embodiments, lightfield sub-displays 1101 at greater distances can have an increased size and / or an increased number of pixels (compared to lightfield sub-displays 1101 closer to the viewer). In other embodiments, more lightfield sub-displays 1101 can be used at greater distances, for example by reducing the spacing between adjacent lightfield sub-displays 1101.

[0132] The control system 110 may include a connection to a network, for example, to receive an image or image display instruction to be displayed by the display device 1100. The display device 1100 may include audio functionality. For example, the display device 1100 may include or be connected to a speaker system 1118 to project audio in combination with the projected image. In some embodiments, the display device 1100 may include a microphone 1119 and voice recognition technology to enable the display device 1100 to receive and process audio commands or comments from viewers. For example, the display device 1100 may be configured to recognize comments from interested viewers and take actions in response to the comments to modify the display device 1100 (e.g., by changing the color of the projected image, changing the projected image, outputting an audio response to the comment, etc.). As an example, in a retail store environment, a display may show an image of a product for sale, and in response to a question about the price of the product, the display may output the price in an audible manner (e.g., "This product is priced at two dollars today") or by a change in the displayed image (e.g., text or graphics showing the price).

[0133] The display device 1100 may include a proximity sensor 1116 to detect whether an object is nearby, and the control system 1110 may take appropriate actions based on the detection. For example, the proximity sensor 1116 may detect a passerby and activate the display device 1100 to display an object that attracts the passerby. In some embodiments, the proximity sensor 1116 may be configured to detect the absence of a viewer and turn off or shut down the display device 1100.

[0134] Not limited to any particular scientific theory, e.g. Fig. 13B As shown, one non-limiting advantage of the embodiments described herein is that a viewer who is not directly in front of the display device 1100 can see a significant field of view of the display device 1100 (e.g., Fig. 12A and 12B The field of view is increased compared to the flat display panel 1105). Fig. 13B Shows Fig.13A 1100 is a top view of an embodiment of a display device 1100 configured to display a 3D representation of an object at a greater angle relative to a reference viewing direction 1120. Fig. 13B A top view of an example of a display device 1100 is shown, in which a display panel 1305 is curved so as to bulge toward a viewer 1220a, 1220b.

[0135] exist Fig. 13B In the illustrated embodiment, as described above, the display panel 1305 is curved outward from a plane perpendicular to the reference viewing direction 1120 to achieve a convexity. One non-limiting advantage of the display device 1100 having a convex display panel 1305 is that a viewer (e.g., viewer 1220b) who is not directly in front of the display device (e.g., like viewer 1220a) can see a significant field of view 1315b of the display device 1100 (e.g., in contrast to the reference viewing direction 1120). Fig. 12A and 12B In some embodiments, the viewer's field of view can be increased in a plane perpendicular to the plane from which the display panel 1305 is curved outward. Fig. 13B As shown, the display panel 1305 is curved outward from the plane formed by the X-axis and the Y-axis, and the field of view 1315b of the viewer 1220b is increased in the plane formed by the X-axis and the Z-axis (relative to the field of view 1215b). Based on the curvature of the display panel 1305, other configurations are also possible, for example, Figures 14A to 14E shown.

[0136] The curvature of the display panel 1305 may be selected to provide a desired field of view for the display device 1100. The curvature need not be constant along the display panel 1305, or need not be the same for each axis of the display panel 1305 (e.g., Figures 14A to 14E For example, the radius of curvature about the Y axis may be different from the radius of curvature about the X axis. Alternatively, the display panel 1305 may have a radius of curvature that varies about one or both axes. Thus, although Fig. 13B shows a similar Fig. 13B The display device 1100 may include a display panel 1305 including a display panel 1305 , but in other embodiments, the display device 1100 may include any display panel as described herein.

[0137] Although examples of apparatus for generating a light field are described herein and with reference to Figures 2A to 4B As shown, it should be understood that it is not necessary to use a single light field sub-display type to display a 3D representation of an object in a display device. Other light field displays can be envisioned so that multiple light field sub-displays are arranged on a curved display panel to produce a 3D representation of an object. For example, any of the light field sub-displays, components or arrangements described in U.S. Patent Application No. 62 / 288,680, entitled “Holographic Propeller,” filed on January 29, 2016, and U.S. Patent Application No. 62 / 343,722, entitled “Display for Three-Dimensional Image,” filed on May 31, 2016, can be implemented for displaying a 3D representation of an object, wherein the entire contents of each patent application are incorporated herein by reference.

[0138] Alternative embodiments for displaying 3D representations of objects

[0139] although Fig.11 An example of a display device 1100 including a display panel 1305 that is curved outward from a plane perpendicular to the viewing direction is shown, but in other embodiments, the display device 1100 may be configured differently. For example, the display device 1100 may include any number of light field sub-displays, for example, the display panel 1305 may include a single light field sub-display disposed over the entire display panel 1305. In another embodiment, in combination or alternatively, the display panel 1305 may have any shape or size. Figures 14A to 14E Some embodiments of display devices 1100 according to the present disclosure are shown, however, other configurations are possible. In some embodiments, various configurations can produce a display panel shaped as a portion of a cylinder, a sphere, an oblate spheroid, or a prolate spheroid.

[0140] Fig.14A The display device 1100 includes a display panel 1305a, and the display panel 1305a is configured to have different Fig.13A The curvature of the display panel 1305. For example, Fig.14A Imaginary axes X, Y, and Z are shown for illustration purposes only and may be substantially similar to Fig.13A The axis shown. Therefore, Fig.14A The display panel 1305a is shown to bend outward from a plane perpendicular to the reference viewing direction 1120 (e.g., Z axis) and convexly bend around a horizontal (e.g., X) axis. The display panel 1305a can have a cylindrical shape, for example, the display panel 1305a can resemble a portion of a cylinder lying on a curved surface.

[0141] In some embodiments, the display device 1100 may include a display panel 1305 that is curved about two axes, for example, Fig. 14B and 14C For example, Fig. 14B An embodiment of a display panel 1305b that is curved about a horizontal (e.g., X) axis and a vertical (e.g., Y) axis is shown. In some embodiments, the radii of curvature about the horizontal and vertical axes can be substantially the same. Similar sized radii can result in the display panel 1305b being shaped as a portion of a sphere. In other embodiments, the radii of curvature about the horizontal and vertical axes can be different, for example, Fig. 14C shown. Fig. 14C Shown with Fig. 14B Similar to the embodiment of display panel 1305c, however, the radius of curvature about the horizontal (e.g., X) axis can be smaller than the radius of curvature about the vertical (e.g., Y) axis. The radius of curvature about the horizontal axis being smaller than the radius about the vertical axis can cause the display panel 1305c to be shaped as a portion of an oblate spheroid. However, in some embodiments, the radius of curvature about the horizontal (e.g., X) axis can be larger than the radius of curvature about the vertical (e.g., Y) axis, which can cause the display panel (not shown) to be shaped as a portion of a prolate spheroid.

[0142] Other configurations are possible. For example, the axes may be viewed as first, second, and third axes. Similar to the axes described above, the third axis may be parallel to the reference viewing direction 1120, and the first and second axes may form a plane perpendicular to the third axis. However, the first and second axes need not be perpendicular to each other, and may be at an angle less than or greater than 90 degrees to each other. In some embodiments, in combination or alternatively, the first and second axes also need not be horizontal or vertical, and may be relative to the combination. Fig.13A and 14A Thus, the display panel 1305 can be bent outward from a plane perpendicular to the viewing direction, and can be bent about a first axis, a second axis, or both.

[0143] In some embodiments, the display device 1100 may include a display panel 1305 including a plurality of display regions (eg, Fig.14D and 14E ). For example, the display panel may be graphically divided into a plurality of display regions, each display region including a portion of a light field sub-display or one or more separate light field sub-displays. Each display region may include a planar surface or a curved surface. For example, Fig.14D An embodiment of a display panel 1305d is shown that includes multiple display regions (e.g., display regions 1460d, 1470d, and 1480d) divided along illustrative lines 1465d and 1485e (shown in dashed lines). Display region 1470d can be located near or around a center region of display panel 1305d. In this embodiment, display region 1470d can be uncurved such that display region 1470d is substantially perpendicular to reference viewing direction 1120. Display regions 1460d and 1480d can be curved outward from a plane perpendicular to the viewing direction in a manner substantially similar to that described herein. Similarly, Fig.14E An embodiment of a display panel 1305e is shown, which includes two display regions 1460e and 1470e (shown by dashed circles) divided along illustrative line 1465e. Display region 1470e may be similar to Fig.14D 1470e is substantially perpendicular to the reference viewing direction 1120. Display region 1460e can be curved outward from a plane perpendicular to the viewing direction in a manner substantially similar to that described herein. Thus, display panel 1305e can be a partial sphere (or, in some embodiments, a sphere) having a planar surface at display region 1470e.

[0144] Although certain embodiments have been described herein, other configurations are possible. For example, the display panel may include any number of display areas, e.g., 1, 2, 4, 5, 6, etc. In some embodiments, the curvature applied to each display area need not be the same, and may be different for each display area compared to other display areas. In other embodiments, the shape or configuration of the display areas of the display panel need not be symmetrical. In some embodiments, the display area disposed near the central area need not be planar and may be curved, while other display areas at the edge of the display panel or away from the central area may be planar.

[0145] Example of routines for displaying 3D representations of objects

[0146] Fig.151500 is a flow chart of an exemplary routine for displaying a 3D representation of an object using a display device described herein. Routine 1500 is an example of a process for processing a light field image and illuminating a light field sub-display to display a 3D representation of an object or image. Routine 1500 may be performed by the control system 1110 of an embodiment of the display device 1100.

[0147] At block 1510, a light field image is selected, for example, from memory 1114 or another separate or remote storage unit. In some embodiments, the image is a light field representation of an object to be displayed. The light field image may be composed of a plurality of rendered frames. Each rendered frame may be a representation of an object to be displayed in a different viewing direction. In this manner, each of the plurality of rendered frames is associated with a viewing direction of the object. In other embodiments, the image of the object may be ordered so that the object appears to move in space. In this case, the selected light field image may include a plurality of light field images, where each light field image is a single frame of the video.

[0148] The routine 1500 continues to block 1520, where the light field image is mapped to a light field sub-display. Fig.13A The control system 1110 may execute instructions to generate an association or mapping of the selected light field image to each light field sub-display 1101 based in part on the position of the light field sub-display 1101 on the display panel 1305. In some embodiments, each rendered frame of the light field image may be mapped to a pixel (e.g., a microlens) of the light field sub-display 1101. The mapping may be based in part on the pixel (e.g., Figure 2A and 2B The mapping of the light field image may also include determining the position of the microlenses of the light field sub-display 1101 on the display panel 1305. Figure 2A and 2B In one embodiment, the mapping of the light field image to the light field sub-display 1101 can be based on the following combination of Fig.16 The routines described in detail are executed.

[0149] The routine 1500 continues to block 1530, where the lightfield sub-display is illuminated. For example, the lightfield sub-display can be illuminated based at least in part on the mapped lightfield image. Fig.13AThe control system 1110 of the embodiment may execute instructions to cause the light field sub-display 1101 to be illuminated based in part on the mapped light field image and the position of the light field sub-display 1101 on the display panel 1305. In one embodiment, the light field sub-display 1101 may be modulated (e.g., turned on and off) over time based in part on switching between object rendering frames configured so that the object appears to move in space. In one embodiment, the illumination of the light field sub-display 1101 may be based on the following in combination with Fig.17 The routine 1500 is executed as described in detail. Thereafter, the routine 1500 ends.

[0150] In various embodiments, routine 1500 may be performed by Fig.13A The hardware processor of the display device 1100 (for example, Fig.13A In other embodiments, a remote computing device (in network communication with the display device) having computer executable instructions can cause the display device to perform aspects of routine 1500.

[0151] Example of routines for mapping light field images to light field sub-displays

[0152] Fig.16 is a flow chart of an exemplary routine for mapping a light field image to a light field sub-display. The routine 1600 may be Fig.13A An example of one method in which a hardware processor 1112 of a control system 1110 or a remote computing device may map a light field image to each light field sub-display 1101 based at least in part on the position of the light field sub-display 101 on the display panel 1305 .

[0153] The routine 1600 begins at block 1610, where one or more rendered frames of a light field image are retrieved. For example, at block 1610, a light field image is selected from the digital memory 1114 of the control system 1110, where the light field image may include a plurality of rendered frames. Each rendered frame may indicate a different view of a plurality of different views of an object. Furthermore, a rendered frame may include a plurality of rendered pixels that may be combined to represent an image of an object to be displayed. The routine continues to a subroutine 1620 for each rendered pixel of a rendered frame.

[0154] For each rendered pixel, subroutine 1620 continues to block 1630 where a position of each rendered pixel is retrieved. Each rendered pixel may have a position within a rendered frame. For example, a rendered frame may be a 2D representation of an object for a given viewing direction, and each rendered pixel may have a coordinate (e.g., X and Y coordinate) position within the rendered frame. In some embodiments, each rendered frame of a light field image may include the same number of rendered pixels such that the positions of the rendered pixels are constant between rendered frames.

[0155] At block 1640, a light field sub-display position is determined based on the position of the light field sub-display on the display panel. Fig.13A Each lightfield sub-display 1101 of can have a position (e.g., X and Y coordinates) within display panel 1305. In some embodiments, a distance from a plane perpendicular to a reference viewing direction 1120 (e.g., a Z coordinate) of each lightfield sub-display 1101 (or, for example, a pixel thereof) can be determined. In some embodiments, the position of the lightfield sub-display can remain stationary, so the determination can be pre-generated or stored in a memory (e.g., memory 1114 or a remote memory device) and retrieved or selected by a processor (e.g., processor 1112).

[0156] At block 1650, each rendered pixel position is associated with a light field sub-display position. In some embodiments, as described above, the position of a rendered pixel (u) may be associated with a position (z) of a light field sub-display on display panel 1305 (e.g., a pixel position of light field sub-display 1101). In some embodiments where the number and position of rendered pixels remain constant between rendered frames, this association may be constant for any rendered frame of the light field image. At block 1660, routine 1600 may generate (and store) a data structure (e.g., a lookup table (LUT)) that associates rendered pixels with light field sub-display positions. Multiple display devices may be able to select the same lookup table to synchronize images displayed by multiple display devices that are separate or physically separated from one another. Thereafter, routine 1600 ends.

[0157] Example of routines for illuminating a light field sub-display

[0158] Fig.17 is used to illuminate a display device (e.g. Fig.13A 1700) of a light field sub-display. Fig.13A An example of a method in which a hardware processor 1112 of a control system 1110 or a remote computing device may be used to illuminate a lightfield sub-display 1101 based at least in part on mapped lightfield data.

[0159] The routine 1700 begins at block 1710 where a light field image is retrieved. As described above, the light field image may include a plurality of rendered frames representing different viewing directions. The plurality of rendered frames may include a color and intensity (e.g., image parameters) and other optical characteristics for the rendered image associated with each rendered pixel of the rendered frame so as to depict an object in the viewing direction associated with the rendered frame. The routine 1700 continues to a subroutine 1720 for each rendered frame.

[0160] For each rendered frame, subroutine 1720 continues to block 1730 where a transformed rendered pixel position is determined. The transformed rendered pixel position may relate to a rendered pixel position that is transformed to a position of an associated light field sub-display (e.g., as in Fig.16 In some embodiments, the converted rendering pixel position may be determined by selecting a data structure (e.g., in Fig.16 1660).

[0161] At block 1740, a color and intensity of light emitted by the lightfield sub-display is determined based at least in part on the rendered frame to be displayed. In one embodiment, the color and intensity may be defined by the rendered pixels to be displayed by the lightfield sub-display 1101. For example, referring to Figure 2A and 2B , each rendered frame is associated with a viewing direction. Each pixel (e.g., pixel 205) in the pixel array 220 of the light field sub-display 101 can be associated with an emission direction based on an association with a microlens 215a, which can be mapped to a given rendered pixel. Therefore, each pixel 205 of the pixel array 220 can be associated with a given viewing direction at any time. Based on this association, it can be determined which rendered pixel of the rendered frame will be associated with a given pixel 205 of the pixel array 220. Through this association, the subroutine 1720 can retrieve the color and intensity of the rendered pixel to determine the color and intensity of light that a given pixel of the light field sub-display 101 will emit based on the viewing direction of the rendered frame.

[0162] In some embodiments, at block 1740, the intensity of light emitted by the lightfield sub-display may be determined based on the position of the lightfield sub-display relative to the viewer. For example, as described above, due to the curvature of the display panel 1305, some lightfield sub-displays 1101 are farther from the viewer than other lightfield sub-displays 1101. Lightfield sub-displays 1101 disposed near the center of the display panel 1305 may be physically closer to the viewer, while lightfield sub-displays 1101 disposed near the edge of the display panel 1305 are farther from the viewer. In some cases, the apparent intensity of light viewed by a viewer of a displayed object from a lightfield sub-display 1101 that is farther from the viewer than the intensity of light emitted from a lightfield sub-display 1101 that is closer to the viewer is often lower because the amount of illumination per unit distance is reduced for lightfield sub-displays 1101 that are farther from the viewer. Therefore, in some embodiments, in order to keep the apparent intensity of the image relatively constant across the display panel 1305, the illumination brightness may be scaled based on the distance from the viewer. For example, the illumination may be scaled based on the distance difference between the nearest lightfield sub-display 1101 and a particular lightfield sub-display 1101. In some embodiments, the scaling may be based on the shape of the display panel 1305. In some embodiments, the scaling may be linear or curved relative to the shape of the display panel 1305.

[0163] The subroutine 1720 continues to block 1750, where each lightfield sub-display is illuminated based on the determined color and intensity and the location of the lightfield sub-display on the display panel. For example, the control system 1110 may execute instructions to illuminate each lightfield sub-display 1101 of the display panel 1305 based on the determined color and intensity associated with the converted rendering pixel location. Thereafter, at block 1750, the routine 1700 ends.

[0164] Other aspects

[0165] In a first aspect, a display device for displaying a 3D representation of an object is disclosed. The display device includes: a rotatable structure; a motor configured to rotate the rotatable structure; a plurality of light field sub-displays disposed on the rotatable structure; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory, the motor, and the light field sub-displays, the processor being programmed with executable instructions to: drive the motor to rotate the rotatable structure about a rotation axis, the rotatable structure being positioned at a time-varying rotation angle; select the light field image; map the light field image to each of the plurality of light field sub-displays based at least in part on the rotation angle; and illuminate the plurality of light field sub-displays based at least in part on the mapped light field image.

[0166] In a second aspect, the device according to aspect 1, wherein the rotatable structure comprises a plurality of elongated elements, and the plurality of light field sub-displays are arranged along the elongated elements or the transparent rotatable element.

[0167] In a third aspect, the device according to aspect 1 or 2, wherein the plurality of elongated elements are curved along a plane perpendicular to the rotation axis.

[0168] In a 4th aspect, the device according to any one of aspects 1 to 3, wherein the plurality of elongated elements are bent outwardly from a plane perpendicular to the rotation axis.

[0169] In a 5th aspect, the device according to any one of aspects 1 to 4, wherein the display device is configured to be viewed from a viewing direction, and the plurality of elongated elements protrude from the viewing direction.

[0170] In a 6th aspect, according to any one of aspects 1 to 5, the device, wherein at least a portion of the rotatable structure is transparent.

[0171] In a seventh aspect, the device according to any one of aspects 1 to 6, wherein the plurality of light field sub-displays are arranged radially from the rotation axis.

[0172] In an eighth aspect, the apparatus of any one of aspects 1 to 7, wherein each light field sub-display has a corresponding radius based on its position relative to the rotation axis, and wherein to illuminate the plurality of light field sub-displays, the processor is programmed to scale the intensity or duration of illumination of the light field sub-displays based on the radius.

[0173] In a 9th aspect, the apparatus of any one of aspects 1 to 8, wherein the scaling is linearly related to the radius of the light field sub-display.

[0174] In aspect 10, the device of any one of aspects 1 to 9, wherein each lightfield sub-display comprises: a microlens array comprising a plurality of microlenses; and a pixel array comprising a plurality of pixel subsets, each pixel subset being associated with a microlens and configured to generate light, wherein each pixel subset and the associated microlens are arranged to generate outgoing light at a plurality of angles, wherein light from a first pixel of the pixel subset propagates from the lightfield sub-display at an angle that is different from an angle of a second pixel of the pixel subset.

[0175] In an 11th aspect, the apparatus of any one of aspects 1 to 10, wherein the subset of pixels is located near a focal point of the associated microlens.

[0176] In a 12th aspect, the apparatus of any one of aspects 1 to 11, wherein each lightfield sub-display comprises a stacked waveguide assembly, the stacked waveguide assembly comprising one or more waveguides, wherein each of the one or more waveguides is configured to project light of one or more of the multiple different views of the object.

[0177] In a 13th aspect, the apparatus of any one of aspects 1 to 12, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels that, when combined, render the rendered frame, each rendered pixel having a position within the rendered frame.

[0178] In a 14th aspect, an apparatus according to any one of aspects 1 to 13, wherein to map the light field image to the plurality of light field sub-displays based at least in part on the rotation angle, the processor is programmed to associate the position of each rendered pixel with the position of each light field sub-display on the rotatable structure, wherein the position of each light field sub-display is based on the rotation angle that varies over time.

[0179] In a 15th aspect, according to any one of aspects 1 to 14, the device, wherein the rendering pixel position does not change between the multiple rendering frames.

[0180] In a 16th aspect, according to the apparatus of any one of aspects 1 to 15, mapping the light field image to each of the plurality of light field sub-displays further comprises: for each light field sub-display, the processor is programmed to determine the color and intensity based on the rendered frame to be displayed and the association of the position of each rendered pixel with the position of each light field sub-display on the rotatable structure.

[0181] In aspect 17, the apparatus of any one of aspects 1 to 16, wherein to illuminate the plurality of light field sub-displays, the processor is programmed to: illuminate each light field sub-display based on the determined color and intensity for a given rendering frame, wherein the direction of illumination is related to a viewing direction of the rendering frame, and strobe the illumination of each light field sub-display based on a rotation of the rotatable structure, the plurality of rendering frames, and an association of a position of each rendering pixel with a position of each light field sub-display on the rotatable structure.

[0182] In an 18th aspect, the apparatus of any one of aspects 1 to 17, further comprising a speaker system configured to project audio in combination with the processor programmed to illuminate the plurality of lightfield sub-displays.

[0183] In a 19th aspect, the apparatus of any one of aspects 1 to 18 further comprises: a microphone configured to receive audio, and wherein the processor is programmed with executable instructions to: receive audio input from the microphone; recognize that the audio input is an audio command; and initiate an action to modify the illumination of the plurality of lightfield sub-displays based on the audio command.

[0184] In aspect 20, the device according to any one of aspects 1 to 19 further comprises a proximity sensor, wherein the proximity sensor is configured to detect an entity within a predetermined distance of the display device, and wherein the processor is programmed with executable instructions to initiate an action based on the detection of the entity by the proximity sensor.

[0185] In a 21st aspect, a method for displaying a 3D representation of an object is disclosed. The method includes: driving a motor to rotate a rotatable structure including a plurality of light field sub-displays about a rotation axis, the rotatable structure being positioned at a rotation angle that varies over time; selecting a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; mapping the light field image to each of the plurality of light field sub-displays based at least in part on the rotation angle; and illuminating the plurality of light field sub-displays based at least in part on the mapped light field image.

[0186] In aspect 22, the method of aspect 21, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels, the plurality of rendered pixels are combined to render the rendered frame, each rendered pixel having a position within the rendered frame.

[0187] In aspect 23, a method according to aspect 21 or 22, wherein mapping the light field image to the multiple light field sub-displays based at least in part on the rotation angle includes: associating the position of each rendered pixel with the position of each light field sub-display on the rotatable structure, wherein the position of each light field sub-display is based on the rotation angle that varies over time.

[0188] In aspect 24, the method according to any one of aspects 21 to 23, wherein the rendered pixel position does not change between the plurality of rendered frames.

[0189] In aspect 25, according to a method according to any one of aspects 21 to 24, mapping the light field image to each of the multiple light field sub-displays further comprises: for each light field sub-display, determining a color and intensity based on a rendered frame to be displayed and an association of a position of each rendered pixel with a position of each light field sub-display on the rotatable structure.

[0190] In aspect 26, a method according to any one of aspects 21 to 25, wherein illuminating the plurality of light field sub-displays comprises: for a given rendering frame, illuminating each light field sub-display based on the determined color and intensity, wherein the direction of illumination is related to the viewing direction of the rendering frame; and strobing the illumination of each light field sub-display based on the rotation of the rotatable structure, the plurality of rendering frames, and the association of the position of each rendering pixel with the position of each light field sub-display on the rotatable structure.

[0191] In a 27th aspect, a display device for displaying a 3D representation of an image is disclosed. The display device includes: a light field sub-display configured to rotate, the light field sub-display having a plurality of display positions; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; a processor operably coupled to the non-transitory memory and the light field sub-display, the processor being programmed with executable instructions to: rotate the light field sub-display about a rotation axis, wherein the plurality of display positions are based on a rotation angle that varies over time; select the light field image; map the light field image to the plurality of display positions based at least in part on the rotation angle; and illuminate the light field sub-display based at least in part on the mapped light field image.

[0192] In aspect 28, a display device for displaying a 3D representation of an image is disclosed. The display device includes: one or more light field sub-displays, each of the one or more light field sub-displays having a plurality of display positions, wherein the one or more light field sub-displays are configured to rotate about one or more rotation axes; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; a processor operably coupled to the non-transitory memory and the one or more light field sub-displays, the processor being programmed with executable instructions to: drive the one or more light field sub-displays to rotate about at least one of the rotation axes, wherein the plurality of display positions are based on a rotation angle that varies over time; and illuminate the one or more light field sub-displays based at least in part on the light field image and the plurality of display positions.

[0193] In aspect 29, a display device for displaying a 3D representation of an object is disclosed. The display device includes: a display panel configured to be viewed from a reference viewing direction, wherein the display panel is curved outward from a plane perpendicular to the reference viewing direction; a plurality of light field sub-displays disposed on the display panel; a non-transitory memory configured to store a light field image to be displayed by the display device; the light field image providing a plurality of different views of the object in different viewing directions; and a processor operably coupled to the non-transitory memory and the light field sub-display, the processor being programmed with executable instructions to: select the light field image; map the light field image to each of the plurality of light field sub-displays based at least in part on a position of the light field sub-display on the display panel; and illuminate the plurality of light field sub-displays based at least in part on the mapped light field image.

[0194] In a 30th aspect, according to the device of aspect 29, the display panel is convex relative to the reference viewing direction.

[0195] In a 31st aspect, the apparatus of aspect 30, wherein the first axis is horizontal and the display panel is curved about the first axis.

[0196] In a 32nd aspect, the apparatus of aspect 30 or 31, wherein the second axis is vertical and the display panel is curved about the second axis.

[0197] In a 33rd aspect, the device of any one of aspects 30 to 32, wherein the first axis and the second axis are perpendicular, and the display panel is curved about the first axis and the second axis.

[0198] In a 34th aspect, the device of any one of aspects 30 to 33, wherein the radius of curvature about the first axis is different from the radius of curvature about the second axis.

[0199] In a 35th aspect, the device of any one of aspects 30 to 33, wherein the radius of curvature about the first axis is substantially the same as the radius of curvature about the second axis.

[0200] In aspect 36, the device of any one of aspects 29 to 35, wherein the display panel comprises a first axis and a second axis perpendicular to the reference viewing direction.

[0201] In aspect 37, the device according to any one of aspects 29 to 36, wherein the display panel includes multiple display areas, at least one display area is bent outward from a plane perpendicular to the reference viewing direction.

[0202] In a 38th aspect, the apparatus of aspect 37, wherein at least one of the display regions is substantially perpendicular to the reference viewing direction.

[0203] In a 39th aspect, the device of any one of aspects 29 to 38, wherein the display panel has a shape of at least one of a cylinder, a sphere, an oblate spheroid, and a prolate spheroid.

[0204] In aspect 40, the apparatus of any one of aspects 29 to 39, wherein to illuminate the plurality of light field sub-displays, the processor is programmed to scale the intensity or duration of illumination of the light field sub-displays based on the position of a light field sub-display on the display panel relative to the position of another light field sub-display.

[0205] In aspect 41, the apparatus of any one of aspects 29 to 40, wherein each lightfield sub-display comprises: a microlens array comprising a plurality of microlenses; and a pixel array comprising a plurality of pixel subsets, each pixel subset being associated with a microlens and configured to generate light, wherein each pixel subset and associated microlens are arranged to generate outgoing light at a plurality of angles, wherein light from a first pixel of the pixel subset propagates from the lightfield sub-display at an angle that is different from an angle of a second pixel of the pixel subset.

[0206] In a 42nd aspect, the apparatus of aspect 41, wherein the subset of pixels is located near a focal point of the associated microlens.

[0207] In a 43rd aspect, a device according to any one of aspects 29 to 42, wherein each light field sub-display comprises a stacked waveguide assembly, the stacked waveguide assembly comprising one or more waveguides, wherein each of the one or more waveguides is configured to project light of one or more views of the multiple different views of the object.

[0208] In a 44th aspect, an apparatus according to any one of aspects 29 to 43, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels that, when combined, render the rendered frame, each rendered pixel having a position within the rendered frame.

[0209] In aspect 45, the apparatus of aspect 44, wherein to map the light field image to the plurality of light field sub-displays, the processor is programmed to associate the position of each rendered pixel with the position of each light field sub-display on the display panel.

[0210] In aspect 46, the apparatus of aspect 45, wherein the rendered pixel position is constant between the plurality of rendered frames.

[0211] In a 47th aspect, according to an apparatus as described in any one of aspects 44 to 46, mapping the light field image to each of the plurality of light field sub-displays further comprises: for each light field sub-display, the processor is programmed to determine the color and intensity based on the rendered frame to be displayed and the association of the position of each rendered pixel with the position of each light field sub-display on the display panel.

[0212] In a 48th aspect, an apparatus according to aspect 47, wherein to illuminate the plurality of light field sub-displays, the processor is programmed to: for a given rendering frame, illuminate each light field sub-display based on the determined color and intensity, wherein the direction of illumination is related to the viewing direction of the rendering frame and the association of the position of each rendering pixel with the position of each light field sub-display on the display panel.

[0213] In a 49th aspect, the apparatus of any one of aspects 29 to 48 further comprises a speaker system configured to project audio in combination with the processor programmed to illuminate the plurality of lightfield sub-displays.

[0214] In a 50th aspect, the apparatus of any one of aspects 29 to 49 further comprises: a microphone configured to receive audio, and wherein the processor is programmed with executable instructions to: receive audio input from the microphone; recognize that the audio input is an audio command; and initiate an action to modify the illumination of the plurality of light field sub-displays based on the audio command.

[0215] In aspect 51, the device according to any one of aspects 29 to 50 further comprises a proximity sensor, wherein the proximity sensor is configured to detect the presence or absence of an entity within a predetermined distance of the display device, and wherein the processor is programmed with executable instructions to initiate an action based on the proximity sensor detecting the presence or absence of the entity.

[0216] In a 52nd aspect, a method for displaying a 3D representation of an object is disclosed. The method comprises: selecting a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; mapping the light field image to each of the plurality of light field sub-displays based at least in part on a position of the light field sub-display on the display panel; and illuminating the plurality of light field sub-displays based at least in part on the mapped light field image.

[0217] In a 53rd aspect, the method of aspect 52, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels, the plurality of rendered pixels are combined to render the rendered frame, each rendered pixel having a position within the rendered frame.

[0218] In aspect 54, according to the method of aspect 53, mapping the light field image to the multiple light field sub-displays based at least in part on the position of the light field sub-display on the display panel includes: associating the position of each rendered pixel with the position of each light field sub-display on the display panel.

[0219] In aspect 55, according to the method of aspect 54, mapping the light field image to each of the multiple light field sub-displays based at least in part on the position of the light field sub-display on the display panel further includes: for each light field sub-display, determining the color and intensity based on the rendered frame to be displayed and the association of the position of each rendered pixel with the position of each light field sub-display on the display panel.

[0220] In a 56th aspect, according to the method of aspect 55, illuminating the multiple light field sub-displays based at least in part on the mapped light field image further comprises: for a given rendering frame, illuminating each light field sub-display based on the determined color and intensity, wherein the direction of illumination is related to the viewing direction of the rendering frame.

[0221] In aspect 57, the method of any one of aspects 52 to 56, wherein the rendered pixel position is constant between the plurality of rendered frames.

[0222] In aspect 58, a display device for displaying a 3D representation of an image is disclosed. The display device includes: a display panel configured to be viewed from a reference viewing direction, wherein the display panel is curved outward from a plane perpendicular to the reference viewing direction; one or more light field sub-displays, each of the one or more light field sub-displays having a position on the display panel; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; a processor operably coupled to the non-transitory memory and the light field sub-display, the processor being programmed with executable instructions to: select the light field image, and illuminate the one or more light field sub-displays based at least in part on the light field image and the position of the one or more light field sub-displays on the display panel.

[0223] In aspect 59, a light field display device is disclosed. The light field display device comprises: a curved panel comprising a plurality of light field sub-displays.

[0224] In a 60th aspect, the apparatus of aspect 59, wherein the curved panel is curved in a horizontal direction, in a vertical direction, or in both the horizontal and vertical directions.

[0225] in conclusion

[0226] Each of the processes, methods, and algorithms described herein or depicted in the accompanying drawings may be embodied in a code module executed by one or more physical computing systems, hardware computer processors, dedicated circuits, and / or electronic hardware configured to execute specific and particular computer instructions, and fully or partially automated by the above. For example, a computing system may include a general-purpose computer (e.g., a server) or a dedicated computer, dedicated circuit, or the like programmed with specific computer instructions. The code module may be compiled and linked into an executable program, installed in a dynamic link library, or may be written in an interpreted programming language. In some embodiments, specific operations and methods may be performed by circuits specific to a given function.

[0227] Furthermore, certain implementations of the functionality of the present disclosure are sufficiently mathematically, computationally or technically complex that specialized hardware or one or more physical computing devices (with appropriate specialized executable instructions) or a dedicated graphics processing unit may be required to perform the functionality, e.g., due to the number or complexity of the calculations involved or in order to provide graphical display results in substantially real time. For example, a video may include many frames, each frame having millions of pixels, and specially programmed computer hardware is required to process the video data in order to provide the desired image processing task or application in a commercially reasonable amount of time.

[0228] Code modules or any type of data may be stored on any type of non-transitory computer-readable medium, such as physical computer memory, including hard drives, solid-state memory, random access memory (RAM), read-only memory (ROM), optical disks, volatile or non-volatile memory, combinations of the same and / or the like. Methods and modules (or data) may also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagation signal) on various computer-readable transmission media, including wireless-based and wire / cable-based media, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps may be stored persistently or otherwise in any type of non-transitory, tangible computer memory, or may be transmitted via a computer-readable transmission medium.

[0229] Any process, frame, state, step or function in the flowchart described herein / or depicted in the accompanying drawings should be understood to potentially represent a code module, code segment or code portion, which includes one or more executable instructions for implementing a specific function (e.g., logic or arithmetic) or step. The process, frame, state, step or function can be combined, rearranged, added, deleted, modified or otherwise changed with the illustrative examples provided herein. In some embodiments, additional or different computing systems or code modules can perform some or all of the functions described herein. The methods and processes described herein are also not limited to any particular order, and the blocks, steps or states associated therewith can be performed in other appropriate orders, such as serial, parallel or in some other manner. Tasks or events can be added to or removed from the disclosed example embodiments. In addition, the separation of various system components in the embodiments described herein is for illustrative purposes, and should not be understood as requiring such separation in all embodiments. It should be understood that the described program components, methods and systems can generally be integrated together in a single computer product or packaged into multiple computer products. Many implementation variations are possible.

[0230] The processes, methods and systems can be implemented in a network (or distributed) computing environment. For example, the control system 110 can communicate with a network environment. Network environments include enterprise-wide computer networks, intranets, local area networks (LANs), wide area networks (WANs), personal area networks (PANs), cloud computing networks, crowd-sourced computing networks, the Internet, and the World Wide Web. The network can be a wired or wireless network or any other type of communication network.

[0231] The systems and methods of the present disclosure each have several innovative aspects, none of which is solely responsible for or required for the desired attributes disclosed herein. The various features and processes described above can be used independently of each other, or can be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. Various modifications to the embodiments described in the present disclosure may be easily clear to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the claims are not intended to be limited to the embodiments shown herein, but should be given the widest scope consistent with the present disclosure, the principles disclosed herein, and the novel features.

[0232] Certain features described in separate implementations in this specification may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments, respectively. In addition, although features may be described above as working in certain combinations and even initially required, one or more features from the required combination may be deleted from the combination in some cases, and the required combination may be for a sub-combination or a variant of the sub-combination. No single feature or feature group is necessary or indispensable for each embodiment.

[0233] Unless otherwise specifically stated, or understood in the context of use, conditional language used herein, such as "may", "can", "might", "will", "for example", etc., is generally intended to express that certain embodiments include and other embodiments do not include certain features, elements / or steps. Therefore, such conditional language is generally not intended to imply that features, elements / or steps are necessary for one or more embodiments in any way, or that one or more embodiments necessarily include the use of determining whether these features, elements / or steps are included or executed in any particular embodiment with or without author input or prompting. The terms "include", "comprise", "have", etc. are synonyms and are used inclusively in an open-ended manner, and do not exclude additional elements, features, actions, operations, etc. In addition, the term "or" is used with its inclusive meaning (rather than its exclusive meaning), so when used, for example, to connect a list of elements, the term "or" represents one, some or all of the elements in the list. In addition, unless otherwise stated, the articles "one", "an", and "said" used in this application and the appended claims should be interpreted as meaning "one or more" or "at least one".

[0234] As used herein, a phrase referring to "at least one" of a list of items refers to any combination of those items, including single members. As an example, "at least one of A, B, or C" is intended to cover: A, B, C, A and B, A and C, B and C, and A, B and C. Unless otherwise specifically stated, linking language such as the phrase "at least one of X, Y, and Z" as used is understood using the context and is generally used to express that an item, term, etc. can be at least one of X, Y, or Z. Therefore, such linking language is generally not intended to imply that certain embodiments require that at least one of X, at least one of Y, and at least one of Z are all present.

[0235] Similarly, although operations can be shown in a particular order in the accompanying drawings, it should be understood that such operations do not need to be performed in the particular order shown or in a sequential order, or that all operations shown need not be performed to achieve the desired result. In addition, the accompanying drawings may schematically depict one or more example processes in the form of a flow chart. However, other operations not shown may be incorporated into the exemplary methods and processes schematically illustrated. For example, one or more additional operations may be performed before, after, at the same time, or between any of the operations shown. In addition, in other embodiments, operations may be rearranged or reordered. In certain cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the above-mentioned embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or encapsulated into multiple software products. In addition, other embodiments are within the scope of the following claims. In some cases, the actions listed in the claims can be performed in different orders and still achieve the desired result.

Claims

1. A display device for displaying a 3D representation of an object, comprising: a display panel configured to be viewed from a reference viewing direction, wherein the display panel is curved outwardly from a plane perpendicular to the reference viewing direction; a plurality of light field sub-displays, which are arranged on the display panel; a non-transitory memory configured to store a light field image to be displayed by the display device; the light field image providing a plurality of different views of the object in different viewing directions; as well as a processor operably coupled to the non-transitory memory and the lightfield sub-display, the processor being programmed with executable instructions to: Selecting the light field image; mapping the lightfield image into each of the plurality of lightfield sub-displays based at least in part on a position of the lightfield sub-display on the display panel; as well as The plurality of lightfield sub-displays are illuminated based at least in part on the mapped lightfield image. 2 . The device of claim 1 , wherein the display panel comprises a first axis and a second axis perpendicular to the reference viewing direction. 3 . The device of claim 2 , wherein the first axis is horizontal and the display panel is bent about the first axis. 4 . The device of claim 2 , wherein the second axis is vertical and the display panel is bent about the second axis. 5 . The device of claim 2 , wherein the first axis and the second axis are perpendicular, and the display panel is bent around the first axis and the second axis. The device of claim 2 , wherein a radius of curvature about the first axis is different from a radius of curvature about the second axis.

7. The device of claim 2, wherein a radius of curvature about the first axis is substantially the same as a radius of curvature about the second axis. 8 . The device of claim 1 , wherein the display panel comprises a plurality of display regions, at least one display region being bent outward from a plane perpendicular to the reference viewing direction.

9. The apparatus of claim 8, wherein at least one of the display regions is substantially perpendicular to the reference viewing direction.

10. The device of claim 1, wherein the display has a shape that is at least one of a cylinder, a sphere, an oblate spheroid, and a prolate spheroid.

11. The apparatus of claim 1 , wherein each light field sub-display comprises: a microlens array comprising a plurality of microlenses; as well as A pixel array comprising a plurality of pixel subsets, each pixel subset being associated with a microlens and configured to generate light, wherein each pixel subset and associated microlens are arranged to generate outgoing light at a plurality of angles, wherein light from a first pixel of the pixel subset propagates from the lightfield sub-display at an angle that is different from an angle of a second pixel of the pixel subset.

12. The device of claim 11, wherein the subset of pixels is located near a focal point of the associated microlens.

13. The apparatus of claim 1 , wherein each lightfield sub-display comprises a stacked waveguide assembly comprising one or more waveguides, wherein each of the one or more waveguides is configured to project light of one or more of the multiple different views of the object. The device of claim 1 , wherein the display panel is convex relative to the reference viewing direction.

15. The apparatus of claim 1 , wherein to illuminate the plurality of light field sub-displays, the processor is programmed to scale an intensity or duration of illumination of a light field sub-display based on a position of the light field sub-display on the display panel relative to a position of another light field sub-display.

16. The apparatus of claim 1, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels that, when combined, render the rendered frame, each rendered pixel having a position within the rendered frame.

17. The apparatus of claim 16, wherein to map the light field image to the plurality of light field sub-displays, the processor is programmed to associate a position of each rendered pixel with a position of each light field sub-display on the display panel.

18. The apparatus of claim 17, wherein the rendered pixel position is constant between the plurality of rendered frames.

19. The apparatus of claim 1, further comprising a speaker system configured to project audio in combination with the processor programmed to illuminate the plurality of lightfield sub-displays.

20. The apparatus of claim 1, further comprising a microphone configured to receive audio, and wherein the processor is programmed with executable instructions to: receiving audio input from the microphone; recognizing that the audio input is an audio command; as well as An action is initiated to modify illumination of the plurality of lightfield sub-displays based on the audio command.

21. The device of claim 1, further comprising a proximity sensor configured to detect the presence or absence of an entity within a predetermined distance of the display device, and wherein the processor is programmed with executable instructions to initiate an action based on the proximity sensor detecting the presence or absence of the entity.

22. The device of claim 1, further comprising a proximity sensor configured to detect the presence or absence of an entity within a predetermined distance of the display device, and wherein the processor is programmed with executable instructions to initiate an action based on the proximity sensor detecting the presence or absence of the entity.

23. A display device for displaying a 3D representation of an image, comprising: a display panel configured to be viewed from a reference viewing direction, wherein the display panel is curved outwardly from a plane perpendicular to the reference viewing direction; one or more light field sub-displays, each of the one or more light field sub-displays having a position on the display panel; a non-transitory memory configured to store a light field image to be displayed by the display device, the light field image providing a plurality of different views of an object in different viewing directions; a processor operably coupled to the non-transitory memory and the lightfield sub-display, the processor being programmed with executable instructions to: selecting the light field image, and The one or more lightfield sub-displays are illuminated based at least in part on the lightfield image and the positions of the one or more lightfield sub-displays on the display panel.

24. The device of claim 23, wherein the display panel is convex relative to the reference viewing direction.

25. The apparatus of claim 23, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels that, when combined, render the rendered frame, each rendered pixel having a position within the rendered frame.

26. The device of claim 23, further comprising a proximity sensor configured to detect the presence or absence of an entity within a predetermined distance of the display device, and wherein the processor is programmed with executable instructions to initiate an action based on the proximity sensor detecting the presence or absence of the entity.

27. A method for displaying a 3D representation of an object on a display device, the method comprising: selecting a light field image to be displayed by the display device, the light field image providing a plurality of different views of the object in different viewing directions; mapping the lightfield image to each of the plurality of lightfield sub-displays based at least in part on a position of the plurality of lightfield sub-displays on a display panel; as well as The plurality of lightfield sub-displays are illuminated based at least in part on the mapped lightfield image.

28. The method of claim 27, wherein the light field image comprises a plurality of rendered frames, each rendered frame indicating a different view of the plurality of different views of the object, wherein each rendered frame comprises a plurality of rendered pixels that are combined to render the rendered frame, each rendered pixel having a position within the rendered frame.

29. The method of claim 28, wherein mapping the light field image to the plurality of light field sub-displays based at least in part on the positions of the light field sub-displays on the display panel comprises: The position of each rendered pixel is associated with the position of each light field sub-display on the display panel.

30. The method of any one of claims 27-29, wherein the rendered pixel position does not change between the plurality of rendered frames.

31. A light field display device, comprising a curved panel, wherein the curved panel comprises a plurality of light field sub-displays. 32 . The light field display device according to claim 31 , wherein the curved panel is curved in a horizontal direction, in a vertical direction, or in both the horizontal direction and the vertical direction.

Images