Digital signage system based on light field display system

Abstract

A light field (LF) display system presents holographic content to one or more viewers in a public environment for digital signage applications. In some embodiments, the LF display system includes a sensory feedback assembly, a tracking system, and / or a viewer profiling module. The sensory feedback assembly may include a sensory feedback device that provides sensory feedback along with the presented holographic content to a viewer of the LF display system. The tracking system may include a camera that tracks the viewer of the LF display system. Based on the viewer's tracked position and / or tracked gaze, the LF display system may generate holographic content that is perceptible to certain viewers but not viewable to other viewers. The viewer profiling module may identify each viewer to provide personalized holographic content, and may further monitor and record the viewer's behavior to inform the LF display system's subsequent presentation of holographic content.

Description

[0001] Inventor:

[0002] Jonathan Sean Karafin,

[0003] Brendan Elwood Bevensey,

[0004] John Dome

[0005] CROSS-REFERENCE TO RELATED APPLICATIONS

[0006] This application is related to International Application Nos. PCT / US2017 / 042275, PCT / US2017 / 042276, PCT / US2017 / 042418, PCT / US2017 / 042452, PCT / US2017 / 042462, PCT / US2017 / 042466, PCT / US2017 / 042467, PCT / US2017 / 042468, PCT / US2017 / 042469, PCT / US2017 / 042470, and PCT / US2017 / 042679, all of which are incorporated herein by reference in their entirety. Background Art

[0007] The present disclosure relates to a signage system, and more particularly, to a signage system based on light field (LF) display.

[0008] Conventional signage systems present visual content to one or more viewers of the signage system. Conventional signage systems can generally be divided into digital signage systems and physical signage systems. Conventional physical signage systems are typically composed of physical objects that can be two-dimensional (2-D) or three-dimensional (3-D), such as paper signs, billboards, digital electronic displays, 3-D block letters, 3-D sculptures, etc. However, many conventional physical signage systems have difficulty presenting dynamic content that changes in real time. For example, a physical signage system cannot present a movie trailer simply by presenting a movie poster. Conventional digital signage systems are typically composed of digital display screens that can only present 2-D visual content to one or more viewers. Generally speaking, conventional signage systems can only present common visual content that is visible from all vantage points within the direct line of sight of the conventional signage system. In other words, all viewers of a conventional signage system see the same visual content at a given point in time, perhaps from different angles. Summary of the Invention

[0009] A light field display system for digital signage applications is configured to present holographic content to one or more viewers in a public environment. The public environment can be any public area where one or more viewers reside. The holographic content provided in digital signage applications can include holographic information panels, holographic representations of physical goods, holographic advertisements, holographic movies or videos, and the like.

[0010] A light field display system includes a controller and a light field display assembly. The controller is configured to generate holographic content. The light field display assembly includes one or more light field display modules configured to present holographic content in a holographic object volume to a viewer at a viewing volume located in a public environment.

[0011] A light field display system has a tracking system comprising one or more cameras for tracking the movement of a viewer within a viewing volume of the light field display system. The light field display system determines that the viewer is within the viewing volume based in part on the tracked movement. In response to determining that the viewer is within the viewing volume, a light field display assembly of the light field display system, via one or more light field display modules, presents holographic content in a holographic object volume that is viewable by the viewer within the viewing volume.

[0012] In some embodiments, a light field display system generates display instructions for a holographic object based in part on a set of parameters. One or more light field display modules present the holographic object within a viewing volume according to the display instructions, wherein the holographic object is a real image. The light field display system may further record the behavior of each viewer, which may inform a subsequent selection of holographic objects to be presented to each viewer. In other embodiments, the light field display system may further identify individual viewers within the viewing volume. The light field display system may further generate display instructions based on the identified viewers. The light field display system may also receive one or more inputs from the viewer. The light field display system may update the display instructions based on the tracking information and / or the received inputs to modify the presentation of the holographic object. The light field display assembly then presents the modified holographic object using the light field display modules according to the updated display instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a diagram of a light field display module for presenting a holographic object according to one or more embodiments.

[0014] Figure 2A is a cross-section of a portion of a light field display module according to one or more embodiments.

[0015] Figure 2B is a cross-section of a portion of a light field display module according to one or more embodiments.

[0016] Figure 3Ais a perspective view of a light field display module according to one or more embodiments.

[0017] Figure 3B is a cross-sectional view of a light field display module according to one or more embodiments.

[0018] Figure 4A is a perspective view of a portion of a lightfield display system tiled in two dimensions to form a single-sided seamless surface environment in accordance with one or more embodiments.

[0019] Figure 4B is a perspective view of a portion of a lightfield display system in a multi-faceted seamless surface environment in accordance with one or more embodiments.

[0020] Figure 4C is a top view of a lightfield display system with a converging surface in a wing-like configuration according to one or more embodiments.

[0021] Figure 4D is a side view of a lightfield display system with a converging surface in a tilted configuration according to one or more embodiments.

[0022] Figure 4E is a top view of a lightfield display system having a converging surface on the front wall of a room according to one or more embodiments.

[0023] Figure 4F is a side view of a lightfield display system having a converging surface on the front wall of a room in accordance with one or more embodiments.

[0024] Figure 5A is a block diagram of a light field display system according to one or more embodiments.

[0025] Figure 5B is a block diagram of a light field signage application environment incorporating a light field display system for signage according to one or more embodiments.

[0026] Figure 6 is an illustration of a lightfield-based display system for use in a large-scale signage system, according to one or more embodiments.

[0027] Figure 7 is an illustration of a light field based display system for use in a small signage system according to one or more embodiments.

[0028] The accompanying drawings depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein. DETAILED DESCRIPTION

[0029] Overview

[0030] A light field display system is implemented as or as a component of a signage system to present holographic content to one or more viewers of the signage system. The holographic content may include one or more holographic objects and may also include any combination of other visual content (e.g., two-dimensional or three-dimensional visual content) or sensory feedback content. Holographic objects include two-dimensional (2-D) objects, three-dimensional (3-D) objects, or both, generated using one or more four-dimensional (4-D) light field functions. The holographic content provided in signage applications may include holographic information panels, holographic representations of physical goods, holographic advertisements, holographic movies or videos, holographic stories, and the like. The light field display system includes a light field display assembly. The light field display system has one or more light field display modules configured to present holographic content to one or more viewers in a viewing volume of the light field display system. The holographic content provided by the light field display system may also be used to convey information to one or more viewers of the signage system.

[0031] In some embodiments, a lightfield display system includes a sensory feedback assembly, a tracking system, a viewer profiling module, or any combination thereof. The sensory feedback assembly includes one or more sensory feedback devices that provide sensory feedback content, along with holographic content presented by the lightfield display assembly, to one or more viewers of the lightfield display system. Sensory feedback may include haptic feedback, audio feedback, wind feedback, and temperature feedback to enhance the holographic content presented to viewers of the lightfield display system. The tracking system includes one or more cameras that track viewers of the lightfield display system. Tracking may involve monitoring the position of one or more viewers within a viewing volume of the lightfield display system or tracking the viewer's gaze (i.e., monitoring the viewer's response). In one embodiment, the lightfield display system may determine the occurrence of an impression based on the monitored viewer's response. For example, viewer gaze may be used to determine the viewer's impression, such as a response to a holographic object. Based on the viewer's position and gaze, the lightfield display system may generate holographic content that is perceivable by one or more viewers within a portion of the viewing volume, but not viewable by others outside that portion. The viewer profile forming module identifies each viewer to provide personalized holographic content to each viewer. The viewer profile forming module can further monitor and record the viewer's behavior on the holographic content to inform the light field display system of subsequent presentation of the holographic content.

[0032] In some embodiments, the light field display system is implemented in a large-scale signage system. The light field display system can be sized to be larger than the average viewer. For example, the light field display system can be implemented as a billboard to present holographic content to one or more viewers passing by the light field display system. The light field display system can be placed in close proximity to a driveway, a sidewalk, or a combination thereof. The light field display system can be placed on one or more sides of a building, one or more sides of a transportation vehicle, an interior wall of an organization or business, a front desk of an organization or business, a directory kiosk of an organization or business, one or more signs in a public place (e.g., billboards, posters, etc.), or any other surface in a public environment that can be used for digital signage. Viewers can include drivers or passengers in vehicles and / or pedestrians. The light field display system can use a tracking system and a viewer profiling module to record a viewer's impression of the holographic content. In some embodiments, the light field display system is implemented in a small-scale signage system. The light field display system can be sized to be similar to the size of any average viewer. The light field display system can present holographic content to one or more viewers. Similarly, the light field display system can use the tracking system and the viewer profile formation module to record the viewer's impression of the holographic content. In addition, the viewer profile formation module can record the viewer's behavior towards the presented holographic content to update each viewer's holographic content preferences.

[0033] Light Field Display System Overview

[0034] Figure 1 100 is a diagram of a light field display module 110 presenting a holographic object 120 according to one or more embodiments. The light field display module 110 is part of a light field display system. The light field display system uses one or more light field display modules to present holographic content including at least one holographic object. The light field display system can present the holographic content to one or more viewers. In some embodiments, the light field display system can also enhance the holographic content with other sensory content (e.g., touch, audio, smell, temperature, etc.). For example, as discussed below, the projection of focused ultrasound can generate an airborne tactile sensation that can simulate the surface of some or all of the holographic object. The light field display system includes one or more light field display modules 110, and the following description of the light field display system is provided below. Figures 4A-4F , 6 and 7 are discussed in detail.

[0035] The light field display module 110 is a holographic display that presents a holographic object (e.g., holographic object 120) to one or more viewers (e.g., viewer 140). The light field display module 110 includes an energy device layer (e.g., an emissive electronic display or an acoustic projection device) and an energy waveguide layer (e.g., an optical lens array). In addition, the light field display module 110 may include an energy relay layer for combining multiple energy sources or detectors together to form a single surface. At a high level, the energy device layer generates energy (e.g., holographic content) and then uses the energy waveguide layer to guide the energy to a region in space according to one or more four-dimensional (4-D) light field functions. The light field display module 110 can also project and / or sense one or more types of energy simultaneously. For example, the light field display module 110 may be able to project a holographic image as well as an ultrasonic tactile surface in a viewing volume while detecting imaging data from the viewing volume. The operation of the light field display module 110 is described below with respect to Figures 2A-3B Discuss in more detail.

[0036] The light field display module 110 uses one or more 4-D light field functions (e.g., derived from a 5-D plenoptic function) to generate a holographic object within a holographic object volume 160. The holographic object can be three-dimensional (3-D), two-dimensional (2-D), or some combination thereof. Furthermore, the holographic object can be multi-colored (e.g., panchromatic). The holographic object can be projected in front of a screen plane, behind a screen plane, or separated by a screen plane. The holographic object 120 can be presented so that it can be perceived anywhere within the holographic object volume 160. The holographic object within the holographic object volume 160 can appear to float in space to the viewer 140.

[0037] The holographic object volume 160 represents the volume in which the viewer 140 can perceive the holographic object. The holographic object volume 160 can extend in front of the surface of the display area 150 (i.e., towards the viewer 140) so that the holographic object can appear in front of the plane of the display area 150. Additionally, the holographic object volume 160 can extend behind the surface of the display area 150 (i.e., away from the viewer 140), allowing the holographic object to appear as if it is behind the plane of the display area 150. In other words, the holographic object volume 160 can include all light rays that originate from the display area 150 (e.g., are projected) and can converge to create the holographic object. Herein, the light rays can converge at a point in front of the display surface, at the display surface, or behind the display surface. More simply, the holographic object volume 160 encompasses all volumes from which the viewer can perceive the holographic object.

[0038] The viewing volume 130 is a volume of space from which a holographic object (e.g., holographic object 120) presented within the holographic object volume 160 by the light field display system is fully visible. The holographic object can be presented within the holographic object volume 160 and viewed within the viewing volume 130 so that the holographic object is indistinguishable from an actual object. The holographic object is formed by projecting the same light rays that would be generated from the surface of the object if it were physically present.

[0039] In some cases, the holographic object volume 160 and the corresponding viewing volume 130 may be relatively small so that they are designed for a single viewer. In other embodiments, as described below with respect to, for example Figures 4A-4F , 6 and 7 discuss in detail that light field display modules can be enlarged and / or tiled to create larger holographic object volumes and corresponding viewing volumes that can accommodate a wide range of viewers (e.g., one to thousands). The light field display modules presented in the present disclosure can be constructed so that the entire surface of the light field display contains the holographic imaging optics, there are no invalid or dead spaces, and no borders are required. In these embodiments, the light field display modules can be tiled so that the imaging area is continuous across the seams between the light field display modules, and the join lines between the tiled modules are almost undetectable using the visual acuity of the eye. It is worth noting that in some configurations, although not described in detail herein, some portions of the display surface may not contain holographic imaging optics.

[0040] The flexible size and / or shape of the viewing volume 130 allows the viewer to be unconstrained within the viewing volume 130. For example, the viewer 140 can move to different locations within the viewing volume 130 and see different views of the holographic object 120 from corresponding viewing angles. Figure 1 , viewer 140 is located in a first position relative to holographic object 120 such that holographic object 120 appears to be a frontal view of a dolphin. Viewer 140 can move to other positions relative to holographic object 120 to see different views of the dolphin. For example, viewer 140 can move so that he / she sees the dolphin's left side, the dolphin's right side, and so on, much as if viewer 140 were viewing an actual dolphin and changing his / her relative position to the actual dolphin to see different views of the dolphin. In some embodiments, holographic object 120 is visible to all viewers within viewing volume 130, and all viewers have an unobstructed view of holographic object 120 (i.e., not blocked by objects / people). These viewers can be unconstrained, allowing them to move around within the viewing volume to see different perspectives of holographic object 120. Thus, a lightfield display system can present a holographic object such that multiple unconstrained viewers can simultaneously see different perspectives of the holographic object in real-world space, as if the holographic object were physically present.

[0041] In contrast, conventional displays (e.g., stereoscopic, virtual reality, augmented reality, or mixed reality) typically require each viewer to wear some kind of external device (e.g., 3-D glasses, a near-eye display, or a head-mounted display) in order to see the content. Additionally and / or alternatively, conventional displays may require the viewer to be constrained to a specific viewing position (e.g., in a chair with a fixed position relative to the display). For example, when viewing an object presented by a stereoscopic display, the viewer always focuses on the display surface, not on the object, and the display will always present only two views of the object, which will follow the viewer as they attempt to move around the perceived object, resulting in a distortion of the object's perception. However, with light field displays, viewers of holographic objects presented by a light field display system do not need to wear external devices or be constrained to a specific position in order to see the holographic object. A light field display system presents holographic objects in a manner visible to the viewer that is much the same as a physical object, without the need for special goggles, glasses, or head-mounted accessories. Furthermore, the viewer can view holographic content from any position within the viewing volume.

[0042] It is worth noting that there is a limitation on the size of the potential position receptors of the holographic object within the holographic object volume 160. In order to increase the size of the holographic object volume 160, the size of the display area 150 of the light field display module 110 can be increased and / or multiple light field display modules can be tiled together in a manner to form a seamless display surface. The effective display area of the seamless display surface is larger than the display area of each light field display module. Figures 4A-4F , 6 and 7 discuss some embodiments related to tiled light field display modules. Figure 1 As shown, the display area 150 is rectangular, resulting in a pyramidal shape for the holographic object volume 160. In other embodiments, the display area may have some other shape (eg, hexagonal), which also affects the shape of the corresponding viewing volume.

[0043] Additionally, while the above discussion focuses on presenting the holographic object 120 within a portion of the holographic object volume 160 located between the lightfield display module 110 and the viewer 140, the lightfield display module 110 can additionally present content in the holographic object volume 160 behind the plane of the display area 150. For example, the lightfield display module 110 can make the display area 150 appear to be the surface of the ocean with the holographic object 120 emerging from it. The displayed content can then enable the viewer 140 to look through the displayed surface to see underwater marine life. Furthermore, the lightfield display system can generate content that seamlessly moves around the holographic object volume 160, both behind and in front of the plane of the display area 150.

[0044] Figure 2AA cross-section 200 of a portion of a light field display module 210 is shown, according to one or more embodiments. The light field display module 210 may be the light field display module 110. In other embodiments, the light field display module 210 may be another light field display module having a display area shaped differently than the display area 150. In the illustrated embodiment, the light field display module 210 includes an energy device layer 220, an energy relay layer 230, and an energy waveguide layer 240. Some embodiments of the light field display module 210 may have different components than those described herein. For example, in some embodiments, the light field display module 210 does not include the energy relay layer 230. Similarly, functionality may be distributed among the components differently than described herein.

[0045] The display system described herein presents an energy emission that replicates the energy that typically surrounds objects in the real world. Here, the emitted energy is directed in a specific direction from each coordinate on the display surface. In other words, each coordinate on the display surface serves as the projection location for the emitted energy. The directed energy from the display surface causes many energy rays to converge, which can thereby create a holographic object. For example, with visible light, a lightfield display projects a large number of light rays from the projection location that can converge at any point in the volume of the holographic object. Therefore, from the perspective of a viewer positioned farther away than the projected object, the light rays appear to originate from the surface of a real object located in this area of space. In this way, the lightfield display generates reflected light rays that reflect away from the surface of the object from the viewer's perspective. The viewing angle can vary for any given holographic object, and the viewer will see different views of the holographic object.

[0046] As described herein, the energy device layer 220 includes one or more electronic displays (e.g., emissive displays such as OLEDs) and one or more other energy projection and / or energy receiving devices. The energy device layer 220 can be configured to provide multiple energy source locations. The one or more electronic displays are configured to display content according to display instructions (e.g., from a controller of the light field display system). The one or more electronic displays include a plurality of pixels, each having an independently controlled intensity. Many types of commercially available displays can be used in light field displays, such as emissive LED and OLED displays.

[0047] The energy device layer 220 may also include one or more acoustic projection devices and / or one or more acoustic receiving devices. The acoustic projection device generates one or more pressure waves that complement the holographic object 250. The generated pressure waves may be, for example, audible, ultrasonic, or a combination thereof. An array of ultrasonic pressure waves may be used for somatotactile sensations (e.g., at the surface of the holographic object 250). Audible pressure waves are used to provide audio content (e.g., immersive audio) that may complement the holographic object 250. For example, assuming that the holographic object 250 is a dolphin, one or more acoustic projection devices may be used to (1) generate a tactile surface juxtaposed with the surface of the dolphin so that the viewer can touch the holographic object 250; and (2) provide audio content corresponding to the sounds made by the dolphin (e.g., clicks, chirps, or squeaks). An acoustic receiving device (e.g., a microphone or microphone array) may be configured to monitor ultrasonic and / or audible pressure waves within a local area of the light field display module 210.

[0048] Energy device layer 220 may also include one or more imaging sensors. Imaging sensors may be sensitive to light in the visible wavelength band and, in some cases, may be sensitive to light in other wavelength bands (e.g., infrared). Imaging sensors may be, for example, complementary metal oxide semiconductor (CMOS) arrays, charge coupled devices (CCDs), photodetector arrays, some other sensor that captures light, or some combination thereof. The light field display system may use data captured by the one or more imaging sensors to locate and track the viewer's position.

[0049] In some configurations, the energy relay layer 230 relays energy (e.g., electromagnetic energy, mechanical pressure waves, etc.) between the energy device layer 220 and the energy waveguide layer 240. The energy relay layer 230 includes one or more energy relay elements 260. Each energy relay element includes a first surface 265 and a second surface 270, and relays energy between the two surfaces. The first surface 265 of each energy relay element can be coupled to one or more energy devices (e.g., an electronic display or an acoustic projection device). The energy relay elements can be constructed from, for example, glass, carbon, optical fiber, optical film, plastic, polymer, or some combination thereof. Additionally, in some embodiments, the energy relay elements can adjust the amplification (increase or decrease) of energy passing between the first surface 265 and the second surface 270. If the relays provide amplification, the relays can take the form of an array of bonded conical repeaters, referred to as cones, where the area of one end of the cone can be substantially larger than the area of the opposite end. The large ends of the cones can be bonded together to form a seamless energy surface 275. One advantage is that space is created on the multiple small ends of each cone to accommodate the mechanical envelope of multiple energy sources, such as the bezels of multiple displays. This additional room allows energy sources to be placed side by side on the small cone sides, with the active area of each energy source directing energy into the small cone surface and relaying it to the large seamless energy surface. Another advantage of using cone-shaped repeaters is that there is no non-imaging dead space on the combined seamless energy surface formed by the large ends of the cones. There are no borders or borders, and therefore the seamless energy surfaces can then be tiled together to form a larger surface with virtually no seams, depending on the visual acuity of the eye.

[0050] The second surfaces of adjacent energy-relay elements converge to form energy surface 275. In some embodiments, the spacing between edges of adjacent energy-relay elements is less than the minimum perceptible outline defined by the visual acuity of a human eye having, for example, 20 / 40 vision, such that energy surface 275 is effectively seamless from the perspective of viewer 280 within viewing volume 285.

[0051] In some embodiments, the second surfaces of adjacent energy relay elements are fused together using a process that may include one or more of pressure, heat, and a chemical reaction, such that no seams exist between them. In still other embodiments, an array of energy relay elements is formed by molding one side of a continuous block of relay material into an array of small tapered ends, each energy relay element being configured to transfer energy from an energy device attached to the small tapered end to a larger, undivided, single, combined surface.

[0052] In some embodiments, one or more of the energy relay elements exhibit energy localization, wherein the efficiency of energy transfer in a longitudinal direction substantially perpendicular to surfaces 265 and 270 is much higher than the efficiency of transfer in a perpendicular transverse plane, and wherein the energy density is highly localized in this transverse plane as the energy wave propagates between surfaces 265 and 270. This localization of energy enables efficient relaying of energy distributions (e.g., images) between these surfaces without any significant loss of resolution.

[0053] The energy waveguide layer 240 includes a plurality of energy waveguides, each of which is configured to project energy from at least one energy source location from the display surface into at least one specific direction dependent on the energy source location according to an energy 4-D light field function to form a holographic object. The energy propagation direction (or path) is defined by at least two angular dimensions determined by the waveguide's energy surface coordinate position relative to the waveguide. The waveguide is associated with a spatial 2-D coordinate. These four coordinates together form a 4-D energy field. As an example, for electromagnetic energy, the waveguide elements in the energy waveguide layer 240 guide light from locations on the seamless energy surface 275 through the viewing volume 285 along different propagation directions. In various examples, the light is guided according to the 4-D light field function to form a holographic object 250 within the holographic object volume 255. In some embodiments, the viewing angle of the holographic object 250 can vary based in part on the viewer's position within the viewing volume 285 relative to the holographic object 250 within the holographic object volume 255.

[0054] Each waveguide element in the energy waveguide layer 240 can be, for example, a lenslet comprised of one or more elements. In some configurations, the lenslet can be a positive lens. The positive lens can have a spherical, aspherical, or free-form surface profile. Furthermore, in some embodiments, some or all of the waveguide elements can include one or more additional optical components. The additional optical components can be, for example, energy-reducing structures such as baffles, positive lenses, negative lenses, spherical lenses, aspherical lenses, free-form lenses, liquid crystal lenses, liquid lenses, refractive elements, diffractive elements, or some combination thereof. In some embodiments, at least one of the lenslets and / or additional optical components can dynamically adjust its optical power. For example, the lenslet can be a liquid crystal lens or a liquid lens. Dynamic adjustment of the surface profile of the lenslet and / or at least one additional optical component can provide additional directional control of light projected from the waveguide element.

[0055] In the example shown, the holographic object volume 255 of the lightfield display has boundaries formed by light rays 256 and 257, but other light rays may be used. The holographic object volume 255 is a continuous volume that extends both in front of (i.e., toward) and behind (i.e., away from) the energy waveguide layer 240. In the example shown, the light rays 256 and 257 that are perceptible to the user are projected from opposite edges of the lightfield display module 210 at the maximum angle relative to the normal of the display surface, but other projected light rays may be used. The light rays define the display's field of view and, therefore, the boundaries of the holographic viewing volume 285. In some cases, the light rays define a holographic viewing volume within which the entire display can be viewed without vignetting (e.g., an ideal viewing volume). As the display's field of view increases, the convergence point of light rays 256 and 257 moves closer to the display. Consequently, displays with larger fields of view allow the viewer 280 to see the entire display at a closer viewing distance. Alternatively, the rays 256 and 257 may form an ideal holographic object volume. A holographic object presented as an ideal holographic object volume may be viewed anywhere in the viewing volume 285.

[0056] In some instances, a holographic object can be presented to only a portion of the viewing volume 285. In other words, the holographic object volume can be divided into any number of viewing sub-volumes (e.g., viewing sub-volume 290). In addition, a holographic object can be projected outside the holographic object volume 255. For example, the holographic object 251 is presented outside the holographic object volume 255. Since the holographic object 251 is presented outside the holographic object volume 255, it cannot be viewed from every position in the viewing volume 285. For example, the holographic object 251 is visible from a position in the viewing sub-volume 290, but not from the position of the viewer 280.

[0057] For example, go to Figure 2B To illustrate viewing the holographic content from different viewing sub-bodies. Figure 2B A cross-section 200 of a portion of a light field display module is shown in accordance with one or more embodiments. Figure 2B The cross section and Figure 2A The cross section is the same. However, Figure 2BA set of different light rays projected from the light field display module 210 is shown. Light rays 256 and 257 still form the holographic object volume 255 and the viewing volume 285. However, as shown, the light rays projected from the top of the light field display module 210 and the light rays projected from the bottom of the light field display module 210 overlap to form various viewing sub-volumes (e.g., viewing sub-volumes 290A, 290B, 290C, and 290D) within the viewing volume 285. A viewer in the first viewing sub-volume (e.g., 290A) may be able to perceive the holographic content presented in the holographic object volume 255, while viewers in the other viewing sub-volumes (e.g., 290B, 290C, and 290D) may not be able to perceive the holographic content presented in the holographic object volume 255.

[0058] More simply, Figure 2A As shown, holographic object volume 255 is a volume in which holographic objects can be presented by a lightfield display system so that the holographic objects can be perceived by a viewer (e.g., viewer 280) in viewing volume 285. In this manner, viewing volume 285 is an example of an ideal viewing volume, and holographic object volume 255 is an example of an ideal object volume. However, in various configurations, a viewer can perceive holographic objects presented by a lightfield display system in other example holographic object volumes. More generally, when viewing holographic content projected from a lightfield display module, "sight guidelines" apply. The sight guidelines assert that the line formed by the viewer's eye position and the holographic object being viewed must intersect the lightfield display surface.

[0059] Because the holographic content is presented according to the 4-D light field function, when viewing the holographic content presented by the light field display module 210, each eye of the viewer 280 sees a different perspective of the holographic object 250. Furthermore, as the viewer 280 moves within the viewing volume 285, he / she will also see different perspectives of the holographic object 250, just like other viewers within the viewing volume 285. As one of ordinary skill in the art will appreciate, 4-D light field functions are well known in the art and will not be described in further detail herein.

[0060] As described in more detail herein, in some embodiments, a lightfield display can project more than one type of energy. For example, a lightfield display can project two types of energy, such as mechanical energy and electromagnetic energy. In this configuration, the energy relay layer 230 can include two separate energy relays that are intertwined at the energy surface 275 but separated so that energy is relayed to two different energy device layers 220. Here, one relay can be configured to transmit electromagnetic energy, while the other relay can be configured to transmit mechanical energy. In some embodiments, mechanical energy can be projected from locations between electromagnetic waveguide elements on the energy waveguide layer 240, thereby helping to form a structure that inhibits light from being transmitted from one electromagnetic waveguide element to another. In some embodiments, the energy waveguide layer 240 can also include waveguide elements that transmit focused ultrasound along specific propagation paths based on displayed instructions from the controller.

[0061] It should be noted that in an alternative embodiment (not shown), the light field display module 210 does not include the energy relay layer 230. In this case, the energy surface 275 is an emitting surface formed using one or more adjacent electronic displays within the energy device layer 220. And in some embodiments, without the energy relay layer, the spacing between the edges of the adjacent electronic displays is less than the minimum perceptible outline defined by the visual acuity of a human eye with 20 / 40 vision, so that the energy surface is effectively seamless from the perspective of the viewer 280 within the viewing volume 285.

[0062] Light field display module

[0063] Figure 3A FIG3 is a perspective view of a light field display module 300A according to one or more embodiments. The light field display module 300A may be the light field display module 110 and / or the light field display module 210. In other embodiments, the light field display module 300A may be some other light field display module. In the illustrated embodiment, the light field display module 300A includes an energy device layer 310, an energy relay layer 320, and an energy waveguide layer 330. The light field display module 300A is configured to present holographic content from a display surface 365, as described herein. For convenience, the display surface 365 is shown in dashed outline on the frame 390 of the light field display module 300A, but is more accurately the surface directly in front of the waveguide element defined by the inner edge of the frame 390. The display surface 365 includes a plurality of projection locations from which energy can be projected. Some embodiments of the light field display module 300A have components different from those described herein. For example, in some embodiments, the light field display module 300A does not include the energy relay layer 320. Similarly, functionality may be distributed among components in ways other than described here.

[0064] Energy device layer 310 is an embodiment of energy device layer 220. Energy device layer 310 includes four energy devices 340 (three of which are visible in the figure). Energy devices 340 can all be of the same type (e.g., all electronic displays) or can include one or more different types (e.g., including an electronic display and at least one acoustic energy device).

[0065] Energy relay layer 320 is an embodiment of energy relay layer 230. Energy relay layer 320 includes four energy relay devices 350 (three of which are visible in the figure). Energy relay devices 350 can all relay the same type of energy (e.g., light) or can relay one or more different types (e.g., light and sound). Each relay device 350 includes a first surface and a second surface, with the second surface of each energy relay device 350 arranged to form a single seamless energy surface 360. In the illustrated embodiment, each energy relay device 350 is tapered, with the first surface having a smaller surface area than the second surface. This allows the mechanical envelope of energy device 340 to be accommodated at the small end of the cone. Because energy can be projected over the entire area, this also makes the seamless energy surface borderless. This means that this seamless energy surface can be tiled by placing multiple instances of light field display module 300A together without dead space or borders, making the entire combined surface seamless. In other embodiments, the first and second surfaces have the same surface area.

[0066] Energy waveguide layer 330 is an embodiment of energy waveguide layer 240. Energy waveguide layer 330 includes a plurality of waveguide elements 370. As discussed above with respect to FIG. 2 , energy waveguide layer 330 is configured to guide energy from seamless energy surface 360 along a specific propagation path according to a 4-D light field function to form a holographic object. Note that in the illustrated embodiment, energy waveguide layer 330 is bounded by frame 390. In other embodiments, frame 390 is absent and / or its thickness is reduced. Removing or reducing the thickness of frame 390 can facilitate tiling lightfield display module 300A with additional lightfield display modules.

[0067] It should be noted that in the embodiment shown, the seamless energy surface 360 and the energy waveguide layer 330 are flat. In alternative embodiments not shown, the seamless energy surface 360 and the energy waveguide layer 330 can be curved in one or more dimensions.

[0068] The light field display module 300A can be configured with additional energy sources present on the surface of the seamless energy surface 360 or in the energy waveguide layer 330, allowing for the projection of energy fields in addition to light fields. In one embodiment, an acoustic energy field can be projected from electrostatic speakers (not shown) mounted at any number of locations (e.g., as an array) on the seamless energy surface 360. In one embodiment, the electrostatic speaker array is coupled to a plurality of energy waveguides. The electrostatic speaker array includes: at least one transparent membrane configured to generate acoustic energy when driven; and a plurality of electrodes configured to acoustically drive the transparent membrane. Each electrode can be located between one or more energy waveguides. In addition, the electrostatic speakers of the light field display module 300A are positioned within the light field display module 300A, allowing the dual energy surface to simultaneously project an acoustic field and holographic content. For example, the electrostatic speakers can be formed with one or more diaphragm elements that transmit electromagnetic energy of a certain wavelength and driven by one or more conductive elements (e.g., a plane sandwiching the one or more diaphragm elements). The electrostatic speaker can be mounted on the seamless energy surface 360 so that the diaphragm element covers some of the waveguide elements. The conductive electrodes of the speaker can be juxtaposed with a structure designed to inhibit light transmission between the electromagnetic waveguides and / or located between the electromagnetic waveguide elements (e.g., frame 390). In various configurations, the speaker can project audible sound and / or generate a variety of sources of focused ultrasonic energy for the tactile surface.

[0069] In some configurations, the energy device 340 can sense energy. For example, the energy device can be a microphone, a light sensor, an acoustic transducer, etc. Therefore, the energy relay device can also relay energy from the seamless energy surface 360 to the energy device layer 310. That is, when the energy device and the energy relay device 340 are configured to simultaneously emit and sense energy (e.g., emit a light field and sense sound), the seamless energy surface 360 of the light field display module forms a bidirectional energy surface.

[0070] More generally, the energy device 340 of the light field display module can be an energy source or an energy sensor. The light field display module 300A can include various types of energy devices that act as energy sources and / or energy sensors to facilitate the projection of high-quality holographic content to the user. Other sources and / or sensors can include thermal sensors or sources, infrared sensors or sources, image sensors or sources, mechanical energy transducers that generate acoustic energy, feedback sources, and the like. Multiple other sensors or sources are possible. Furthermore, the light field display module can be tiled so that the light field display module can form an assembly that projects and senses multiple types of energy from a large aggregate seamless energy surface.

[0071] In various embodiments of the light field display module 300A, the seamless energy surface 360 can have various surface portions, each configured to project and / or emit a specific type of energy. For example, when the seamless energy surface is a dual-energy surface, the seamless energy surface 360 includes one or more surface portions that project electromagnetic energy and one or more other surface portions that project ultrasonic energy. The surface portions that project ultrasonic energy can be located on the seamless energy surface 360 between electromagnetic waveguide elements and / or juxtaposed with structures designed to inhibit light transmission between electromagnetic waveguide elements. In instances where the seamless energy surface is a dual-energy surface, the energy relay layer 320 can include two types of energy relay devices interwoven at the seamless energy surface 360. In various embodiments, the seamless energy surface 360 can be configured such that the portion of the surface beneath any particular waveguide element 370 is all energy sources, all energy sensors, or a mixture of energy sources and energy sensors.

[0072] Figure 3B FIG2 is a cross-sectional view of a light field display module 300B including interwoven energy relay devices, according to one or more embodiments. Energy relay device 350A transfers energy between energy relay first surface 345A connected to energy device 340A and seamless energy surface 360. Energy relay device 350B transfers energy between energy relay first surface 345B connected to energy device 340B and seamless energy surface 360. The two relay devices are interwoven at interwoven energy relay device 352 connected to seamless energy surface 360. In this configuration, seamless energy surface 360 contains interwoven energy locations for both energy devices 340A and 340B, which can be energy sources or energy sensors. Thus, light field display module 300B can be configured as a dual energy projection device for projecting more than one type of energy, or as a bidirectional energy device for simultaneously projecting one type of energy and sensing another type of energy. Light field display module 300B can be light field display module 110 and / or light field display module 210. In other embodiments, the light field display module 300B may be some other light field display module.

[0073] The light field display module 300B includes Figure 3A For example, in the embodiment shown, the light field display module 300B includes an energy device layer 310, an energy relay layer 320, a seamless energy surface 360, and an energy waveguide layer 330, including at least the same components as those of the light field display module 300A. Figure 3A Additionally, the light field display module 300B can present and / or receive energy from the display surface 365. Figure 3AThe components of the light field display module 300B may be connected and / or oriented differently than the components of the light field display module 300A in FIG. Some embodiments of the light field display module 300B may have components that differ from those described herein. Similarly, functionality may be allocated between components in a manner different from that described herein. Figure 3B A design of a single light field display module 300B is shown that can be tiled to create a dual energy projection surface or a bidirectional energy surface with a larger area.

[0074] In one embodiment, the light field display module 300B is a light field display module of a bidirectional light field display system. The bidirectional light field display system can simultaneously project energy from the display surface 365 and sense energy. The seamless energy surface 360 contains both energy projection locations and energy sensing locations that are closely interwoven on the seamless energy surface 360. Figure 3B In the example of Figure 3A For convenience, the energy relay layer of the light field display module 300B will be referred to herein as an "interwoven energy relay layer."

[0075] The interwoven energy relay layer 320 includes two legs: a first energy relay device 350A and a second energy relay device 350B. Figure 3B In the example shown, each of the legs is shown as a light shaded area. Each of the legs can be made of a flexible relay material and formed to a sufficient length for use with energy devices of various sizes and shapes. In some areas of the interwoven energy relay layer, two legs are tightly interwoven when approaching the seamless energy surface 360. In the example shown, the interwoven energy relay device 352 is shown as a dark shaded area.

[0076] When interwoven at seamless energy surface 360, the energy relay devices are configured to relay energy to / from different energy devices. The energy devices are located at energy device layer 310. As shown, energy device 340A is connected to energy relay device 350A, and energy device 340B is connected to energy relay device 350B. In various embodiments, each energy device can be an energy source or an energy sensor.

[0077] The energy waveguide layer 330 includes waveguide elements 370 to guide energy waves from the seamless energy surface 360 along a projected path toward a series of convergence points. In this example, a holographic object 380 is formed at the series of convergence points. Notably, as shown, the convergence of energy at the holographic object 380 occurs on the viewer side (i.e., the front side) of the display surface 365. However, in other examples, the convergence of energy can be anywhere in the holographic object volume, extending both in front of the display surface 365 and behind the display surface 365. The waveguide elements 370 can also guide the incoming energy to an energy device (e.g., an energy sensor), as described below.

[0078] In one example embodiment of the lightfield display module 300B, an emissive display serves as an energy source (e.g., energy device 340A), and an imaging sensor serves as an energy sensor (e.g., energy device 340B). In this manner, the lightfield display module 300B can simultaneously project holographic content and detect light from a volume in front of the display surface 365. In this manner, this embodiment of the lightfield display module 300B functions as both a lightfield display and a lightfield sensor.

[0079] In one embodiment, the light field display module 300B is configured to simultaneously project a light field from a projection location on the display surface to the front of the display surface and capture the light field from the front of the display surface at the projection location. In this embodiment, the energy relay device 350A connects a first set of locations on the seamless energy surface 360 that are positioned below the waveguide element 370 to the energy device 340A. In one example, the energy device 340A is an emissive display having an array of source pixels. The energy relay device 340B connects a second set of locations on the seamless energy surface 360 that are positioned below the waveguide element 370 to the energy device 340B. In one example, the energy device 340B is an imaging sensor having an array of sensor pixels. The light field display module 300B can be configured so that the locations on the seamless energy surface 365 that are positioned below a particular waveguide element 370 are all emissive display locations, all imaging sensor locations, or some combination of these locations. In other embodiments, the bidirectional energy surface can project and receive various other forms of energy.

[0080] In another example embodiment of the light field display module 300B, the light field display module is configured to project two different types of energy. For example, in one embodiment, the energy device 340A is an emissive display configured to emit electromagnetic energy, and the energy device 340B is an ultrasonic transducer configured to emit mechanical energy. Thus, both light and sound can be projected from various locations on the seamless energy surface 360. In this configuration, an energy relay device 350A connects the energy device 340A to the seamless energy surface 360 and relays electromagnetic energy. The energy relay device is configured to have properties that enable efficient transmission of electromagnetic energy (e.g., a varying refractive index). The energy relay device 350B connects the energy device 340B to the seamless energy surface 360 and relays mechanical energy. The energy relay device 350B is configured to have properties that facilitate efficient transmission of ultrasonic energy (e.g., a distribution of materials with different acoustic impedances). In some embodiments, mechanical energy can be projected from locations between waveguide elements 370 on the energy waveguide layer 330. The locations where the mechanical energy is projected can form structures that inhibit the transmission of light from one electromagnetic waveguide element to another. In one example, an array of spatially separated locations where ultrasonic mechanical energy is projected can be configured to create three-dimensional tactile shapes and surfaces in mid-air. The surface can coincide with a projected holographic object (e.g., holographic object 380). In some examples, phase delay and amplitude variations across the array can help form the tactile shape.

[0081] In various embodiments, the light field display module 300B with interwoven energy relay devices can include multiple energy device layers, where each energy device layer includes a specific type of energy device. In these examples, the energy relay layer is configured to relay the appropriate type of energy between the seamless energy surface 360 and the energy device layer 310.

[0082] Tiled light field display module

[0083] Figure 4A FIG4 is a perspective view of a portion of a light field display system 400 tiled in two dimensions to form a single-sided seamless surface environment, according to one or more embodiments. Light field display system 400 includes a plurality of light field display modules tiled to form an array 410. More specifically, each of the small squares in array 410 represents a tiled light field display module 412. Light field display module 412 can be the same as light field display module 300A or 300B. Array 410 can cover some or all of a surface (e.g., a wall) of a room, for example. The light field array can also cover other surfaces, such as tabletops, billboards, circular buildings, panels, and the like.

[0084] Array 410 can project one or more holographic objects. For example, in the embodiment shown, array 410 projects holographic object 420 and holographic object 422. Tiling of light field display modules 412 allows for a larger viewing volume and allows objects to be projected at a greater distance from array 410. For example, in the embodiment shown, the viewing volume is approximately the entire area in front of and behind array 410, rather than a localized volume in front of (and behind) light field display module 412.

[0085] In some embodiments, lightfield display system 400 presents holographic object 420 to viewer 430 and viewer 434. Viewer 430 and viewer 434 receive different perspectives of holographic object 420. For example, viewer 430 is presented with a direct view of holographic object 420, while viewer 434 is presented with a more oblique view of holographic object 420. As viewer 430 and / or viewer 434 move, they are presented with different perspectives of holographic object 420. This allows viewers to visually interact with the holographic object by moving relative to the holographic object. For example, as viewer 430 walks around holographic object 420, viewer 430 sees different sides of holographic object 420, as long as holographic object 420 remains within the holographic object volume of array 410. Consequently, viewer 430 and viewer 434 can simultaneously see holographic object 420 in real-world space, as if the holographic object were actually present. In addition, viewers 430 and 434 do not need to wear external devices to view holographic object 420, as holographic object 420 is visible to the viewers in much the same way a physical object would be visible. Furthermore, here, holographic object 422 is displayed behind the array, as the viewing volume of the array extends behind the surface of the array. In this way, holographic object 422 can be presented to viewers 430 and / or 434.

[0086] In some embodiments, the light field display system 400 presents holographic content to a viewer 430 (i.e., a first viewer) and a viewer 434 (i.e., a second viewer). The first viewer and the second viewer may be located in different viewing volumes. For example, the first viewer may be located in the viewing volume, and the second viewer may be located in the second viewing volume. The light field display system 400 presents additional holographic content from the holographic object volume of the array 410 to the second viewer located in a second viewing volume different from the viewing volume. In some embodiments, the light field display system 400 presents holographic content that is viewable from the viewing volume but not from the second viewing volume.

[0087] In some embodiments, light field display system 400 may include a tracking system that tracks the location of viewer 430 and viewer 434. In some embodiments, the tracked location is the location of the viewer. In other embodiments, the tracked location is the location of the viewer's eyes. Eye location tracking is different from gaze tracking, which tracks where the eyes are looking (e.g., using orientation to determine gaze location). The eyes of viewer 430 and the eyes of viewer 434 are located in different locations.

[0088] In various configurations, the light field display system 400 may include one or more tracking systems for collecting information about one or more viewers of the light field display system, including the viewer's response to the holographic content projected by the light field display system 400 and characteristics of the viewer of the light field display system 400. For example, in the illustrated Figure 4A In an embodiment of the present invention, the light field display system includes a tracking system 440 external to the array 410. Here, the tracking system can be a camera system coupled to the array 410. Figure 5A 4. A tracking system is described in more detail in

[0065] . In other example embodiments, a tracking system may be incorporated into array 410 as described herein. For example, an energy device (e.g., energy device 340) of one or more lightfield display modules 412 containing a bidirectional energy surface included in array 410 may be configured to capture an image of a viewer in front of array 410. In any case, one or more tracking systems of lightfield display system 400 determine tracking information about a viewer (e.g., viewer 430 and / or viewer 434) viewing holographic content presented by array 410. In one embodiment, the tracking system of lightfield display system 400 tracks the movement of the viewer, and one or more viewing volumes may be defined by the tracked movement.

[0089] In some embodiments, the light field display system 400 can identify an opportunity to present holographic content to a viewer of the light field display system 400 based on determining that the viewer 430 is within the viewing volume. The tracking system 440 can track the movement of the viewer 430 within the viewing volume of the light field display system 400 and can determine that the viewer 430 is within the viewing volume based on the tracked movement. In other embodiments, the light field display system 400 can select holographic content from a content store (i.e., a location where holographic content is stored on a third-party system or online storage), and the holographic content can include a set of parameters that define when and how the holographic content should be presented. The set of parameters can be derived in part from the responses or characteristics of one or more viewers determined by the tracking system. For example, if the age of one or more viewers is determined, content appropriate for a specific age range can be presented. In another example, the frequency with which holographic content is presented can be adjusted depending on the position or movement of the tracked viewer. In general, the tracking system can provide parameters that describe when or how the holographic content should be presented, or some combination thereof. The light field display system 400 may generate display instructions for holographic content based on a set of parameters and may present the holographic content to one or more viewers based on the set of parameters. The display instructions instruct the light field display system 400 when and how to display the holographic content.

[0090] The tracking information describes the position of a viewer or a portion of a viewer (e.g., one or both eyes of the viewer, or a limb of the viewer) in space (e.g., relative to the tracking system). The tracking system may use any number of depth determination techniques to determine the tracking information. Depth determination techniques may include, for example, structured light, time-of-flight, stereoscopic imaging, some other depth determination technique, or some combination thereof. The tracking system may include various systems configured to determine the tracking information. For example, the tracking system may include one or more infrared sources (e.g., a structured light source), one or more imaging sensors (e.g., a red-blue-green-infrared camera) that can capture infrared images, and a processor that executes a tracking algorithm. The tracking system may use depth estimation techniques to determine the position of the viewer. In some embodiments, the light field display system 400 generates holographic objects based on the tracked position, motion, or gesture of the viewer 430 and / or viewer 434 as described herein. For example, the light field display system 400 may generate a holographic object in response to the viewer entering within a threshold distance and / or a specific position of the array 410.

[0091] The light field display system 400 can present one or more holographic objects customized for each viewer based in part on the tracking information. For example, the holographic object 420 can be presented to the viewer 430 instead of the holographic object 422. Similarly, the holographic object 422 can be presented to the viewer 434 instead of the holographic object 420. For example, the light field display system 400 tracks the position of each of the viewers 430 and 434. The light field display system 400 determines the viewing angle of the holographic object that should be visible to the viewer based on the position of the viewer relative to where the holographic object is to be presented. The light field display system 400 selectively projects light from specific pixels corresponding to the determined viewing angle. Therefore, the viewer 434 and the viewer 430 can have potentially completely different experiences at the same time. In other words, the light field display system 400 can present holographic content to a viewing sub-volume of the viewing volume (i.e., similar to Figure 2B 4. For example, as shown, because light field display system 400 can track the position of viewer 430, light field display system 400 can present spatial content (e.g., holographic object 420) to viewing sub-volumes surrounding viewer 430, and present safari content (e.g., holographic object 422) to viewing sub-volumes surrounding viewer 434. In contrast, conventional systems would require the use of separate headphones to provide a similar experience.

[0092] In some embodiments, the light field display system 400 may include one or more sensory feedback systems. The sensory feedback system provides other sensory stimuli (e.g., tactile, audio, or scent) that enhance the holographic objects 420 and 422. For example, Figure 4A In the illustrated embodiment of the light field display system 400, the sensory feedback system 442 includes a sensory feedback system 442 that is external to the array 410. In one example, the sensory feedback system 442 can be an electrostatic speaker coupled to the array 410. Figure 5A The external sensory feedback system is described in more detail. In other example embodiments, the sensory feedback system can be incorporated into the array 410 as described herein. For example, the energy devices (e.g., Figure 3B The energy device 340A in the array 410 may be configured to project ultrasound energy to a viewer in front of the array and / or receive imaging information from a viewer in front of the array. In any case, the sensory feedback system presents sensory content to and / or receives sensory content from a viewer (e.g., viewer 430 and / or viewer 434) viewing holographic content (e.g., holographic object 420 and / or holographic object 422) presented by the array 410.

[0093] The light field display system 400 may include a sensory feedback system 442 comprising one or more acoustic projection devices external to the array. Alternatively or additionally, the light field display system 400 may include one or more acoustic projection devices integrated into the array 410, as described herein. The acoustic projection device may be comprised of an array of ultrasound sources (e.g., an ultrasound energy projection device or a body tactile projection device) configured to project a body tactile surface. In one embodiment, the ultrasound energy projection device may provide tactile feedback for a holographic object. The ultrasound energy projection device may generate a body tactile surface that is proximate to or coincides with the surface of the holographic object. In some embodiments, for one or more surfaces of the holographic object, if a portion of the viewer is within a threshold distance of the one or more surfaces, the tactile surface may coincide with the holographic object (e.g., at the surface of the holographic object 420). The holographic content may be a representation of a physical item having a texture, and the body tactile projection device may simulate the texture of the physical item (i.e., provide a body tactile surface). The body tactile sensation may allow a user to touch and feel the surface of the holographic object. The plurality of acoustic projection devices may also project audible pressure waves that provide audio content (eg, immersive audio) to viewers. Thus, ultrasonic pressure waves and / or audible pressure waves may serve to supplement the holographic object.

[0094] In various embodiments, the light field display system 400 can provide other sensory stimulation based in part on the tracked position of the viewer. For example, Figure 4A The holographic object 422 shown in the figure is a lion, and the light field display system 400 can make the holographic object 422 roar both visually (i.e., the holographic object 422 appears to be roaring) and audibly (i.e., one or more acoustic projection devices project pressure waves), so that the viewer 430 perceives it as the roar of a lion emitted by the holographic object 422.

[0095] It should be noted that in the configuration shown, the holographic viewing volume can be similar to Figures 2A-2B The viewing volume 285 of the light field display system 200 in FIG. 2 is limited. This can limit the perceived immersion that a viewer would experience with a single wall display unit. One way to address this problem is to use multiple light field display modules tiled along multiple sides, as described below with respect to FIG. Figures 4B to 4F described.

[0096] Figure 4Bis a perspective view of a portion of a lightfield display system 402 in a multi-sided seamless surface environment in accordance with one or more embodiments. Each of the one or more lightfield display modules has a display surface from which a holographic object is projected. The lightfield display system 402 is substantially similar to the lightfield display system 400, except that the seamless display surface is formed by tiling the display surfaces of the one or more lightfield display modules across a plurality of walls, floors, and ceilings to create the multi-sided seamless surface environment. More specifically, the lightfield display modules are tiled to form an array that is a six-sided aggregate seamless surface environment. In some embodiments, the seamless display surface may be larger than the surface area of the display surface of a single lightfield display module. Figure 4B In one embodiment, the multiple light field display modules cover all walls, ceiling, and floor of the room. In other embodiments, the multiple light field display modules may cover some, but not all, of the walls, floor, ceiling, or some combination thereof. In other embodiments, the multiple light field display modules are tiled to form some other cohesive seamless surface. For example, the walls may be curved to form a cylindrical cohesive energy environment.

[0097] Lightfield display system 402 can project one or more holographic objects. For example, in the illustrated embodiment, lightfield display system 402 projects holographic object 420 into an area surrounded by a six-sided polymeric seamless surface environment. In this example, the viewing volume of the lightfield display system is also contained within the six-sided polymeric seamless surface environment. Note that in the illustrated configuration, viewer 434 can be positioned between holographic object 420 and lightfield display module 414, which projects the energy (e.g., light and / or pressure waves) used to form holographic object 420. Therefore, the positioning of viewer 434 can prevent viewer 430 from perceiving holographic object 420 formed by the energy from lightfield display module 414. However, in the illustrated configuration, there is at least one other lightfield display module, such as lightfield display module 416, which is unobstructed (e.g., by viewer 434) and can project energy to form holographic object 420 and be observed by viewer 430. In this way, being occluded by the viewer in space may cause some portions of the holographic projection to disappear, but this effect is much smaller than if only one side of the volume were filled with a holographic display panel. Holographic object 422 is displayed as "outside" of the walls of the six-sided, aggregated, seamless surface environment because the holographic object volume extends behind the aggregated surface. Thus, viewer 430 and / or viewer 434 can perceive holographic object 422 as "outside" of the enclosed, six-sided environment in which they can move.

[0098] As mentioned above Figure 4AAs described, in some embodiments, the light field display system 402 actively tracks the viewer's position and can dynamically instruct different light field display modules to present holographic content based on the tracked position. Thus, the multi-faceted configuration can provide a more robust environment (e.g., relative to the Figure 4A ) to provide holographic objects where an unconstrained viewer can move freely throughout the area surrounded by a multi-faceted seamless surface environment.

[0099] It is worth noting that various light field display systems can have different configurations. Furthermore, each configuration can have a specific orientation of surfaces that aggregate to form a seamless display surface ("aggregate surface"). That is, the light field display modules of a light field display system can be tiled to form various aggregate surfaces. For example, in Figure 4B In FIG, light field display system 402 includes light field display modules that are tiled to form a six-sided converging surface that approximates the wall of a room. In some other examples, the converging surface may appear on only a portion of the surface (e.g., half of the wall) rather than the entire surface (e.g., the entire wall). Some examples are described herein.

[0100] In some configurations, the converging surface of a lightfield display system can include a converging surface configured to project energy toward a local viewing volume. Projecting energy toward a local viewing volume allows for a higher quality viewing experience by, for example, increasing the density of projected energy within a particular viewing volume, increasing the field of view (FOV) of the viewer within the viewing volume, and bringing the viewing volume closer to the display surface.

[0101] For example, Figure 4CA top view of a lightfield display system 450A having a converging surface in a "wing-like" configuration is shown. In this example, the lightfield display system 450A is positioned in a room having a front wall 452, a back wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown). The first side wall 456, the second side wall 458, the back wall 454, the floor, and the ceiling are all orthogonal. The lightfield display system 450A includes lightfield display modules that are tiled to form a converging surface 460 that covers the front wall. The front wall 452, and therefore the converging surface 460, comprises three sections: (i) a first section 462 that is substantially parallel to the rear wall 454 (i.e., the center surface), (ii) a second section 464 that connects the first section 462 to the first side wall 456 and is positioned at an angle to project energy toward the center of the room (i.e., the first side surface), and (iii) a third section 466 that connects the first section 462 to the second side wall 458 and is positioned at an angle to project energy toward the center of the room (i.e., the second side surface). The first section is a vertical plane in the room and has a horizontal axis and a vertical axis. The second and third sections are angled along the horizontal axis toward the center of the room.

[0102] In this example, viewing volume 468A of lightfield display system 450A is located in the center of the room and is partially surrounded by three portions of converging surface 460. The converging surface that at least partially surrounds the viewer ("surrounding surface") increases the viewer's immersive experience.

[0103] For illustration, consider, for example, a polymeric surface having only a central surface. Figure 2A As described above, light rays projected from either end of the display surface create an ideal holographic volume and an ideal viewing volume. Now consider, for example, if the central surface includes two side surfaces angled toward the viewer. In this case, light rays 256 and 257 would be projected at a greater angle from the normal to the central surface. Consequently, the viewing volume's field of view would increase. Similarly, the holographic viewing volume would be closer to the display surface. Furthermore, because the two second and third portions are tilted closer to the viewing volume, the holographic object projected at a fixed distance from the display surface is closer to the viewing volume.

[0104] For simplicity, a display surface with only a central surface has a planar field of view, a planar threshold separation between the (central) display surface and the viewing volume, and a planar proximity between the holographic object and the viewing volume. Adding one or more side surfaces angled toward the viewer increases the field of view relative to the planar field of view, decreases the separation between the display surface and the viewing volume relative to the planar separation, and increases the proximity between the display surface and the holographic object relative to the planar proximity. Angling the side surfaces further toward the viewer further increases the field of view, decreases the separation, and increases the proximity. In other words, the angled placement of the side surfaces increases the viewer's immersive experience.

[0105] Return to Figure 4D , in similar instances, Figure 4D A side view of a lightfield display system 450B with a converging surface in a "tilted" configuration is shown. In this example, lightfield display system 450B is positioned in a room having a front wall 452, a back wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. The first side wall, the second side wall, the back wall 454, the floor 474, and the ceiling 472 are all orthogonal. Lightfield display system 450B includes lightfield display modules that are tiled to form a converging surface 460 that covers the front wall. The front wall 452, and therefore the converging surface 460, comprises three sections: (i) a first section 462 that is substantially parallel to the rear wall 454 (i.e., the center surface), (ii) a second section 464 that connects the first section 462 to the ceiling 472 and is angled to project energy toward the center of the room (i.e., the first side surface), and (iii) a third section 464 that connects the first section 462 to the floor 474 and is angled to project energy toward the center of the room (i.e., the second side surface). The first section is a vertical plane in the room and has a horizontal axis and a vertical axis. The second and third sections are angled along the vertical axis toward the center of the room.

[0106] In this example, the viewing volume 468B of the light field display system 450B is located in the center of the room and is partially surrounded by three portions of the converging surface 460. Figure 4C Similar to the configuration shown, two side sections (e.g., second section 464 and third section 466) are angled to surround the viewer and form a wraparound surface. From the perspective of any viewer in holographic viewing volume 468B, the wraparound surface increases the viewing FOV. Additionally, the wraparound surface allows viewing volume 468B to be closer to the surface of the display, making projected objects appear closer. In other words, the angled placement of the side surfaces increases the field of view, reduces separation, and increases the proximity of the converging surfaces, thereby increasing the viewer's immersive experience. Furthermore, as will be discussed below, deflection optics can be used to optimize the size and positioning of viewing volume 468B.

[0107] The tilted configuration of the side portions of the converging surface 460 enables the holographic content to be presented closer to the viewing volume 468B than if the third portion 466 were not tilted. For example, the lower extremities (e.g., legs) of a character presented from a lightfield display system in a tilted configuration may appear closer and more realistic than if a lightfield display system with a flat front wall were used.

[0108] Additionally, the configuration of the light field display system and the environment in which it is located can inform the shape and position of the viewing volume and viewing sub-volumes.

[0109] Figure 4E For example, a top view of a lightfield display system 450C is shown having a converging surface 460 on a front wall 452 of a room. In this example, the lightfield display system 450C is located in a room having a front wall 452, a back wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown).

[0110] Lightfield display system 450C projects various light rays from converging surface 460. Light rays are projected from each location on the display surface into a range of angles centered on the viewing volume. Light rays projected from the left side of converging surface 460 have a horizontal angle range 481, light rays projected from the right side of the converging surface have a horizontal angle range 482, and light rays projected from the center of converging surface 460 have a horizontal angle range 483. This gradient deflection angle in the projected light rays across the display surface creates viewing volume 468C. Furthermore, this configuration avoids wasting display resolution when projecting light rays into sidewalls 456 and 458.

[0111] Figure 4F A side view of a light field display system 450D having a converging surface 460 on a front wall 452 of a room is shown. In this example, the light field display system 450D is located in a room having a front wall 452, a back wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. In this example, the floor is layered so that each layer rises in a stepwise manner moving from the front wall to the back wall. Here, each layer of the floor contains viewing sub-volumes (e.g., viewing sub-volumes 470A and 470B). The layered floor allows for non-overlapping viewing sub-volumes. In other words, each viewing sub-volume has a line of sight from the viewing sub-volume to the converging surface 460 that does not pass through another viewing sub-volume. In other words, this orientation creates a "stadium seating" effect, where the vertical offset between the layers allows each layer to "see" the viewing sub-volumes of the other layers. A light field display system containing non-overlapping viewing sub-volumes can provide a more enhanced viewing experience than a light field display system with viewing sub-volumes that do overlap. For example, in Figure 4FIn the configuration shown, different holographic content can be projected to viewers in viewing sub-volumes 470A and 470B.

[0112] Control of light field display systems

[0113] Figure 5A is a block diagram of a light field display system 500 according to one or more embodiments. The light field display system 500 includes a light field display assembly 510 and a controller 520. The light field display assembly 510 includes one or more light field display modules 512 that project a light field. The light field display module 512 may include a source / sensor system 514 that includes one or more integrated energy sources and / or one or more energy sensors that project and / or sense other types of energy. The controller 520 includes a data store 522, a network interface 524, and a light field processing engine 530. The controller 520 may also include a tracking module 526 and a viewer profiling module 528. In some embodiments, the light field display system 500 also includes a sensory feedback system 540 and a tracking system 550. Figure 1 、 2A The light field display system described in the context of FIG. 2B, 3A-3B, and 4A-4F is an embodiment of the light field display system 500. In other embodiments, the light field display system 500 includes more or fewer modules than those described herein. Similarly, functionality may be allocated between modules and / or different entities in a manner different from that described herein. Applications of the light field display system 500 will also be described below with respect to FIG. Figure 6-7 Discuss in detail.

[0114] The lightfield display assembly 510 provides holographic content in a holographic object volume that can be visible to a viewer positioned within the viewing volume. The lightfield display assembly 510 can provide the holographic content by executing display instructions received from the controller 520. The holographic content can include one or more holographic objects projected in front of a converging surface of the lightfield display assembly 510, behind a converging surface of the lightfield display assembly 510, or some combination thereof. Generating display instructions using the controller 520 is described in more detail below.

[0115] The light field display assembly 510 uses one or more light field display modules (e.g., any of the light field display modules 110, 210, 300A, and 300B) included in the light field display assembly 510 to provide holographic content. For convenience, the one or more light field display modules may be described herein as light field display modules 512. The light field display modules 512 may be tiled to form the light field display assembly 510. The light field display modules 512 may be configured into various seamless surface environments (e.g., single-sided, multi-sided, billboards, curved surfaces, etc.). In other words, the tiled light field display modules form a cohesive surface. As previously described, the light field display module 512 includes an energy device layer (e.g., energy device layer 220) and an energy waveguide layer (e.g., energy waveguide layer 240) that presents holographic content. The light field display module 512 may also include an energy relay layer (eg, the energy relay layer 230 ) that transfers energy between the energy device layer and the energy waveguide layer when presenting holographic content.

[0116] In some embodiments, the light field display system 500 provides holographic content based on a hardware configuration. The hardware configuration is the configuration of the physical components of the light field display system 500 that influence the presentation of holographic content. The physical components may include the light field display module 512, sensory feedback devices (e.g., acoustic projection devices, force actuation devices, pressure sensors, etc.), and tracking system devices (e.g., energy sensors, cameras, depth sensors, etc.). In one embodiment, the hardware configuration may include the arrangement of the light field display module, sensory feedback devices, and tracking system devices for the light field display system 500. In other embodiments, the hardware configuration may include the design of the light field display module. Design considerations for the light field display module may include resolution (i.e., the level of detail of the presented holographic content), the number of projection rays per degree (i.e., the ray density that determines angular resolution and projection distance), the field of view (i.e., the open observable area of the viewing volume), the deflection angle on the display surface (i.e., the angle between the projection rays and the normal of the display surface), the dimensions of the display surface (i.e., the height and width of the display panel), any other light field display module design considerations that influence the presentation of holographic content, or some combination thereof.

[0117] The light field display module 512 may also include other integrated systems configured for energy projection and / or energy sensing as previously described. For example, the light field display module 512 may include any number of energy devices (e.g., energy device 340) configured to project and / or sense energy. For convenience, the integrated energy projection system and the integrated energy sensing system of the light field display module 512 may be collectively described herein as the source / sensor system 514. The source / sensor system 514 is integrated within the light field display module 512 such that the source / sensor system 514 and the light field display module 512 share the same seamless energy surface. In other words, the aggregate surface of the light field display assembly 510 includes the functionality of both the light field display module 512 and the source / sensor module 514. In other words, the light field display assembly 510, including the light field display module 512 with the source / sensor system 514, can project energy and / or sense energy while simultaneously projecting a light field. For example, the light field display assembly 510 may include a light field display module 512 and a source / sensor system 514 configured as a dual energy surface or a bidirectional energy surface as previously described.

[0118] In some embodiments, the lightfield display system 500 uses a sensory feedback system 540 to enhance the generated holographic content with other sensory content (e.g., coordinated touch, audio, or scent). The sensory feedback system 540 can enhance the projection of the holographic content by executing display instructions received from the controller 520. The sensory feedback system 540 may include one or more sensory feedback devices and may be configured to provide sensory feedback simultaneously with the holographic object. Typically, the sensory feedback system 540 includes any number of sensory feedback devices (e.g., sensory feedback system 442) external to the lightfield display assembly 510. Some example sensory feedback devices may include coordinated acoustic projection and reception devices, haptic feedback, scent feedback, temperature feedback, force-actuated devices, pressure sensors, transducers, and the like. In some cases, the sensory feedback system 540 may have similar functionality to the lightfield display assembly 510, and vice versa. For example, both the sensory feedback system 540 and the lightfield display assembly 510 may be configured to generate a sound field. As another example, sensory feedback system 540 may be configured to generate a tactile surface while lightfield display assembly 510 is not configured to generate a tactile surface.

[0119] For illustration, in an example embodiment of the light field display system 500, the sensory feedback system 540 may include one or more acoustic projection devices. The one or more acoustic projection devices are configured to generate one or more pressure waves that complement the holographic content when executing the display instructions received from the controller 520. The generated pressure waves can be, for example, audible (for sound), ultrasonic (for touch), or a combination thereof. Similarly, the sensory feedback system 540 may include a fragrance projection device. The fragrance projection device may be configured to provide fragrance to some or all of the target area when executing the display instructions received from the controller. The fragrance device may be connected to an air circulation system (e.g., a duct, fan, or vent) to coordinate air flow within the target area. In addition, the sensory feedback system 540 may include a temperature adjustment device. The temperature adjustment device is configured to increase or decrease the temperature in some or all of the target area when executing the display instructions received from the controller 520.

[0120] In some embodiments, the sensory feedback system 540 is configured to receive input from a viewer of the light field display system 500. In this case, the sensory feedback system 540 includes various sensory feedback devices for receiving input from the viewer. The sensory feedback devices may include, for example, acoustic receiving devices (e.g., microphones), pressure sensors, joysticks, motion detectors, transducers, etc. The sensory feedback system may transmit the detected input to the controller 520 to coordinate the generation of holographic content and / or sensory feedback.

[0121] To illustrate, in an example embodiment of a lightfield display assembly, the sensory feedback system 540 includes a microphone. The microphone is configured to record audio generated by one or more viewers (e.g., voice commands, audio responses to the presentation of holographic content, etc.). The sensory feedback system 540 provides the recorded audio as viewer input to the controller 520. The controller 520 can use the viewer input to generate holographic content. Similarly, the sensory feedback system 540 can include a pressure sensor. The pressure sensor is configured to measure the force applied to the pressure sensor by the viewer. The sensory feedback system 540 can provide the measured force as viewer input to the controller 520.

[0122] In some embodiments, the light field display system 500 includes a tracking system 550. The tracking system 550 includes any number of tracking devices configured to track the movement of a viewer within the viewing volume of the light field display system 500, monitor the viewer's response to holographic content, and determine characteristics of the viewer within the viewing volume of the light field display system 500. Typically, the tracking devices are external to the light field display assembly 510. Some example tracking devices include a camera assembly ("camera"), a depth sensor, a structured light, a LIDAR system, a card scanning system, or any other tracking device that can track a viewer within a target area.

[0123] Tracking system 550 may include one or more energy sources that illuminate some or all of the target area with light. However, in some cases, when presenting holographic content, the target area is illuminated by natural and / or ambient light from lightfield display assembly 510. The energy sources project light in response to instructions received from controller 520. The light may be, for example, a structured light pattern, a light pulse (e.g., an IR flash), or some combination thereof. The tracking system may project light in the visible band (approximately 380 nm to 750 nm), the infrared (IR) band (approximately 750 nm to 1700 nm), the ultraviolet band (10 nm to 380 nm), some other portion of the electromagnetic spectrum, or some combination thereof. The sources may include, for example, light-emitting diodes (LEDs), micro-LEDs, laser diodes, time-of-flight depth sensors, tunable lasers, and the like.

[0124] Tracking system 550 can adjust one or more emission parameters when executing instructions received from controller 520. Emission parameters are parameters that affect how light is projected from the source of tracking system 550. Emission parameters can include, for example, brightness, pulse rate (including continuous illumination), wavelength, pulse length, some other parameter that affects how light is projected from the source assembly, or some combination thereof. In one embodiment, the source projects light pulses in time-of-flight operation.

[0125] Tracking system 550 may include one or more cameras configured to capture images of the area in front of one or more light field display modules 512. The cameras of tracking system 550 capture images of light (e.g., structured light patterns) reflected from the target area. The cameras capture images when executing tracking instructions received from controller 520. As previously described, light may be projected by a source of tracking system 550. The cameras may include one or more cameras. That is, the cameras may be, for example, an array of photodiodes (1-D or 2-D), a CCD sensor, a CMOS sensor, some other device that detects some or all of the light projected by tracking system 550, or some combination thereof. In one embodiment, tracking system 550 may contain one or more cameras located external to light field display assembly 510. In other embodiments, the cameras are included as part of light field display source / sensor module 514 included in light field display assembly 510. For example, as previously described, if the energy relay element of light field module 512 is a bidirectional energy layer that interweaves both an emissive display and an imaging sensor at energy device layer 220, then light field display assembly 510 can be configured to simultaneously project a light field and record imaging information from a viewing area in front of the display. In one embodiment, the image captured from the bidirectional energy surface forms a light field camera. The camera provides the captured image to controller 520.

[0126] When executing tracking instructions received from controller 520, the camera of tracking system 550 can adjust one or more imaging parameters. Imaging parameters are parameters that affect how the camera captures an image. Imaging parameters can include, for example, frame rate, aperture, gain, exposure length, frame timing, rolling shutter or global shutter capture mode, some other parameter that affects how the camera captures an image, or some combination thereof.

[0127] Tracking system 550 may include one or more depth sensors configured to detect the depth of objects in front of one or more light field display modules 512. The depth sensors may track the position of a viewer within the viewing volume.

[0128] Controller 520 controls light field display assembly 510 and any other components of light field display system 500. Controller 520 includes a data store 522, a network interface 524, a tracking module 526, a viewer profiling module 528, and a light field processing engine 530. In other embodiments, controller 520 includes more or fewer modules than described herein. Similarly, functionality can be allocated between modules and / or different entities in a manner different from that described herein. For example, tracking module 526 can be part of light field display assembly 510 or tracking system 550.

[0129] Data storage area 522 is a memory that stores information for light field display system 500. The stored information may include display instructions, tracking instructions, emission parameters, imaging parameters, a virtual model of the target area, tracking information, images captured by a camera, one or more viewer profiles, calibration data for light field display assembly 510, configuration data for light field display system 510 (including the resolution and orientation of light field module 512), a desired viewing volume geometry, content for graphic creation including 3-D models, scenes and environments, materials and textures, and other information that may be used by light field display system 500, or some combination thereof. Data storage area 522 is a memory such as read-only memory (ROM), dynamic random access memory (DRAM), static random access memory (SRAM), or some combination thereof.

[0130] The network interface 524 allows the light field display system to communicate with other systems or environments via a network. In one example, the light field display system 500 receives holographic content from a remote light field display system via the network interface 524. In another example, the light field display system 500 uses the network interface 524 to transmit holographic content to a remote data storage area.

[0131] Tracking module 526 tracks a viewer viewing content presented by light field display system 500. To do so, tracking module 526 generates tracking instructions that control the operation of a source and / or camera of tracking system 550 and provides the tracking instructions to tracking system 550. Tracking system 550 executes the tracking instructions and provides tracking input to tracking module 526.

[0132] Tracking module 526 can determine the position of one or more viewers within the target area. The determined position can be relative to, for example, a reference point (e.g., a display surface). In other embodiments, the determined position can be within a virtual model of the target area. The tracked position can be, for example, the tracked position of the viewer and / or the tracked position of a portion of the viewer (e.g., eye position, hand position, etc.). Tracking module 526 uses one or more captured images from cameras of tracking system 550 to determine the position. The cameras of tracking system 550 can be distributed around light field display system 500 and can capture stereo images, allowing tracking module 526 to passively track the viewer. In other embodiments, tracking module 526 actively tracks the viewer. That is, tracking system 550 illuminates a portion of the target area, images the target area (i.e., captures a light field from an area in front of one or more light field display modules 512), and tracking module 526 uses time-of-flight and / or structured light depth determination techniques to determine the position. Tracking module 526 uses the determined position to generate tracking information.

[0133] The tracking module 526 can also receive tracking information as input from a viewer of the light field display system 500. In one embodiment, the light field display assembly 510 can receive input based on a light field captured from an area in front of one or more light field display modules 512. The tracking information can include body movements corresponding to various input options provided to the viewer by the light field display system 500. For example, the tracking module 526 can track the viewer's body movements and distribute any of the various movements as input to the light field processing engine 530. The tracking module 526 can provide the tracking information to the data store 522, the network interface 524, the light field processing engine 530, the viewer profiling module 528, any other component of the light field display system 500, or some combination thereof.

[0134] To provide context for tracking module 526, consider an example embodiment of a light field display system 500 that displays holographic merchandise to a viewer. In response to displaying certain holographic merchandise, the viewer may respond with hand and / or arm movements that can be assigned to various inputs. Tracking system 550 may record the viewer's hand and / or arm movements and transmit the recording to tracking module 526. Tracking module 526 tracks the viewer's hand and / or arm movements in the recording and sends the input to light field processing engine 530. As described below, viewer profiling module 528 determines that information in the image indicates that the viewer's hand movements are associated with an input, such as to purchase the merchandise. Accordingly, light field processing engine 530 generates appropriate holographic content to confirm the purchase of the merchandise.

[0135] The light field display system 500 includes a viewer profiling module 528 configured to identify and profile a viewer. The viewer profiling module 528 generates a profile of a viewer (or multiple viewers) viewing holographic content displayed by the light field display system 500. The viewer profiling module 528 generates the viewer profile based in part on viewer input, viewer characteristics, and monitored viewer behavior, actions, and reactions. The viewer profiling module 528 can access information obtained from the tracking system 550 (e.g., recorded images, video, sound, etc.) and process the information to determine various information. In various examples, the viewer profiling module 528 can use any number of machine vision or machine hearing algorithms to determine viewer behavior, actions, and reactions. The monitored viewer behavior can include, for example, smiling, frowning, cheering, clapping, laughing, excitement, other changes in facial expression, gestures, or viewer movement.

[0136] More generally, a viewer profile may include any information received and / or determined about a viewer viewing holographic content from a light field display system. For example, each viewer profile may record the viewer's actions or responses to content displayed by the light field display system 500. Some example information that may be included in a viewer profile is provided below.

[0137] In some embodiments, the viewer profile may describe the viewer's responses to displayed holographic merchandise, displayed holographic content objects, etc. For example, the viewer profile may indicate that the viewer generally responds positively to printed clothing (displayed holographically).

[0138] In some embodiments, the viewer profile may indicate characteristics of a viewer viewing news information, such as a news broadcast using the light field display system 500 in a doctor's waiting room. The light field display system 500 is further configured to update the presentation of holographic content in response to characteristics of the viewer profile corresponding to the viewer. In this same example, the viewer in the waiting room is wearing a sweatshirt with a university logo printed on it. In this case, the viewer profile may indicate that the viewer is wearing a sweatshirt and may prefer holographic content associated with the university identified on the sweatshirt, such as holographic content related to upcoming events on the university campus, weather forecasts for the university's geographic location, recent sports scores for the university's athletic teams, and so on. More broadly, viewer characteristics that may be indicated in a viewer profile may include, for example, age, gender, race, clothing, viewing location, and so on.

[0139] In some embodiments, a viewer profile may indicate a viewer's preferences regarding desired holographic content. For example, a viewer profile may indicate that a viewer only prefers to view holographic content that is appropriate for the age of each member of their family. In another example, a viewer profile may indicate that a holographic object should display holographic content (e.g., on a wall) and that a holographic object should not display holographic content (e.g., above their head). A viewer profile may also indicate that a viewer prefers a haptic interface to be presented near them, or prefers to avoid the haptic interface.

[0140] In another example, a viewer profile indicates a history of holographic merchandise viewed by a particular viewer. For example, the viewer profile forming module 528 determines that the viewer has previously viewed a holographically displayed table. Accordingly, the light field display system 500 may display another similar table that the viewer may also appreciate, or an office chair that may be suitable for the previously viewed table.

[0141] In some embodiments, a viewer profile can also describe the characteristics and preferences of a group of viewers rather than a specific viewer. For example, the viewer profile formation module 528 can generate a viewer profile for a family using the light field display system 500 implemented in a home. In one example, the viewer profile formation module 528 creates a viewer profile for a family that has characteristics describing common interests among the family members. The profile can be further broken down to identify the percentage of family members who may have different interests.

[0142] The viewer profile formation module 528 can also access profiles associated with a particular viewer (or multiple viewers) from one or more third-party systems to build a viewer profile. For example, a viewer purchases merchandise from a third-party vendor that is linked to the viewer's social media account. Thus, the viewer's purchase history is linked to their social media account. When the viewer enters a retail store that implements the light field display system 500, the viewer profile formation module 528 can access information from their social media account to build (or enhance) the viewer's profile.

[0143] In some embodiments, the data storage area 522 includes a viewer profile storage area that stores viewer profiles generated, updated, and / or maintained by the viewer profile formation module 528. The viewer profile formation module 528 can update the viewer profile in the data storage area at any time. For example, in one embodiment, when a particular viewer views holographic content provided by the light field display system 500, the viewer profile storage area receives and stores information about the particular viewer in its viewer profile. In this example, the viewer profile formation module 528 includes a facial recognition algorithm that can recognize the viewer and positively identify the viewer when viewing the presented holographic content. To illustrate, when a viewer enters a target area of the light field display system 500, the tracking system 550 obtains an image of the viewer. The viewer profile formation module 528 inputs the captured image and uses the facial recognition algorithm to identify the viewer's face. The identified face is associated with the viewer profile in the profile storage area, and therefore, all input information obtained about the viewer can be stored in the viewer's profile. The viewer profiling module may also utilize a card identification scanner, voice identifier, radio frequency identification (RFID) chip scanner, bar code scanner, or the like to positively identify the viewer.

[0144] In embodiments where the viewer profiling module 528 can positively identify a viewer, the viewer profiling module 528 can determine each visit to the light field display system 500 by each viewer. The viewer profiling module 528 can then store the time and date of each visit in the viewer profile for each viewer. Similarly, the viewer profiling module 528 can store input received from the viewer from any combination of the sensory feedback system 540, the tracking system 550, and / or the light field display assembly 510 each time it occurs. The viewer profiling module 528 can also receive other information about the viewer from other modules or components of the controller 520, which can then be stored with the viewer profile. Other components of the controller 520 can then also access the stored viewer profile to determine subsequent content to provide to the viewer.

[0145] The light field processing engine 530 generates holographic content that includes light field data and data for all sensory domains supported by the light field display system 500. For example, the light field processing engine 530 can generate 4-D coordinates in a rasterized format ("rasterized data") that, when executed by the light field display assembly 510, enable the light field display assembly 510 to render holographic content. The light field processing engine 530 can access the rasterized data from the data store 522. Additionally, the light field processing engine 530 can construct the rasterized data from a vectorized dataset. Vectorized data is described below. The light field processing engine 530 can also generate the sensory instructions necessary to provide sensory content that enhances the holographic object. As described above, when executed by the light field display system 500, the sensory instructions can generate tactile surfaces, sound fields, and other forms of sensory energy supported by the light field display system 500. The light field processing engine 530 can access the sensory instructions from the data store 522 or construct the sensory instructions from the vectorized dataset. In general, the 4-D coordinate and sensory data represent holographic content as display instructions that can be executed by a lightfield display system to generate holographic and sensory content. More generally, holographic content can take the form of CG content having the following: ideal lightfield coordinates, live-action content, rasterized data, vectorized data, electromagnetic energy transmitted by a set of repeaters, instructions sent to a set of energy devices, energy locations on one or more energy surfaces, a set of energy propagation paths projected from a display surface, a holographic object visible to a viewer or audience, and many other similar forms.

[0146] The amount of rasterized data describing the energy flow through the various energy sources in light field display system 500 is very large. While it is possible to display the rasterized data on light field display system 500 when accessed from data storage 522, it is not possible to efficiently transmit, receive (e.g., via network interface 524), and subsequently display the rasterized data on light field display system 500. For example, consider the rasterized data representing a short film of holographic projection performed by light field display system 500. In this example, light field display system 500 includes a display containing several gigapixels, and the rasterized data contains information for every pixel position on the display. The corresponding size of the rasterized data is enormous (e.g., several gigabytes per second of movie display time) and is unmanageable for efficient transmission over commercial networks via network interface 524. This efficient transmission problem can be magnified for applications involving real-time streaming of holographic content. Additional problems arise when storing rasterized data solely on data storage 522 when an interactive experience is desired using input from sensory feedback system 540 or tracking module 526. To enable an interactive experience, the light field content generated by the light field processing engine 530 may be modified in real time in response to sensory or tracking input. In other words, in some cases, the light field content cannot be simply read from the data store 522.

[0147] Thus, in some configurations, data representing holographic content displayed by the light field display system 500 can be transmitted to the light field processing engine 530 in a vectorized data format ("vectorized data"). Vectorized data can be several orders of magnitude smaller than rasterized data. In addition, vectorized data provides high image quality while having a data set size that enables efficient sharing of data. For example, the vectorized data can be a sparse data set derived from a denser data set. Thus, based on how the sparse vectorized data is sampled from the dense rasterized data, the vectorized data can have an adjustable balance between image quality and data transfer size. The adjustable sampling used to generate the vectorized data enables optimization of image quality at a given network speed. Thus, the vectorized data enables efficient transmission of holographic content via the network interface 524. The vectorized data also enables real-time streaming of holographic content over commercial networks.

[0148] In some embodiments, the light field display system 500 may be configured to receive holographic content in an encoded format via a network and further configured to decode the holographic content into a format for presentation to a viewer. In some embodiments, the encoded format may be a vectorized format, and the decoded format may be a rasterized format. In summary, the light field processing engine 530 may generate holographic content from rasterized data accessed from the data storage area 522, vectorized data accessed from the data storage area 522, or vectorized data received via the network interface 524. In various configurations, the vectorized data may be encoded prior to data transmission and decoded after receipt by the light field controller 520. In some instances, the vectorized data is encoded for additional data security and performance improvements related to data compression. For example, the vectorized data received via the network interface may be encoded vectorized data received from a holographic streaming application. In some instances, the vectorized data may require a decoder, the light field processing engine 530, or both to access the information content encoded in the vectorized data. The encoder and / or decoder systems may be available for consumer use or licensed to third-party vendors.

[0149] The vectorized data contains all the information for each sensory domain supported by the light field display system 500 in a manner that can support an interactive experience. For example, the vectorized data used for an interactive holographic experience can include any vectorized properties that can provide accurate physical effects for each sensory domain supported by the light field display system 500. The vectorized properties can include any properties that can be synthetically programmed, captured, computationally evaluated, and so on. The light field processing engine 530 can be configured to convert the vectorized properties in the vectorized data into rasterized data. The light field processing engine 530 can then use the light field display assembly 510 to project holographic content converted from the vectorized data. In various configurations, vectorized features may include: one or more red / green / blue / alpha channel (RGBA) + depth images; multiple view images with or without depth information of varying resolutions, which may include a high-resolution center image and other views of lower resolution; material features such as albedo and reflectivity; surface normals; other optical effects; surface identification; geometric object coordinates; virtual camera coordinates; display plane position; lighting coordinates; tactile stiffness of the surface; tactile extensibility; tactile intensity; amplitude and coordinates of the sound field; environmental conditions; somatosensory energy vectors associated with mechanoreceptors for texture or temperature, audio; and any other sensory domain features. Many other vectorized features are also possible.

[0150] The light field display system 500 can also generate an interactive viewing experience. That is, the holographic content can respond to input stimuli containing information about the viewer's position, gestures, interactions with the holographic content, or other information from the viewer profiling module 528 and / or the tracking module 526. For example, in an embodiment, the light field processing engine 530 uses vectorized data received via the network interface 524 for real-time performance to create the interactive viewing experience. In another example, if a holographic object needs to move immediately in a certain direction in response to a viewer interaction, the light field processing engine 530 can update the rendering of the scene so that the holographic object moves in the desired direction. This may require the light field processing engine 530 to use the vectorized data set to render a light field in real time based on a 3-D graphics scene with appropriate object placement and movement, collision detection, occlusion, color, shadows, lighting, etc. to correctly respond to viewer interactions. The light field processing engine 530 converts the vectorized data into rasterized data for presentation by the light field display assembly 510.

[0151] The rasterized data includes holographic content instructions and sensory instructions (display instructions) representing the real-time performance. The light field display assembly 510 simultaneously projects the holographic and sensory content of the real-time performance by executing the display instructions. The light field display system 500 monitors viewer interactions (e.g., voice responses, touch, etc.) with the presented real-time performance via the tracking module 526 and the viewer profiling module 528. In response to viewer interactions, the light field processing engine can create an interactive experience by generating additional holographic and / or sensory content for display to the viewer.

[0152] To illustrate, consider an example embodiment of a light field display system 500 that includes a light field processing engine 530 that generates a holographic object representing a viewer's bicycle. The viewer can move to touch the holographic object representing the bicycle. Accordingly, a tracking system 550 tracks the movement of the viewer's hand relative to the holographic object. The viewer's movements are recorded by the tracking system 550 and sent to the controller 520. The tracking module 526 continuously determines the movement of the viewer's hand and sends the determined movement to the light field processing engine 530. The light field processing engine 530 determines the placement of the viewer's hand and adjusts the real-time rendering of the graphics to include any desired changes (e.g., positioning, color, or occlusion) in the holographic object. The light field processing engine 530 instructs the light field display assembly 510 (and / or the sensory feedback system 540) to generate a tactile surface using a volumetric tactile projection system (e.g., using an ultrasonic speaker). The generated tactile surface corresponds to at least a portion of the holographic object and occupies substantially the same space as some or all of the exterior surfaces of the holographic object. The light field processing engine 530 uses the tracking information to dynamically instruct the light field display assembly 510 to move the position of the tactile surface and the position of the rendered holographic object, thereby providing the viewer with both a visual and tactile perception of touching the bicycle. More simply, as the viewer watches their hand touch the holographic bicycle, they simultaneously feel tactile feedback indicating that their hand has touched the holographic bicycle, and that the bicycle changes position or motion in response to the touch. In some examples, rather than presenting the interactive bicycle accessed from the data store 522, the interactive bicycle may be received as part of holographic content received from a live streaming application via the network interface 524.

[0153] The light field processing engine 530 can also create holographic content for display by the light field display system 500. Importantly, creating holographic content for display is distinct from accessing or receiving holographic content for display. That is, when creating content, the light field processing engine 530 generates entirely new content for display rather than accessing previously generated and / or received content. The light field processing engine 530 can use information from the tracking system 550, the sensory feedback system 540, the viewer profiling module 528, the tracking module 526, or some combination thereof to create holographic content for display. In some examples, the light field processing engine 530 can access information from elements of the light field display system 500 (e.g., tracking information and / or viewer profiles), create holographic content based on that information, and, in response, display the created holographic content using the light field display system 500. When displayed by the light field display system 500, the created holographic content can be enhanced with other sensory content (e.g., touch, audio, or scent). Furthermore, the light field display system 500 can store the created holographic content so that it can be displayed in the future.

[0154] Dynamic content generation for light field display systems

[0155] In some embodiments, the light field processing engine 530 incorporates an artificial intelligence (AI) model to create holographic content for display by the light field display system 500. The AI model may include a supervised or unsupervised learning algorithm, including but not limited to a regression model, a neural network, a classifier, or any other AI algorithm. The AI model can be used to determine viewer preferences based on viewer information recorded by the light field display system 500 (e.g., by the tracking system 550), which may include information about the viewer's behavior.

[0156] The AI model can access information from the data store 522 to create holographic content. For example, the AI model can access viewer information from one or more viewer profiles in the data store 522, or can receive viewer information from various components of the light field display system 500. To illustrate, the AI model can determine that a viewer may enjoy viewing another holographic merchandise, considering the viewing history of various other holographic merchandise. The AI model can also store the learned preferences of each viewer in a viewer profile store in the data store 522. In some instances, the AI model can create holographic content for a single viewer rather than a group of viewers.

[0157] One example of an AI model that can be used to identify characteristics of a viewer, identify reactions, and / or generate holographic content based on the identified information is a convolutional neural network model with node layers, where the values at the nodes of the current layer are transformations of the values at the nodes of the previous layer. The transformations in the model are determined by a set of weights and parameters connecting the current layer and the previous layer. For example, the AI model may include five node layers: layers A, B, C, D, and E. The transformation from layer A to layer B is given by function W1, the transformation from layer B to layer C is given by W2, the transformation from layer C to layer D is given by W3, and the transformation from layer D to layer E is given by W4. In some instances, the transformation can also be determined by a set of weights and parameters used to transform between previous layers in the model. For example, the transformation W4 from layer D to layer E can be based on the parameters used to complete the transformation W1 from layer A to B.

[0158] The input to the model can be an image encoded onto convolutional layer A acquired by the tracking system 550, and the output of the model is holographic content decoded from the output layer E. Alternatively or in addition, the output can be determined characteristics of the viewer in the image. In this example, the AI model identifies potential information in the image that represents the viewer's characteristics in the identification layer C. The AI model reduces the dimensionality of convolutional layer A to the dimensionality of identification layer C to identify any characteristics, actions, responses, etc. in the image. In some examples, the AI model then increases the dimensionality of identification layer C to generate holographic content.

[0159] The image from the tracking system 550 is encoded into convolutional layer A. The image input in convolutional layer A can be related to various characteristics and / or reaction information in identification layer C, etc. The relevant information between these elements can be retrieved by applying a set of transformations between the corresponding layers. That is, convolutional layer A of the AI model represents the encoded image, and identification layer C of the model represents a smiling viewer. The smiling viewer in a given image can be identified by applying transformations W1 and W2 to the pixel values of the image in the space of convolutional layer A. The weights and parameters used for the transformation can indicate the relationship between the information contained in the image and the identification of the smiling viewer. For example, the weights and parameters can be quantifications of the shape, color, size, etc. contained in the information representing the smiling viewer in the image. The weights and parameters can be based on historical data (e.g., previously tracked viewers).

[0160] Smiling viewers in the image are identified in identification layer C. Identification layer C represents the identified smiling viewers based on the underlying information about the smiling viewers in the image.

[0161] The identified smiling viewer in the image can be used to generate holographic content. To generate the holographic content, the AI model starts at the identification layer C and applies transformations W2 and W3 to the values of the given identified smiling viewer in the identification layer C. The transformation produces a set of nodes in the output layer E. The weights and parameters used for the transformation can indicate the relationship between the identified smiling viewer and specific holographic content and / or preferences. In some cases, the holographic content is output directly from the nodes of the output layer E, while in other cases, the content generation system decodes the nodes of the output layer E into holographic content. For example, if the output is a set of identified characteristics, the light field processing engine 530 can use the characteristics to generate holographic content.

[0162] Additionally, an AI model may include layers referred to as intermediate layers. Intermediate layers are layers that do not correspond to images, do not identify features / reactions, etc., or do not generate holographic content. For example, in a given example, layer B is an intermediate layer between convolutional layer A and identification layer C. Layer D is an intermediate layer between identification layer C and output layer E. Hidden layers are latent representations of different aspects of identification that are not observed in the data, but can control the relationships between image elements when identifying features and generating holographic content. For example, a node in a hidden layer may have a strong connection (e.g., a large weight value) to input values and identification values that share the commonality of "a happy person smiling." As another example, another node in a hidden layer may have a strong connection to input values and identification values that share the commonality of "a scared person screaming." Of course, any number of connections can exist in a neural network. Furthermore, each intermediate layer is a combination of functions, such as residual blocks, convolutional layers, pooling operations, skip connections, concatenation, etc. Any number of intermediate layers B can be used to reduce a convolutional layer to an identification layer, and any number of intermediate layers D can be used to add an identification layer to the output layer.

[0163] In one embodiment, the AI model comprises a deterministic method that has been trained with reinforcement learning (thereby creating a reinforcement learning model). The model is trained to improve the quality of performance using measurements from the tracking system 550 as input and changes in the created holographic content as output.

[0164] Reinforcement learning is a machine learning system in which the machine learns "what to do" - how to map situations to actions - so as to maximize a digital reward signal. Instead of telling the learner (e.g., the light field processing engine 530) which actions to take (e.g., generate a prescribed holographic content), the learner discovers which actions produce the highest reward by trying the actions (e.g., improving the quality of the holographic content by getting more people to cheer). In some cases, an action can affect not only the immediate reward, but also the next situation, and therefore all subsequent rewards. These two features - trial and error search and delayed rewards - are two of the distinguishing characteristics of reinforcement learning.

[0165] Reinforcement learning is defined not by characterizing a learning method, but by characterizing a learning problem. Essentially, a reinforcement learning system captures the important aspects of the problem faced by a learning agent interacting with its environment to achieve a goal. That is, in the example of generating advertisements for a brand of jeans, the reinforcement learning system captures information about the viewers in a venue (e.g., age, personality, etc.). Such an agent senses the state of the environment and takes actions that influence that state in order to achieve one or more goals (e.g., create content that generates the most impressions, and thus the most conversions). In its most basic form, a reinforcement learning formulation encompasses three aspects of the learner: sensations, actions, and goals.

[0166] One of the challenges that arise in reinforcement learning is the trade-off between exploration and exploitation. To increase the reward in the system, a reinforcement learning agent prefers actions that have been tried in the past and found to be effective in producing reward. However, to discover actions that produce reward, the learning agent will choose actions that it has not previously chosen. The agent "exploits" information that it already knows to obtain reward, but it also "explores" information to make better action choices in the future. The learning agent tries various actions and gradually prefers those that appear to be the best as it continues to try new actions. On a random task, each action is typically tried many times to get a reliable estimate of its expected reward. For example, if a light field processing engine creates holographic content that the light field processing engine knows will cause a viewer to laugh after a long period of time, the light field processing engine can change the holographic content so that the time until the viewer laughs is reduced.

[0167] Furthermore, reinforcement learning considers the entire problem of goal-oriented agents interacting with uncertain environments. Reinforcement learning agents have clear goals, can sense aspects of their environment, and can choose actions that receive high rewards (i.e., high conversion rates). Furthermore, agents typically operate despite significant uncertainty in the environment they face. When reinforcement learning involves planning, the system addresses the interaction between planning and real-time action selection, as well as the problem of how to acquire and improve environmental elements. In order for reinforcement learning to make progress, it is necessary to separate and study important subproblems that play a clear role in the complete interactive goal-seeking agent.

[0168] The reinforcement learning problem is a framework for machine learning problems in which interactions are processed and actions are performed to achieve a goal. The learner and decision maker are called agents (e.g., the light field processing engine 530). The things that interact with it, including everything outside the agent, are called the environment (e.g., viewers in a venue, viewers in a shopping mall, viewers on the subway, etc.). The two interact continuously, the agent chooses actions (e.g., creating holographic content), and the environment responds to these actions and presents new situations to the agent. The environment also brings rewards, that is, special numerical values that the agent tries to maximize over time. In one context, the reward plays the role of maximizing the viewer's positive response to the holographic content. The complete specification of the environment defines a task, which is an instance of a reinforcement learning problem.

[0169] To provide more context, the agent (e.g., light field processing engine 530) and the environment interact at each discrete time step in a series of discrete time steps, t = 0, 1, 2, 3, etc. At each time step t, the agent receives the environment state s t Some representation of (e.g., measurements from tracking system 550). State s t In S, where S is the set of possible states. Based on state s t At time step t, the agent chooses action a t (For example, offering a discount on a brand of jeans.) Action a t In A(s t ), where A(s t ) is the set of possible actions. One time state later (partially as a result of its actions), the agent receives a numerical reward r t+1 . Status r t+1 Within R, where R is the set of possible rewards. Once the agent receives the reward, it chooses a new state s t+1 .

[0170] At each time step, the agent implements a mapping from states to the probability of choosing each possible action. This mapping is called the agent’s policy and is denoted as π t , where π t (s,a) is if a t =a, then s t = probability of s. Reinforcement learning methods can determine how an agent changes its policy due to the states and rewards generated by its actions. The agent's goal is to maximize the total number of rewards received over time.

[0171] This reinforcement learning framework is very flexible and can be applied to many different problems in many different ways (for example, generating holographic content). The framework proposes that any problem (or purpose) of learning goal-directed behavior, regardless of the details of the sensory, memory, and control devices, can be reduced to three signals passed back and forth between the agent and its environment: a signal representing the choice made by the agent (the action), a signal representing the basis for the choice (the state), and a signal defining the agent's goal (the reward).

[0172] Of course, the AI model can include any number of machine learning algorithms. Some other AI models that can be employed are linear and / or logistic regression, classification and regression trees, k-means clustering, vector quantization, and the like. Regardless, the light field processing engine 530 typically takes input from the tracking module 526 and / or the viewer profiling module 528 and, in response, the machine learning model creates holographic content. Similarly, the AI model can guide the rendering of the holographic content.

[0173] The light field processing engine 530 can create holographic content based on the displayed holographic product. For example, the displayed holographic product can be associated with a set of metadata describing the product characteristics. The metadata can include, for example, color, material, ratings from other buyers, cost, sales volume, etc. The light field processing engine 530 can access any metadata describing the holographic product and generate holographic content for presentation. For example, a holographic product representing a sofa is being presented by the light field display system 500 implemented in the viewer's home. The light field processing engine 530 accesses the metadata of the sofa to create holographic content of the walls of the home to complement the sofa. Here, the metadata can include color and material. The light field processing engine 530 inputs the metadata into the AI model and, in response, receives holographic content displayed on the walls of the home.

[0174] In an example, the light field processing engine 530 can convert a traditional two-dimensional (2-D) movie into holographic content for display by the light field display system. For example, the light field processing engine 530 can input the traditional movie into an AI model, and the AI model converts any portion of the traditional movie into holographic content. In an example, the AI model can convert the traditional movie into holographic content by using a machine learning algorithm trained by converting two-dimensional data into holographic data. In various cases, the training data can be previously generated, created, or some combination of the two. The light field display system 500 can then display the holographic version of the movie instead of the traditional two-dimensional version of the movie.

[0175] Digital Signage Content Distribution System

[0176] Figure 5B is a block diagram of a light field digital signage content distribution system 560 in conjunction with a light field display system for digital signage according to one or more embodiments. Figure 5BThe illustrated light field digital signage content distribution system 560 includes one or more client light field display systems 500A and 500B, a network 570, one or more third-party systems 580, and an online system 590. In alternative configurations, different and / or additional components may be included in the light field digital signage content distribution system 560. For example, the online system 590 may include a social networking system, a content sharing network, or another system that provides content to viewers.

[0177] The client light field display systems 500A and 500B are capable of displaying holographic content, receiving input, and transmitting and / or receiving data via a network 570. The client light field display systems 500A and 500B are embodiments of the light field display system 500. Thus, each client light field display system includes a controller configured to receive holographic content via the network 570 and a light field display assembly (e.g., light field display assembly 510). The light field display assembly may include one or more light field display modules (e.g., light field display module 512) that display the holographic content as digital signage within a holographic object volume to a viewer located within a viewing volume. The client light field display systems 500A and 500B are configured to communicate via the network 570. In some embodiments, the client light field display systems 500A and 500B execute an application that allows a viewer of the client light field display system to interact with the online system 590. For example, the client light field display system 500A executes a browser application to implement interaction between the client light field display system 500A and the online system 590 via the network 570. In other embodiments, the client light field display system 500A executes a browser application to implement interaction between the client light field display system 500A and the online system 590 via the network 570. or ANDROID TM The client light field display systems 500A and 500B interact with the online system 590 through an application programming interface (API) running on the client light field display system 500A. As previously discussed, for efficient transmission speed, the data of the client light field display systems 500A and 500B can be transmitted as vectorized data via the network 570. The light field processing engine (e.g., the light field processing engine 530) at each client light field display system can decode the vectorized data and convert it into a rasterized format for display on a corresponding light field display assembly (e.g., the light field display assembly 510).

[0178] Client light field display systems 500A and 500B are configured to communicate via network 570 using both wired and / or wireless communication systems, which may include any combination of local area networks and / or wide area networks. In some embodiments, network 570 utilizes standard communication technologies and / or protocols. For example, network 570 includes communication links utilizing technologies such as Ethernet, 802.11, Worldwide Interoperability for Microwave Access (WiMAX), 3G, 4G, Code Division Multiple Access (CDMA), Digital Subscriber Line (DSL), and the like. Examples of networking protocols used for communication over network 570 include Multi-Protocol Label Switching (MPLS), Transmission Control Protocol / Internet Protocol (TCP / IP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), and File Transfer Protocol (FTP). Data exchanged over network 570 may be represented using any suitable format, such as Hypertext Markup Language (HTML) or Extensible Markup Language (XML). In some embodiments, all or a portion of the communication links of network 570 may be encrypted using any suitable technology or technologies.

[0179] One or more third-party systems 580 can be coupled to the network 570 to communicate with the online system 590. In some embodiments, the third-party system 580 is a signage control system, such as a content provider, that transmits holographic content to be distributed to the client light field display systems 500A and 500B via the network 570. In some embodiments, the third-party system 580 can also transmit the holographic content to the online system 590, which can then distribute the holographic content to the client light field display systems 500A and 500B. Each third-party system 580 has a content storage area 582 that can store holographic content items that can be distributed for presentation to the client light field display systems 500A and 500B. The third-party system 580 can provide holographic content to one or more client light field display systems 500A and 500B in exchange for payment. In one embodiment, the holographic content items can be associated with a cost that the online system 590 can collect when the holographic content items are distributed to the client light field display systems 500A and 500B for presentation.

[0180] Online system 590 facilitates the distribution of holographic content by providing it to client light field display systems 500A and 500B in exchange for payment. Holographic content is provided via network 570. Online system 590 includes a viewer profile store 592, a content store 594, a transaction module 596, and a content distribution module 598. In other embodiments, online system 590 may include additional, fewer, or different components for various applications. Conventional components, such as network interfaces, security features, load balancers, failover servers, management and network operations consoles, etc., are not shown to avoid obscuring the details of the system architecture.

[0181] Each viewer of the online system 590 can be associated with a viewer profile, which is stored in the viewer profile storage area 592. The viewer profile contains declarative information about the viewer that is explicitly shared by the viewer, and may also contain profile information inferred by the online system 590. In some embodiments, the viewer profile contains multiple data fields, each of which describes one or more attributes of the corresponding online system viewer. Examples of information stored in a viewer profile include biographical, demographic, and other types of descriptive information, such as work experience, educational history, gender, hobbies or preferences, location, etc. The viewer profile may also store other information provided by the viewer, such as an image or video. In some embodiments, the viewer's image can be tagged with information identifying the online system viewer displayed in the image, as well as information identifying the tagged viewer's image stored in the viewer's viewer profile. The viewer profiles in viewer profile storage 592 may also maintain references to actions performed by corresponding viewers on content items in content storage 594, including monitored viewer responses or viewer characteristics captured by a tracking system (e.g., tracking system 550) and determined by a tracking module (e.g., tracking module 526). The monitored viewer responses may include the viewer's position within the viewing volume, the viewer's movements, the viewer's gestures, the viewer's facial expressions, and the viewer's gaze. The light field display assembly may update the presentation of holographic content in response to the monitored viewer responses. The viewer's characteristics may include the viewer's demographic information, work experience, educational history, gender, income, amount spent on purchases, hobbies, location, age, viewing history, time spent on items, categories of previously viewed items, and purchase history. The light field display assembly may update the presentation of holographic content in response to the viewer's characteristics. In some embodiments, viewer profile storage 592 may store viewer characteristics and viewer information inferred by the online system. In some embodiments, a viewer profile may store information provided by one or more client light field display systems, which may include provided information and / or information recorded or inferred from a viewer profile formation module (e.g., viewer profile formation module 528).

[0182] While viewer profiles in the viewer profile storage area 592 are often associated with individuals, allowing individuals to interact with each other via the online system 590, viewer profiles can also be stored for entities such as businesses or organizations. This allows entities to establish a presence on the online system 590 to connect and exchange content with other online system viewers. An entity can use a brand page associated with the entity's viewer profile to publish information about itself, its products, or provide other information to viewers of the online system 590. The viewer profile associated with the brand page may contain information about the entity itself, thereby providing viewers with context or information data about the entity. In one embodiment, other viewers of the online system 590 can interact with the brand page (e.g., connect to the brand page to receive information posted to the brand page or receive information from the brand page). The viewer profile in the viewer profile storage area 592 can maintain references to interactions performed by the corresponding viewer. As described above, any information stored in the viewer profile (e.g., in the viewer profile formation module 528) can be used as input to a machine learning model to create holographic content displayed to the viewer.

[0183] The content storage area 594 stores holographic content, such as holographic content to be distributed to viewers of one or more client light field display systems 500A and 500B. Examples of holographic content may include advertisements (e.g., promoting upcoming sales, promoting a brand, etc.), announcements (e.g., political speeches, motivational speeches, etc.), public service alerts (e.g., tornado warnings, AMBER alerts, etc.), news information (e.g., headlines, sports scores, etc.), weather information (e.g., local weather forecasts, air quality index, etc.), venue information (e.g., box office hours, upcoming show schedules, etc.), information about traffic or travel conditions (e.g., traffic reports, road closures, etc.), information about business entities (e.g., office directories, business hours, etc.), performances (e.g., concerts, plays, etc.), artistic content (e.g., sculptures, ceramics, etc.), any other holographic content, or any combination thereof. In some embodiments, online system viewers can create the holographic content stored by the content storage area 594. In other embodiments, the holographic content is received from a third-party system 580 independent of the online system 590. An object in content store 594 may represent a single piece of content or a content "item."

[0184] The transaction module 596 provides holographic content to one or more client light field display systems 500A and 500B in exchange for payment. In one embodiment, the transaction module 596 manages the distribution of holographic content stored in the content storage area 594 to the client light field display systems 500A and 500B via the network 570. In one embodiment, the client light field display systems 500A and / or 500B, or the networked entity owner of the client light field display systems 500A and 500B, can provide payment for a particular item of holographic content, and the transaction can be managed by the transaction module 596. Alternatively, the third-party system 580 can provide content from the content storage area 582 to the light field display systems 500A and / or 500B in exchange for a transaction fee provided to the transaction module 596. In other embodiments, the online system 590 can directly distribute content to the client light field display systems 500A and 500B, regardless of whether the transaction module 596 charges the account of a particular entity. In some embodiments, the client light field display systems 500A and 500B are associated with one or more viewer profiles, which are charged by the transaction module 596 for the cost of presenting the holographic content item. In some embodiments, the holographic content item can be purchased and used indefinitely or rented for a period of time. All or a portion of the payment collected by the transaction module 596 can then be provided to the provider of the holographic content item. For example, a third-party system that provides holographic advertisements for products can receive a portion of the payment collected from the client light field display systems 500A and 500B for product purchases associated with the holographic advertisements.

[0185] The content distribution module 598 provides holographic content items to the client light field display systems 500A and 500B. The content distribution module 598 can receive a request from the transaction module 596 for a holographic content item to be presented to the client light field display system 500A and / or 500B. The content distribution module 598 retrieves the holographic content item from the content storage area 594 and provides the holographic content item to the client light field display system 500A and / or 500B for display to a viewer.

[0186] In some embodiments, the client-side light field display systems 500A and 500B may record a presentation instance of holographic content depending in part on whether input is received. In one embodiment, the client-side light field display systems 500A and 500B may be configured to receive input in response to the presentation of holographic content. The holographic content may be a holographic advertisement for a physical good (e.g., a pair of shoes), a digital asset (e.g., a downloadable album), a service to be rendered (e.g., a house painter), other holographic advertisements, or some combination thereof. For holographic advertisements, the received input may be used to confirm the impression or reach of the holographic advertisement. In some embodiments, if a viewer responds to a prompt provided during the presentation of the holographic content, the client-side light field display systems 500A and 500B may confirm the presentation instance of the holographic content. For example, the client-side light field display system 500A receives voice input from the viewer (e.g., after being prompted), and the client-side light field display system 500A uses the voice input to confirm the presentation of the holographic content. The client light field display systems 500A and 500B can use a combination of the received input and other metrics (e.g., information obtained by the tracking system 550) to confirm the presentation instance of the holographic content. In other embodiments, the client light field display systems 500A and 500B can be configured to update the presentation of the holographic content in response to the received input. For example, the client light field display system 500A receives voice input from a viewer (e.g., after being prompted), where the viewer expresses interest in a product presented in a holographic advertisement, and the client light field display system 500A updates the presented holographic content with the presented product (e.g., providing more pricing information about the product).

[0187] In other embodiments with holographic advertisements, the client-side light field display systems 500A and 500B provide viewers with transition opportunities. In some embodiments, the received input may correspond to a transition associated with the holographic content. A transition opportunity is an opportunity for the viewer to respond to the holographic advertisement. Responding to the holographic advertisement may typically correspond to providing the viewer with an option to make a purchase based on the content presented within the holographic advertisement. For example, the client-side light field display system 500A may present a purchase option for a leather jacket. In other cases, responding to the holographic advertisement may provide the viewer with an option to request further information based on the content presented within the holographic advertisement. The client-side light field display systems 500A and 500B may provide a prompt with the presentation of the holographic advertisement. Upon receiving an input to transition after presenting the holographic advertisement, the client-side light field display systems 500A and 500B may present subsequent holographic content based on the received input to complete the transition. In these cases, the light field processing engine 530 provides the subsequent holographic content. In other cases, the client-side light field display system 500A can direct the viewer to the advertiser to proceed with the conversion, for example, by directing the viewer to a web browser (e.g., on a mobile device or presented by the client-side light field display system 500A) or by directing the viewer to an application (e.g., on a mobile device). The conversion can be made through a purchase. Following the above example, the viewer provides input corresponding to a purchase option, and the client-side light field display system 500A can respond by presenting a holographic purchase page to allow the viewer to complete the purchase of the leather jacket. In some embodiments, the conversion of the presented instance of the holographic advertisement can be recorded by the client-side light field display system 500A, which can be used to process compensation costs.

[0188] In some configurations, the client light field display systems 500A and 500B in the digital signage content distribution system 560 may have different hardware configurations. Holographic content can be rendered based on the hardware configuration of the client light field display systems 500A and 500B. The hardware configuration may include resolution, the number of projected rays per degree, field of view, deflection angle on the display surface, and the dimensions of the display surface. Each hardware configuration may generate or utilize sensory data in a different data format. As previously discussed, holographic content, including all sensory data (e.g., holographic, audio, and haptic data), can be transmitted to the client light field display systems 500A and 500B in an encoded vectorized format. Thus, given the corresponding hardware configuration of the client light field display system 500A or 500B, the light field processing engine of each client light field display system (e.g., light field processing engine 530) can decode the encoded data to be rendered on the light field display system. For example, a first-client light field display system 500A may have a first hardware configuration, and a second-client light field display system 500B may have a second hardware configuration. The first-client light field display system 500A may receive the same holographic content as the second-client light field display system. Despite the differences between the first and second hardware configurations, the light field processing engines of each light field display system 500A and 500B must render the holographic content, potentially at different resolutions, with different fields of view, and so on.

[0189] Digital signage applications

[0190] Figure 6 FIG6 is an illustration of a large-scale light field display system 600 for use in a digital signage content distribution system according to one or more embodiments. Light field display system 600 is an embodiment of light field display system 500 and client light field display systems 500A and 500B. Light field display system 600 includes a light field display module 620 of a light field display assembly (an embodiment of light field display assembly 510) that provides holographic content to one or more viewers of light field display system 600. Figure 6 The light field display system 600 shown in FIG. 6 forms a single-sided seamless surface environment; however, in other embodiments, the light field display system 600 may form a multi-sided seamless surface environment. Figure 6In the illustration of FIG, the light field display system is sized as a billboard suspended above the ground by multiple support structures. The light field display system 600 can be implemented as part of a digital signage content distribution system (e.g., digital signage content distribution system 560), which controls the distribution of holographic content from a content provider of an online system (e.g., online system 590) or the distribution of holographic content from a third-party system (e.g., third-party system 580). In one embodiment, the content provider can be a third-party system (e.g., an advertising system that generates its own advertisements, also referred to as an advertiser or an advertisement distribution system that generates advertisements on behalf of one or more advertisers). The third-party system can be connected to the light field display system 600 via a network and configured to provide holographic content to the light field display system 600 for presentation to viewers.

[0191] The light field display system 600 presents holographic content to one or more viewers of the light field display system 600. The light field display system 600 can retrieve the holographic content from a data store (e.g., data store 522) or from a content provider. Examples of holographic content include advertisements (e.g., promotions for upcoming sales), announcements (e.g., political speeches), public service alerts (e.g., tornado warnings), news information (e.g., headlines), weather information (e.g., local weather forecasts), venue information (e.g., ticket office hours), information about traffic or travel conditions (e.g., traffic reports), information about business entities (e.g., office directories), performances (e.g., concerts), artistic content (e.g., sculptures), any other holographic content, or any combination thereof. The light field display system 600 (e.g., via the light field processing engine 530) generates display instructions for the holographic content, and more specifically, the holographic content to be presented by the light field display module 620. The light field display system 600 can present holographic content for all viewers within the viewing volume of the light field display system 600, with the viewers being able to see the holographic content unobstructed. In other cases, the light field display system 600 tracks at least some of the viewers within the viewing volume, and in some cases tracks all of the viewers. In some embodiments, the light field display system 600 can provide each tracked viewer with specific holographic content that can be viewed by that viewer, and the specific holographic content will not be visible to other viewers. In other cases, the light field display system 600 can subdivide the viewing volume into different segments and provide different holographic content for each segment. In this way, viewers in different segments are presented with different holographic content and experience holographic content specific to their segment. The light field display system 600 may also include a sensory feedback system (e.g., sensory feedback system 540) for presenting sensory feedback along with the holographic content provided by the light field display module 620. When holographic content is provided to a viewer, the light field display system 600 can record the viewer's behavior in response to the presented holographic content (e.g., via the light field display assembly 510, the sensory feedback system 540, or the tracking system 550). For example, the light field display system 600 can utilize a tracking system (e.g., the tracking system 550) to monitor the viewer's body movements in response to the presentation of the holographic content.

[0192] In some embodiments, the light field display system 600 presents specific holographic content to different types of viewers of the light field display system 600. A tracking system (e.g., tracking system 550) can track the movement of the viewer within the viewing volume of the light field display system 600 (e.g., the viewer moves from the left side of the light field display system to the right side of the light field display system), can monitor the viewer's response to the holographic content (e.g., the viewer is laughing), and can determine the determination of the viewer's presence within the viewing volume of the light field display system (e.g., the viewer is female).

[0193] Tracking the movement of viewers by the tracking system can further determine the velocity based on each viewer's movement. For example, the tracked movement can include the speed at which the viewer moves within the viewing volume. The light field display system can update the presentation of the holographic content based on the viewer's velocity. Figure 6 In the example of , the tracking system may determine that pedestrian 630 is a viewer that is moving slowly (e.g., slower than five miles per hour). The tracking system may then classify pedestrian 630 as a pedestrian. Based on the viewer type determined to be a pedestrian, the light field display system 600 may present a holographic object 635 that is assigned for presentation to the pedestrian type. The light field display system 600 may designate each holographic object in a data storage area (e.g., data storage area 522) for presentation to each type of viewer. The light field display system 600 presents holographic content at a location in the holographic object volume in front of the viewer based on the viewer's speed. For example, based on the position and movement of pedestrian 630 tracked by the tracking system, the light field display system 600 may update the presentation of holographic object 635 to always appear in front of pedestrian 630 while potentially walking or moving around the viewing volume.

[0194] The tracking system may classify the vehicle occupant as a vehicle occupant 640. The tracking system may determine that the vehicle occupant 640 is a different type of viewer than the pedestrian 630. The tracking system may determine that the vehicle occupant 640 is within a vehicle. Additionally, the tracking system may determine that the vehicle occupant 640 is traveling at a faster speed than the pedestrian 630 (e.g., greater than five miles per hour). The light field display system 600 determines a holographic object 645 to provide to the vehicle occupant 640 assigned to the vehicle occupant type. For example, the light field display system 600 may specify less cluttered holographic content (e.g., the name and location of a nearby restaurant, a movie poster for an upcoming or previously released movie) to provide to the vehicle occupant to avoid overly distracting the vehicle driver. The light field display system 600 may also update the presentation of the holographic object 645 so that the vehicle occupant 640 is visible throughout the movement within the viewing volume. In other embodiments, after determining that vehicle occupant 640 is a viewer of the vehicle passenger type, the light field display system 600 presents holographic object 645 in a similar manner at a fixed location in the viewing volume to avoid overly distracting the vehicle driver. In some embodiments, the tracking system can further distinguish between the driver of the vehicle and the passenger of the vehicle by determining the viewer's location within the vehicle to (at least in part) determine whether the viewer is a driver or a passenger. Once distinguished, the light field display system 600 can present holographic content designated for the vehicle driver type, which may include less distracting holographic content (e.g., more static, less cluttered, at a fixed location in the viewing volume), while providing holographic content for other occupants of the vehicle passenger type that can be presented to the vehicle occupant.

[0195] The tracking system can monitor the viewer's response to the holographic content. The monitored response can include the viewer's position in the viewing volume, the viewer's movement, the viewer's gesture, the viewer's facial expression, the viewer's gaze, some other monitored response, or some combination thereof. The light field display system 600 can update the holographic content presented to the viewer in response to the monitored viewer's response. For example, in Figure 6 In the example, a viewer (i.e., pedestrian 630) turns their gaze from the right side of the light field display module 620 to the left side of the light field display module 620. The tracking system monitors the viewer's gaze and updates the holographic content accordingly (i.e., presents the holographic object 635 at a location in the holographic object volume that intersects the viewer's gaze). The monitored viewer's response can be shared with devices on the network. For example, the monitored response can be shared with devices (e.g., computers, servers, etc.) on the network (e.g., network 570).

[0196] The tracking system may determine characteristics of a viewer within the viewing volume of the light field display system 600. The determined viewer characteristics describe characteristics or qualities of the viewer. The determined characteristics may include the viewer's demographic information, work experience, educational history, gender, income, amount of money spent on purchases, hobbies, location, age, viewing history, time spent on items, categories of previously viewed items, purchase history, some other characteristic or quality of the viewer, or some combination thereof. The light field display system 600 may update the holographic content presented to the viewer in response to the determined viewer characteristics. For example, in Figure 6 In the example, the viewer (i.e., pedestrian 630) is determined to be a middle school boy. The holographic content is updated accordingly (i.e., holographic object 635 is changed from the currently displayed diamond ring to an action figure). The determined viewer characteristics can be shared with devices on the network. For example, the monitored responses can be shared with devices (e.g., computers, servers, etc.) on the network (e.g., network 570).

[0197] In some embodiments, the holographic objects 635 and / or 645 may be holographic advertisements presented to viewers (i.e., pedestrians 630 and vehicle passengers 640). Thus, the light field display system 600 records instances of presenting holographic advertisements to viewers. Holographic advertisements may be used for digital assets or physical goods and services, such as holographic movies, holographic shows, holographic merchandise, and the like. Figure 6 In the illustration of FIG, holographic object 635, which is an example of a holographic advertisement, is a diamond ring that is a holographic advertisement for a jewelry store. In other embodiments, holographic objects 635 and / or 645 may be information panels that provide information to their respective viewers. The information may include, but is not limited to, a welcome sign, the name of the establishment, a map with directions, the location of various things (e.g., objects, stores, restaurants, and bathrooms), and the like. Figure 6 In the illustration of , holographic object 645 is an example of a billboard lightfield display surface for the City of Las Vegas Welcome Sign.

[0198] In some embodiments, the light field display system 600 determines the classification of each segment of the viewing volume. Figure 6In the illustration, light field display system 600 is positioned above sidewalk 650, allowing viewers to pass beneath light field display module 620. Light field display system 600 is also positioned proximate to roadway 660, where motor vehicles can pass closely alongside light field display system 600. In one or more embodiments, light field display system 600 implements an AI model to classify various segments of a viewing volume. A tracking system can provide tracking information for various objects and viewers within the viewing volume. Based on the tracking information, light field display system 600 can determine that one segment of the viewing volume is sidewalk 650, where the viewer is likely a pedestrian or cyclist and is likely not among many motor vehicles. Light field display system 600 can then determine that another segment of the viewing volume is roadway 660, where the viewer is likely a passenger in a motor vehicle. Based on the classified segments of the viewing volume, light field display system 600 can provide holographic content based on those segments. For example, light field display system 600 may provide a holographic sign fixed at a location visible to viewers in lane 660 (ie, drivers or passengers in vehicles), while providing specific holographic advertisements to each viewer moving within sidewalk 650 .

[0199] Figure 7 FIG7 is an illustration of a lightfield display system 700 for use in a small signage system (e.g., a panel lightfield display surface) according to one or more embodiments. Lightfield display system 700 is an embodiment of lightfield display system 500. Lightfield display system 700 contains a lightfield display module 720 of a lightfield display assembly (an embodiment of lightfield display assembly 510) that provides holographic content to one or more viewers of lightfield display system 700. Figure 7 The light field display system 700 shown in FIG. 7 forms a single-sided seamless surface environment; however, in other embodiments, the light field display system 700 can form a multi-sided seamless surface environment. Figure 7 In the illustration of , the light field display system is similar in size to an average viewer. The light field display system 700 can be implemented as part of a light field digital signage environment that controls the distribution of holographic content from a content provider.

[0200] The light field display system 700 identifies a viewer 730 of the light field display system 700. The light field display system 700 uses a viewer profiling module (e.g., viewer profiling system 528) to identify the viewer 730. In some embodiments, the viewer profiling module uses image recognition technology with captured image data of the viewer 730. In other embodiments, the viewer profiling module further employs an identification scanner that can scan a physical identification token of the viewer 730. For example, the viewer 730 may be at an amusement park with an RFID wristband that can be scanned by one or more RFID scanners of the viewer profiling module. The light field display system 700 may then access the viewer's viewer profile in a data store (e.g., data store 522). The viewer profile may include various characteristics or information about the viewer. Characteristics may include preferences provided or inferred from monitored behavior. The information may further include a historical log, such as holographic content previously presented to the viewer 730. The light field display system 700 may then select a holographic object 725 to present to the viewer 730 .

[0201] A tracking system (e.g., tracking system 550) tracks viewer 730 while providing holographic content to viewer 730. The tracking system may continuously monitor the position of viewer 730, the gaze of viewer 730, or a combination thereof. The light field display system 700 may update the presentation of the holographic content based on the tracking information. For example, the light field display system 700 may determine that viewer 730 is moving from one side of the light field display system 700 to the other. In response to the determined movement of viewer 730, the light field display system 700 may update the presentation of the holographic object 735 to follow the viewer 730 as they move from side to side. In another example, viewer 730 may be standing approximately perpendicular to the light field display module 720. As viewer 730 shifts their gaze, the tracking system may monitor the viewer's response and record the gaze. Based on the recorded gaze, the light field display system 700 can update the presentation of the holographic object 735 to follow the gaze of the viewer 730 , eg, presenting modified holographic content at a location in the holographic object volume that intersects the gaze of the viewer 730 .

[0202] In one or more embodiments, the light field display system 700 includes a sensory feedback system (e.g., sensory feedback system 540) to provide sensory feedback with holographic content. The sensory feedback system can generate sensory feedback based on parameters stored in a data storage area (e.g., data storage area 522). In some cases, each holographic object can have parameters that indicate the presentation of sensory feedback and the holographic object presented by the light field display module 720. For example, when presenting a holographic chicken, the sensory feedback system can receive instructions including sensory feedback parameters to present a tactile texture of feathers near the location of the holographic chicken in the viewing volume. In addition, the sensory feedback system can provide audio feedback corresponding to the chicken's crow (e.g., "ba ba ba ba"). The viewer 730 can then perceive sensory feedback that provides a more immersive experience and interaction with the holographic object 735.

[0203] In some embodiments, the light field display system 700 is configured to receive input from a viewer 730. The input may be received via any combination of the light field display module 720, a sensory feedback system (e.g., sensory feedback system 540), a tracking system (e.g., tracking system 550), a viewer profiling module (e.g., viewer profiling system 528), and any additional input devices. The received input may include, but is not limited to, audio input (e.g., voice input from the viewer), tracking input (e.g., gestures of the viewer tracked by the tracking system), button input (e.g., pressing a button on a remote control or touchscreen display, etc.), or any combination thereof. In response to the received input, the light field display system 700 may update the presentation of holographic content. In some cases, the light field display system 700 may associate some input with modifying the display of holographic content presented by the light field display system 700, including holographic content presented by the light field display module 720 or sensory feedback content provided by the sensory feedback assembly.

[0204] In embodiments where light field display system 700 presents a holographic advertisement, light field display system 700 may receive input to transition the presentation of the holographic advertisement. While presenting the holographic advertisement, light field display system 700 may also present options to viewer 730 as transition opportunities. Viewer 730 may provide input to light field display system 700. For example, if viewer 730 wishes to purchase holographic object 735, light field display system 700 may prompt viewer 730 to raise their hand. A tracking system may track viewer 730 and determine that viewer 730 has raised their hand. The raised hand is received as input by tracking system 730. In response, light field display system 700 presents additional holographic content to facilitate the transition of the holographic advertisement. For example, light field display system 700 may present a holographic webpage to allow viewer 730 to complete their purchase of holographic object 735.

[0205] In one or more embodiments, holographic object 735 is a holographic advertisement presented to viewer 730. Thus, lightfield display system 700 records instances of presenting a holographic advertisement to viewer 730. Holographic advertisements can be used for digital assets or physical goods and services, holographic movies, holographic shows, other holographic content, and the like. In other embodiments, holographic object 735 can be an information panel that provides information to its corresponding viewer. This information can include, but is not limited to, a welcome sign, an establishment name, a map with directions, the locations of various items (e.g., objects, stores, restaurants, and bathrooms), and the like.

[0206] Additional configuration information

[0207] The foregoing description of the embodiments of the present disclosure is presented for illustrative purposes; it is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Those skilled in the relevant art will appreciate that many modifications and variations are possible based on the above disclosure.

[0208] Some parts of this description describe embodiments of the present disclosure in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are typically used by those skilled in the art of data processing to effectively convey the essence of their work to other persons skilled in the art. When functionally, computationally, or logically described, these operations are understood to be implemented by computer programs or equivalent circuits, microcode, etc. In addition, without loss of generality, it has been demonstrated that it is sometimes convenient to refer to these operational arrangements as modules. The described operations and their associated modules can be embodied in software, firmware, hardware, or any combination thereof.

[0209] Any of the steps, operations, or processes described herein may be performed or implemented using one or more hardware or software modules alone or in combination with other devices. In some embodiments, the software modules are implemented as a computer program product comprising a computer-readable medium containing computer program code that can be executed by a computer processor to perform any or all of the steps, operations, or processes described.

[0210] Embodiments of the present disclosure may also relate to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the desired purpose, and / or it may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a non-transitory tangible computer-readable storage medium or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing system referred to in the specification may include a single processor, or may be an architecture that employs a multi-processor design to obtain increased computing power.

[0211] Embodiments of the present disclosure may also relate to products produced by the computing processes described herein. Such products may include information resulting from the computing processes, wherein the information is stored on a non-transitory tangible computer-readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

[0212] Finally, the language used in the specification is primarily selected for readability and instructional purposes and may not be selected to describe or limit the subject matter of the present invention. Accordingly, it is intended that the scope of the present disclosure be limited not by this detailed description, but rather by any claims based upon the application appended hereto. Accordingly, the disclosure of the embodiments is intended to illustrate, but not to limit, the scope of the present disclosure as set forth in the following claims.

Claims

1. A light field display system, comprising: a controller configured to generate holographic content; as well as a light field display assembly comprising one or more light field display modules configured to present the holographic content in a holographic object volume to a viewer in a viewing volume located in a public environment; Each of the one or more light field display modules comprises: an energy device layer configured to provide a plurality of energy source locations; an energy waveguide layer having a plurality of energy waveguides, wherein each waveguide is configured to project energy from at least one energy source position from a display surface into at least one specific direction dependent on the energy source position according to a four-dimensional light field function to form the holographic object; and wherein a viewing angle of the holographic object changes based in part on a position of a viewer in the viewing volume relative to the holographic object within the holographic object volume; The one or more light field display modules are further configured to present additional holographic content in the holographic object volume to a second viewer located in a second viewing volume different from the viewing volume; wherein the holographic content is viewable from the viewing volume but not from the second viewing volume.

2. The light field display system of claim 1 , wherein the holographic content comprises one or more of the following: advertise; notice; Public Service Alerts; News information; Weather information; Location information; information about traffic or travel conditions; information about business entities; Broadcasting of events; performances; and Artistic content.

3. The light field display system of claim 1 , further comprising a tracking system configured to track movement of a viewer, wherein The viewing volume and the second viewing volume are defined based on the tracked movement of the viewer.

4. The light field display system of claim 1 , further comprising a tracking system configured to perform one or more of the following: tracking movement of the viewer within the viewing volume of the light field display system; monitoring a response of the viewer to the holographic content; and Characteristics of the viewer within the viewing volume of the lightfield display system are determined. 5 . The light field display system of claim 4 , wherein the tracking system comprises one or more cameras configured to capture images of an area in front of the one or more light field display modules.

6. The light field display system of claim 5, wherein the one or more cameras are external to the light field display assembly. 7 . The light field display system of claim 4 , wherein the one or more light field display modules are further configured to capture a light field from an area in front of the one or more light field display modules.

8. The light field display system of claim 7, wherein the light field display assembly is further configured to receive input based on the light field from the area in front of the one or more light field display modules captured by the one or more light field display modules.

9. The light field display system of claim 4, wherein the tracking system comprises one or more depth sensors configured to detect the depth of an object in front of the one or more light field display modules.

10. The light field display system of claim 4, wherein the tracked movement comprises a speed at which the viewer moves within the viewing volume, and wherein the light field display assembly is configured to update the presentation of the holographic content based on the speed of the viewer.

11. The light field display system of claim 10, wherein the holographic content is presented at a location in the holographic object volume in front of the viewer based on the speed of the viewer.

12. The light field display system of claim 4, wherein the monitored viewer response comprises one or more of: The position of the viewer in the viewing volume; Movement of said viewer; the gestures of said viewer; the viewer's facial expression; and The gaze of the viewer.

13. The light field display system of claim 12, wherein the light field display assembly is further configured to update the presentation of the holographic content in response to the monitored response of the viewer.

14. The light field display system of claim 12, wherein the monitored response of the viewer is shared with devices on a network.

15. The light field display system of claim 4, wherein the monitored response of the viewer includes a gaze of the viewer, and wherein the light field display assembly is configured to present the holographic content based on the gaze of the viewer.

16. The light field display system of claim 15, wherein the holographic content is presented at a location in the holographic object volume that intersects the gaze of the viewer.

17. The light field display system of claim 15, wherein the controller is further configured to determine an impression occurrence based in part on the gaze of the viewer.

18. The light field display system of claim 4, wherein the determined characteristics of the viewer include one or more of the following: demographic information, work experience, educational history, gender, income, amount spent on purchases, hobbies, location, age, viewing history, time spent on items, categories of previously viewed items, and purchase history of the viewer.

19. The light field display system of claim 18, wherein the light field display assembly is further configured to update a presentation of the holographic content in response to the determined characteristics of the viewer.

20. The light field display system of claim 18, wherein the determined characteristics of the viewer are shared with devices on a network.

21. The light field display system of claim 1 , further comprising: A viewer profile forming module is configured to generate a viewer profile comprising characteristics of the viewer.

22. The light field display system of claim 21, wherein the light field display assembly is further configured to update the presentation of the holographic content in response to the characteristic corresponding to a viewer profile of the viewer.

23. The light field display system of claim 1, wherein the light field display system is further configured to receive input in response to the presentation of the holographic content.

24. The light field display system of claim 23, wherein the light field display assembly is further configured to update the presentation of the holographic content in response to received input.

25. The light field display system of claim 23, wherein the input corresponds to a transformation associated with the holographic content.

26. The light field display system of claim 25, wherein the light field display assembly renders subsequent holographic content based on the received input to complete the transformation.

27. The light field display system of claim 26, wherein the conversion is a purchase.

28. The light field display system of claim 25, wherein the holographic content is a holographic advertisement for one or more of the following: physical goods; digital assets; and Services to be provided.

29. The light field display system according to claim 1, wherein the energy waveguide layer further comprises: an electrostatic speaker array coupled to the plurality of energy waveguides, the electrostatic speaker array comprising: at least one transparent membrane configured to generate acoustic energy when driven; and A plurality of electrodes are configured to acoustically drive the transparent membrane, each electrode of the plurality of electrodes being positioned between one or more energy waveguides of the plurality of energy waveguides.

30. The light field display system of claim 1, wherein each of the one or more light field display modules has a display surface from which a holographic object is projected, wherein a seamless display surface is formed by tiling the display surfaces of the one or more light field display modules.

31. The light field display system of claim 30, wherein a surface area of the seamless display surface is greater than a surface area of the display surface of a single light field display module.

32. The light field display system of claim 1, further comprising: A sensory feedback system includes one or more sensory feedback devices and is configured to provide sensory feedback simultaneously with the holographic object.

33. The light field display system of claim 32, wherein the sensory feedback comprises tactile feedback, audio feedback, fragrance feedback, temperature feedback, or any combination thereof.

34. The light field display system of claim 32, wherein the sensory feedback system comprises an ultrasonic energy projection device for providing tactile feedback through the holographic object, wherein the ultrasonic energy projection device is configured to generate a three-dimensional tactile surface close to or coincident with a surface of the holographic object.

35. The light field display system of claim 34, wherein the holographic content is a representation of a physical item having a texture, wherein the stereoscopic haptic surface simulates the texture of the physical item.

36. The light field display system of claim 34, wherein a stereoscopic tactile projection device is part of the light field display assembly.

37. The light field display system of claim 1, wherein the light field display system is a component of a light field digital signage environment, the light field digital signage environment comprising: A signage control system is connected to the light field display system via a network and is configured to provide the holographic content to the light field display system for presentation to the viewer.

38. The light field display system of claim 37, wherein the holographic content comprises one or more of: advertise; notice; Public Service Alerts; News information; Weather information; Location information; information about traffic or travel conditions; information about business entities; Broadcasting of events; performances; and Artistic content.

39. The light field display system of claim 37, wherein the light field display system is configured to receive the holographic content in an encoded format via the network, and further configured to decode the holographic content into a format for presentation to the viewer.

40. The light field display system of claim 39, wherein the encoding format is a vectorized format and the decoding format is a rasterized format.

41. The light field display system of claim 1, wherein the holographic content is presented based on a hardware configuration of the light field display system.

42. The light field display system of claim 41 , wherein the hardware configuration comprises one or more of the following: resolution; The number of projected rays per degree; Field of view; a deflection angle on the display surface; and The dimensions of the display surface.

43. A method using the light field display system according to any one of claims 1 to 42, comprising: Tracking the movement of a viewer within a viewing volume of a light field display system; determining that the viewer is within the viewing volume based in part on the tracked movement; as well as Based in part on the determination that the viewer is within the viewing volume, holographic content in a holographic object volume is presented to the viewer via one or more light field display modules of the light field display system.

44. The method of claim 43, further comprising: Based on the determination that the viewer is within the viewing volume, identifying an opportunity to present holographic content to the viewer of the light field display system; selecting the holographic content from a content storage area, wherein the holographic content includes a set of parameters; generating display instructions for the holographic content based on the set of parameters; and Wherein presenting the holographic content to the viewer is further based on the set of parameters.

45. The method of claim 43, further comprising one or more of the following: tracking movement of the viewer within the viewing volume; monitoring a response of the viewer to the holographic content; and Characteristics of the viewer within the view are determined.

46. The method of claim 45, further comprising: An occurrence of an impression is determined based on the monitored response of the viewer.

47. The method of claim 45, further comprising: The presentation of the holographic content is updated in response to one or more of the tracked movement of the viewer, the monitored response of the viewer, and the determined characteristic of the viewer.

48. The method of claim 43, further comprising: Input is received from the viewer in response to the presentation of the holographic content.

49. The method of claim 45, wherein monitoring the response of the viewer comprises recording the viewer's gaze, the method further comprising: The modified holographic content is presented at a location in the holographic object volume that intersects the gaze of the viewer.

50. The method of claim 49, further comprising: Whether an impression occurs is determined based in part on the gaze of the viewer.

51. The method of claim 43, further comprising: receiving an input corresponding to a transformation associated with the holographic content; as well as Subsequent holographic content is rendered based on the received input to complete the transition.

52. A light field (light field) digital signage system comprising: One or more client light field display systems, each client light field display system comprising: a controller configured to receive holographic content from an online system via a network, the online system configured to regulate distribution of the holographic content; and a light field display assembly comprising one or more light field display modules configured to display the holographic content as digital signage in a holographic object volume to a viewer located in a viewing volume; Each of the one or more light field display modules comprises: an energy device layer configured to provide a plurality of energy source locations; an energy waveguide layer having a plurality of energy waveguides, wherein each waveguide is configured to project energy from at least one energy source position from a display surface into at least one specific direction dependent on the energy source position according to a four-dimensional light field function to form the holographic object; and wherein a viewing angle of the holographic object changes based in part on a position of a viewer in the viewing volume relative to the holographic object within the holographic object volume; The one or more light field display modules are further configured to present additional holographic content in the holographic object volume to a second viewer located in a second viewing volume different from the viewing volume; wherein the holographic content is viewable from the viewing volume but not from the second viewing volume.

53. The light field digital signage system of claim 52, wherein the one or more client light field display systems further comprise a tracking system configured to perform one or more of the following: monitoring a response of the viewer to the holographic content; and Characteristics of the viewer within the viewing volume are determined.

54. The light field digital signage system of claim 53, wherein: The monitored responses of the viewer include one or more of the following: The position of the viewer in the viewing volume; Movement of said viewer; the gestures of said viewer; the viewer's facial expression; and The gaze of the viewer.

55. The light field digital signage system of claim 53, wherein: The lightfield display assembly is further configured to update the presentation of the holographic content in response to the monitored response of the viewer.

56. The light field digital signage system of claim 53, wherein: The determined characteristics of the viewer include one or more of the following: the viewer's demographic information, work experience, educational history, gender, income, amount spent on purchases, hobbies, location, age, viewing history, time spent on items, categories of previously viewed items, and purchase history.

57. The light field digital signage system of claim 53, wherein the light field display assembly is further configured to update the presentation of the holographic content in response to the determined characteristics of the viewer.

58. The light field digital signage system of claim 53, further comprising: A viewer profile forming module is configured to generate a viewer profile comprising characteristics of the viewer.

59. The light field digital signage system of claim 52, wherein the one or more client light field display systems receive holographic content in exchange for payment.

60. The light field digital signage system of claim 52, wherein the holographic content is rendered based on a hardware configuration of the client light field display system.

61. The light field digital signage system of claim 60, wherein the hardware configuration comprises one or more of the following: resolution; The number of projected rays per degree; Field of view; a deflection angle on the display surface; and The dimensions of the display surface.

Images