Light field display system for performance events

Abstract

A light field (LF) display system for displaying holographic performance content (e.g., a live performance) to viewers in a venue. The LF display system in the venue includes LF display modules tiled together to form an LF module array. The LF module array creates a performance volume (e.g., a stage) for displaying the performance content in the venue. The LF module array displays the performance content to viewers in a viewing volume. The LF display system may be included in an LF rendering network. The LF rendering network allows holographic performance content to be recorded at one location and displayed at another location (simultaneously or non-simultaneously). The LF rendering network includes a network system that manages digital rights to the holographic performance content.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is related to International Application Nos. PCT / US2017 / 042275, PCT / US2017 / 042276, PCT / US2017 / 042418, PCT / US2017 / 042452, PCT / US2017 / 042462, PCT / US2017 / 042466, PCT / US2017 / 042467, PCT / US2017 / 042468, PCT / US2017 / 042469, PCT / US2017 / 042470, and PCT / US2017 / 042679, all of which are incorporated herein by reference in their entirety. Background Art

[0003] The present disclosure relates to performances in performance venues, and in particular, to light field display systems for displaying performances in performance venues.

[0004] Traditionally, performance venues (e.g., theaters, concert halls, etc.) are configured to allow viewers (e.g., fans, patrons, etc.) to watch a live performance (e.g., a concert, a play, a comedy show, etc.) by a performer (e.g., a band, a theater group, a comedian, etc.) at the venue in real time. Unfortunately, in some cases, hosting a performance at a venue can limit the ability of viewers who want to do so to view the performance. For example, the performance may be sold out, may be at an inconvenient time, or may be far away from the viewer. Sometimes performances are recorded and later reproduced on a two-dimensional surface such as a movie screen or television, but these reproductions rarely reproduce the atmosphere and excitement of the venue where the live performance was performed. Therefore, it would be beneficial to configure the performance venue so that viewers can perceive the performance as if they were at a live performance. Summary of the invention

[0005] A light field (LF) display system for displaying holographic content of a performance in a venue (e.g., a theater, a concert hall, etc.). The LF display system includes LF display modules forming a surface in the venue (e.g., a stage, etc.), the LF display modules each having a display area and being tiled together to form a seamless display surface having an effective display area larger than the display area. The LF display modules display the holographic content of the performance from the performance body so that viewers in the venue can perceive the performance.

[0006] In some embodiments, the holographic content of the performance can be a reproduction of a performance that is simultaneously taking place at another venue, created by a content creation system for display at the venue, and / or accessed from a data storage device for display at the venue. The holographic content can be managed by a network system responsible for managing digital rights to the holographic content. For example, viewers at the venue can pay a transaction fee to access the holographic content for display at the venue.

[0007] In some embodiments, the LF display system includes a tracking system and / or a viewer profiling system. The tracking system and viewer profiling system may monitor and store characteristics of viewers in the venue, viewer profiles describing the viewers, and / or viewer responses to holographic content in the venue. Holographic content created for display in the venue may be based on any of the monitored or stored information.

[0008] In some embodiments, the user may interact with the holographic content, and the interaction may serve as an input to the holographic content creation system. For example, in some embodiments, some or all of the LF display systems include multiple ultrasonic speakers. The multiple ultrasonic speakers are configured to generate a tactile surface consistent with at least a portion of the holographic content. The tracking system is configured to track the user's interaction with the holographic object (e.g., through images captured by the LF display module and / or the imaging sensor of some other camera). And the LF display system is configured to provide creation of holographic content based on the interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a diagram of a light field display module that presents a holographic object according to one or more embodiments.

[0010] Figure 2A is a cross-section of a portion of a light field display module according to one or more embodiments.

[0011] Figure 2B is a cross-section of a portion of a light field display module according to one or more embodiments.

[0012] Figure 3A is a perspective view of a light field display module according to one or more embodiments.

[0013] Figure 3B is a cross-sectional view of a light field display module including interwoven energy relay devices according to one or more embodiments.

[0014] Figure 4A is a perspective view of a portion of a light field display system tiled in two dimensions to form a single-sided seamless surface environment in accordance with one or more embodiments.

[0015] Figure 4Bis a perspective view of a portion of a light field display system in a multi-faceted seamless surface environment according to one or more embodiments.

[0016] Figure 4C is a top view of a light field display system with a converging surface in a wing-like configuration according to one or more embodiments.

[0017] Figure 4D is a side view of a light field display system with a converging surface in a tilted configuration according to one or more embodiments.

[0018] Figure 4E is a top view of a lightfield display system having a converging surface on the front wall of a room according to one or more embodiments.

[0019] Figure 4F is a side view of a side view of a LF display system having a polymeric surface on the front wall of a room according to one or more embodiments.

[0020] Figure 5A is a block diagram of a light field display system according to one or more embodiments.

[0021] Figure 5B An example LF movie network 550 is shown in accordance with one or more embodiments.

[0022] Figure 6 Shown is a side view of a venue 600 that is a conventional theater that has been enhanced with a LF display system in accordance with one or more embodiments.

[0023] Fig. 7A A cross-section of a first venue including a LF display system for displaying performance content to viewers at viewing positions in a viewing volume in accordance with one or more embodiments is shown.

[0024] Figure 7B A cross-section of a second venue including a LF display system for displaying performance content to viewers at viewing positions in a viewing volume in accordance with one or more embodiments is shown.

[0025] Figure 8 A venue is shown in a viewer's living room that also functions as a home theater in accordance with one or more embodiments.

[0026] Fig. 9 is a flow chart illustrating a method for displaying a hologram of a performance within a LF performance network.

[0027] The accompanying drawings depict various embodiments of the present invention for purposes of illustration only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles of the present invention described herein. DETAILED DESCRIPTION

[0028] Overview

[0029] A light field (LF) display system is implemented in a venue that is set up as a performance. For example, the performance may be a concert, a play, a comedy show, a dance performance, etc. The LF display system includes an LF display component that is configured to present holographic content including one or more holographic objects that will be visible to one or more users in the viewing volume of the LF display system. The LF display component may form a multi-faceted seamless surface on some or all of one or more surfaces (e.g., a stage) in the venue. The LF display system may present the holographic content to viewers in the venue. Viewers typically watch performances in the venue, but may be anyone in the venue who is able to view the holographic content in the venue. The holographic objects may also be enhanced with other sensory stimuli (e.g., tactile and / or audio). For example, an ultrasonic transmitter in the LF display system may emit ultrasonic pressure waves that provide a tactile surface for some or all of the holographic objects. The holographic content may include additional visual content (i.e., 2D or 3D visual content). Coordinating the transmitters to ensure that a cohesive experience is achieved is part of the system in a multi-transmitter implementation (i.e., a holographic object that provides the correct tactile sensation and sensory stimulation at any given point in time).

[0030] In some embodiments, the LF display system includes a plurality of LF display modules that form a stage in a venue. The LF display modules that form the stage may be configured to project holographic content of the performance to viewers in the venue. In this way, viewers in the venue may perceive the holographic performance on the stage. For example, the LF display system may display a band giving a concert, a dance company premiering a ballet, or a theater company performing a play. In some embodiments, the LF display system may create holographic content for display to viewers in the venue. For example, the LF display system may create a virtual performer to perform a song created for the virtual performer to viewers in the venue.

[0031] In some embodiments, the LF display system may include elements that enable the system to simultaneously emit at least one type of energy, and simultaneously absorb at least one type of energy for the purpose of responsively creating holographic content. For example, the LF display system may emit both holographic objects for viewing and ultrasound for tactile perception, and simultaneously absorb imaging information for tracking the viewer and other scene analysis, while also absorbing ultrasound to detect the user's touch response. As an example, such a system may project a holographic performer that surfs through a crowd, giving the viewer the illusion that the performer is on their hand when the viewer virtually "touches" the performer. The display system components that perform ambient energy sensing may be integrated into the display surface via bidirectional energy elements that simultaneously emit and absorb energy, or the display system components may be dedicated sensors separate from the display surface, such as ultrasonic speakers and imaging capture devices such as cameras.

[0032] The LF display system may be part of a LF performance network. The LF performance network allows LF data to be recorded in one location, encoded, transmitted to a different location, decoded, and displayed as holographic content to viewers in a venue. This allows viewers at multiple venues to perceive live performances occurring at other venues. In some embodiments, the LF display system includes a network system that manages digital rights to holographic content.

[0033] Light Field Display System

[0034] Figure 1 is a light field (LF) display module 110 for presenting a holographic object 120 according to one or more embodiments. Figure 1 . The LF display module 110 is part of a light field (LF) display system. The LF display system uses one or more LF display modules to present holographic content including at least one holographic object. The LF display system can present the holographic content to one or more viewers. In some embodiments, the LF display system can also enhance the holographic content with other sensory content (e.g., touch, audio, smell, temperature, etc.). For example, as discussed below, the projection of focused ultrasound can generate an air-tactile sensation that can simulate the surface of some or all of the holographic object. The LF display system includes one or more LF display modules 110, and the following description of the LF display system is provided below. Figure 2A , Figure 2B , Figure 3A , Figure 3B , Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E , Figure 4F , Figure 5A and Figure 5B Discuss in detail.

[0035] The LF display module 110 is a holographic display that presents a holographic object (e.g., holographic object 120) to one or more viewers (e.g., viewer 140). The LF display module 110 includes an energy device layer (e.g., an emissive electronic display or an acoustic projection device) and an energy waveguide layer (e.g., an optical lens array). In addition, the LF display module 110 may include an energy relay layer for combining multiple energy sources or detectors together to form a single surface. At a high level, the energy device layer generates energy (e.g., holographic content) and then uses the energy waveguide layer to guide the energy to an area in space according to one or more four-dimensional (4D) light field functions. The LF display module 110 can also project and / or sense one or more types of energy simultaneously. For example, the LF display module 110 may be able to project a holographic image and an ultrasonic tactile surface in a viewing volume while detecting imaging data from the viewing volume. The holographic image between the display area 150 of the LF display module 110 and the viewer 140 is a real image, in which the collection of focal points forming the image are converging line rays. In addition, some images generated by the LF display may be virtual images (eg, images displayed behind the display area 150) where a set of focal points are formed by extension of divergent light rays. The operation of the LF display module 110 is described below with respect to Figure 2A , Figure 2B , Figure 3A and Figure 3B Discuss in detail.

[0036] The LF display module 110 uses one or more 4D light field functions (e.g., derived from a plenoptic function) to generate a holographic object within a holographic object volume 160. The holographic object may be three-dimensional (3D), two-dimensional (2D), or some combination thereof. In addition, the holographic object may be multi-color (e.g., full color). The holographic object may be projected in front of a screen plane, behind a screen plane, or separated by a screen plane. The holographic object 120 may be presented so that the holographic object may be perceived anywhere within the holographic object volume 160. Here, the holographic object 120 is a real image 122 when light rays converge, and is presented between the display area 150 and the viewer 140. The holographic object within the holographic object volume 160 may appear to be floating in space to the viewer 140.

[0037] The holographic object volume 160 represents a volume in which the viewer 140 can perceive a holographic object. The holographic object volume 160 can extend in front of the surface of the display area 150 (i.e., toward the viewer 140) so that the holographic object can appear in front of the plane of the display area 150. In addition, the holographic object volume 160 can extend behind the surface of the display area 150 (i.e., away from the viewer 140), allowing the holographic object to be presented as if it is behind the plane of the display area 150. In other words, the holographic object volume 160 can contain all light rays originating from the display area 150 (e.g., being projected) and can converge to create a holographic object. Herein, the light rays can converge at a point in front of the display surface, at the display surface, or behind the display surface. More simply, the holographic object volume 160 encompasses all volumes from which a viewer can perceive a holographic object.

[0038] The viewing volume 130 is a volume of space from which a holographic object (e.g., holographic object 120) rendered within the holographic object volume 160 by the LF display system is fully visible. A holographic object can be rendered within the holographic object volume 160 and viewed within the viewing volume 130 such that the holographic object is indistinguishable from an actual object. The holographic object is formed by projecting the same light rays that would be generated from the surface of the object if it physically existed.

[0039] In some cases, the holographic object volume 160 and the corresponding viewing volume 130 may be relatively small so that they are designed for a single viewer. In other embodiments, as described below with respect to, for example Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E , Figure 4F , Figure 6 , Fig. 7A , Figure 7B , Figure 8 and Fig. 9 As discussed in detail, LF display modules can be enlarged and / or tiled to create larger holographic object volumes and corresponding viewing volumes that can accommodate a wide range of viewers (e.g., 1 to thousands). The LF display modules presented in the present disclosure can be constructed so that the entire surface of the LF display contains holographic imaging optics, there are no invalid or dead space, and no bezel is required. In these embodiments, the LF display modules can be tiled so that the imaging area is continuous across the seams between the LF display modules, and the join lines between the tiled modules are barely detectable using the visual acuity of the eye. It is worth noting that in some configurations, although not described in detail herein, some portions of the display surface may not contain holographic imaging optics.

[0040] The flexible size and / or shape of the viewing volume 130 allows the viewer to be unconstrained within the viewing volume 130. For example, the viewer 140 can move to different locations within the viewing volume 130 and see different views of the holographic object 120 from corresponding viewing angles. Figure 1 , the viewer 140 is located in a first position relative to the holographic object 120 so that the holographic object 120 appears to be a front view of the dolphin. The viewer 140 can move to other positions relative to the holographic object 120 to see different views of the dolphin. For example, the viewer 140 can move so that he / she sees the left side of the dolphin, the right side of the dolphin, etc., just as if the viewer 140 is watching an actual dolphin and changes his / her relative position to the actual dolphin to see different views of the dolphin. In some embodiments, the holographic object 120 is visible to all viewers within the viewing volume 130, and all viewers have an unobstructed view of the holographic object 120 (i.e., not blocked by objects / people). These viewers can be unconstrained so that they can move around within the viewing volume to see different perspectives of the holographic object 120. Therefore, the LF display system can present the holographic object so that multiple unconstrained viewers can see different perspectives of the holographic object simultaneously in the real world space, as if the holographic object was physically present.

[0041] In contrast, conventional displays (e.g., stereoscopic, virtual reality, augmented reality, or mixed reality) typically require each viewer to wear some kind of external device (e.g., 3-D glasses, near-eye displays, or head-mounted displays) to see the content. Additionally and / or alternatively, conventional displays may require the viewer to be constrained to a specific viewing position (e.g., in a chair with a fixed position relative to the display). For example, when viewing an object shown by a stereoscopic display, the viewer always focuses on the display surface, rather than on the object, and the display will always present only two views of the object, which will follow the viewer attempting to move around the perceived object, resulting in a perceived distortion of the object. However, with a light field display, a viewer of a holographic object presented by an LF display system does not need to wear an external device, nor does he have to be constrained to a specific position to see the holographic object. The LF display system presents the holographic object in a manner visible to the viewer, in much the same way that the viewer can see a physical object, without the need for special goggles, glasses, or head-mounted accessories. Further, the viewer can view the holographic content from any position within the viewing volume.

[0042] It is worth noting that there is a limit to the size of potential position receptors of the holographic object within the holographic object volume 160. In order to increase the size of the holographic object volume 160, the size of the display area 150 of the LF display module 110 can be increased and / or multiple LF display modules can be tiled together in a manner to form a seamless display surface. The effective display area of ​​the seamless display surface is larger than the display area of ​​each LF display module. Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E , Figure 4F , Figure 5A , Figure 5B , Figure 6 , Fig. 7A , Figure 7B , Figure 8 and Fig. 9 Some embodiments related to tiled LF display modules are discussed. Figure 1 As shown, the display area 150 is rectangular, resulting in a pyramidal shape for the holographic object volume 160. In other embodiments, the display area may have some other shape (eg, hexagonal), which also affects the shape of the corresponding viewing volume.

[0043] In addition, although the above discussion focuses on presenting the holographic object 120 within a portion of the holographic object volume 160 between the LF display module 110 and the viewer 140, the LF display module 110 may additionally present content in the holographic object volume 160 behind the plane of the display area 150. For example, the LF display module 110 may make the display area 150 appear to be the surface of the ocean from which the holographic object 120 is jumping out. And the displayed content may enable the viewer 140 to look through the displayed surface to see the marine life underwater. Furthermore, the LF display system may generate content that moves seamlessly around the holographic object volume 160, including behind and in front of the plane of the display area 150.

[0044] Figure 2A A cross section of a portion of an LF display module 210 is illustrated in accordance with one or more embodiments. The LF display module 210 may be the LF display module 110. In other embodiments, the LF display module 210 may be another LF display module having a display area that is shaped differently than the display area 150. In the illustrated embodiment, the LF display module 210 includes an energy device layer 220, an energy relay layer 230, and an energy waveguide layer 240. Some embodiments of the LF display module 210 have components that differ from those described herein. For example, in some embodiments, the LF display module 210 does not include the energy relay layer 230. Similarly, functionality may be allocated between components in a manner different from that described herein.

[0045] The display system described herein presents an energy emission that replicates the energy that normally surrounds an object in the real world. Here, the emitted energy is directed toward a specific direction from each coordinate on the display surface. In other words, each coordinate on the display surface acts as a projection location for the emitted energy. The directed energy from the display surface causes many energy rays to converge, which can thereby create a holographic object. For example, for visible light, the LF display will project a very large number of light rays from the projection position that can converge at any point in the holographic object body, so from the perspective of a viewer positioned farther than the projected object, the light rays appear to come from the surface of a real object positioned in an area of ​​this space. In this way, the LF display generates reflected light rays that leave the surface of this object from the viewer's perspective. The viewer's perspective can change on any given holographic object, and the viewer will see different views of the holographic object.

[0046] As described herein, the energy device layer 220 includes one or more electronic displays (e.g., emissive displays such as OLEDs) and one or more other energy projection and / or energy receiving devices. The one or more electronic displays are configured to display content according to display instructions (e.g., from a controller of the LF display system). The one or more electronic displays include a plurality of pixels, each with an independently controlled intensity. Many types of commercial displays can be used in LF displays, such as emissive LED and OLED displays.

[0047] The energy device layer 220 may also include one or more acoustic projection devices and / or one or more acoustic receiving devices. The acoustic projection device generates one or more pressure waves that complement the holographic object 250. The generated pressure waves may be, for example, audible, ultrasonic, or some combination thereof. An array of ultrasonic pressure waves may be used for volumetric tactile sensations (e.g., at the surface of the holographic object 250). Audible pressure waves are used to provide audio content (e.g., immersive audio) that may complement the holographic object 250. For example, assuming that the holographic object 250 is a dolphin, one or more acoustic projection devices may be used to (1) generate a tactile surface juxtaposed with the surface of the dolphin so that the viewer may touch the holographic object 250; and (2) provide audio content corresponding to the sounds made by the dolphin (e.g., clicks, chirps, or squeaks). An acoustic receiving device (e.g., a microphone or microphone array) may be configured to monitor ultrasonic and / or audible pressure waves within a local area of ​​the LF display module 210.

[0048] The energy device layer 220 may also include one or more imaging sensors. The imaging sensor may be sensitive to light in the visible light band, and in some cases, may be sensitive to light in other bands (e.g., infrared). The imaging sensor may be, for example, a complementary metal oxide semiconductor (CMOS) array, a charge coupled device (CCD), a photodetector array, some other sensor that captures light, or some combination thereof. The LF display system may use the data captured by the one or more imaging sensors to locate and track the position of the viewer.

[0049] In some configurations, the energy relay layer 230 relays energy (e.g., electromagnetic energy, mechanical pressure waves, etc.) between the energy device layer 220 and the energy waveguide layer 240. The energy relay layer 230 includes one or more energy relay elements 260. Each energy relay element includes a first surface 265 and a second surface 270, and it relays energy between the two surfaces. The first surface 265 of each energy relay element can be coupled to one or more energy devices (e.g., an electronic display or an acoustic projection device). The energy relay element can be made of, for example, glass, carbon, optical fiber, optical film, plastic, polymer, or some combination thereof. In addition, in some embodiments, the energy relay element can adjust the magnification (increase or decrease) of the energy passing between the first surface 265 and the second surface 270. If the repeater provides magnification, the repeater can take the form of an array of bonded conical repeaters called cones, where the area of ​​one end of the cone can be substantially larger than the area of ​​the opposite end. The large ends of the cones can be bonded together to form a seamless energy surface 275. One advantage is that space is created on the multiple small ends of each cone to accommodate the mechanical envelope of multiple energy sources, such as the bezels of multiple displays. This additional room allows the energy sources to be placed side by side on the small cone sides, where the active area of ​​each energy source directs energy into the small cone surface and relays to the large seamless energy surface. Another advantage of using a cone repeater is that there is no non-imaging dead space on the combined seamless energy surface formed by the large ends of the cones. There are no borders or borders, and therefore the seamless energy surfaces can then be tiled together to form a larger surface with almost no seams, depending on the visual acuity of the eye.

[0050] The second surfaces of adjacent energy-relay elements converge to form energy surface 275. In some embodiments, the spacing between edges of adjacent energy-relay elements is less than the minimum perceptible outline defined by the visual acuity of a human eye having, for example, 20 / 40 vision, such that energy surface 275 is effectively seamless from the perspective of viewer 280 within viewing volume 285.

[0051] In some embodiments, the second surfaces of adjacent energy relay elements are fused together with a processing step that may include one or more of pressure, heat, and chemical reaction in such a way that there are no seams between them. And in still other embodiments, an array of energy relay elements is formed by molding one side of a continuous block of relay material into an array of small tapered ends, each energy relay element being configured to transfer energy from an energy device attached to the small tapered end to a larger area of ​​a single combined surface that is not subdivided.

[0052] In some embodiments, one or more of the energy relay elements exhibit energy localization, wherein the efficiency of energy transfer in a longitudinal direction substantially perpendicular to surfaces 265 and 270 is much higher than the efficiency of transfer in a perpendicular transverse plane, and wherein the energy density is highly localized in this transverse plane as the energy wave propagates between surface 265 and surface 270. This localization of energy enables energy distributions (such as images) to be efficiently relayed between these surfaces without any significant loss of resolution.

[0053] The energy waveguide layer 240 guides energy from a location (e.g., a coordinate) on the energy surface 275 to a specific energy propagation path from the display surface outward into the holographic viewing volume 285 using waveguide elements in the energy waveguide layer 240. The energy propagation path is defined by at least two angular dimensions determined by the energy surface coordinate location relative to the waveguide. The waveguide is associated with a spatial 2D coordinate. These four coordinates together form a four-dimensional (4D) energy field. As an example, for electromagnetic energy, the waveguide elements in the energy waveguide layer 240 guide light from a location on the seamless energy surface 275 through the viewing volume 285 along different propagation directions. In various examples, the light is guided according to a 4D light field function to form a holographic object 250 within the holographic object volume 255.

[0054] Each waveguide element in the energy waveguide layer 240 can be, for example, a lenslet composed of one or more elements. In some configurations, the lenslet can be a positive lens. The positive lens can have a spherical, non-spherical, or free-form surface profile. In addition, in some embodiments, some or all of the waveguide elements can include one or more additional optical components. The additional optical components can be, for example, energy suppression structures such as baffles, positive lenses, negative lenses, spherical lenses, non-spherical lenses, free-form lenses, liquid crystal lenses, liquid lenses, refractive elements, diffractive elements, or some combination thereof. In some embodiments, at least one of the lenslets and / or additional optical components is capable of dynamically adjusting its optical power. For example, the lenslet can be a liquid crystal lens or a liquid lens. Dynamic adjustment of the surface profile of the lenslet and / or at least one additional optical component can provide additional directional control of the light projected from the waveguide element.

[0055] In the illustrated example, the holographic object volume 255 of the LF display has boundaries formed by rays 256 and ray 257, but may be formed by other rays. The holographic object volume 255 is a continuous volume that extends both in front of the energy waveguide layer 240 (i.e., toward the viewer 280) and behind it (i.e., away from the viewer 280). In the illustrated example, the rays 256 and ray 257 that can be perceived by the user are projected from opposite edges of the LF display module 210 at the maximum angle relative to the normal of the display surface 277, but the rays may be other projected rays. The rays define the field of view of the display and therefore define the boundaries of the holographic viewing volume 285. In some cases, the rays define a holographic viewing volume in which the entire display can be viewed without vignetting (e.g., an ideal viewing volume). As the field of view of the display increases, the convergence point of the rays 256 and rays 257 will be closer to the display. Therefore, a display with a larger field of view allows the viewer 280 to see the entire display at a closer viewing distance. In addition, rays 256 and 257 may form an ideal holographic object volume. A holographic object presented in an ideal holographic object volume may be seen anywhere in viewing volume 285.

[0056] In some instances, the holographic object may be presented to only a portion of the viewing volume 285. In other words, the holographic object volume may be divided into any number of viewing sub-volumes (e.g., viewing sub-volumes 290). In addition, the holographic object may be projected outside the holographic object volume 255. For example, the holographic object 251 is presented outside the holographic object volume 255. Because the holographic object 251 is presented outside the holographic object volume 255, it cannot be viewed from every position in the viewing volume 285. For example, the holographic object 251 may be visible from a position in the viewing sub-volume 290, but not from the position of the viewer 280.

[0057] For example, go to Figure 2B To show the holographic content viewed from different viewing sub-bodies. Figure 2B A cross section of a portion of a LF display module is illustrated according to one or more embodiments. Figure 2B The cross section and Figure 2A The cross-sections are the same. However, Figure 2B2 illustrates a set of different light rays projected from the LF display module 210. Rays 256 and ray 257 still form the holographic object volume 255 and the viewing volume 285. However, as shown, the rays projected from the top of the LF display module 210 and the rays projected from the bottom of the LF display module 210 overlap to form various viewing sub-volumes (e.g., viewing sub-volumes 290A, 290B, 290C, and 290D) within the viewing volume 285. Viewers in the first viewing sub-volume (e.g., 290A) may be able to perceive the holographic content presented in the holographic object volume 255, while viewers in other viewing sub-volumes (e.g., 290B, 290C, and 290D) may not be able to perceive it.

[0058] More simply, Figure 2A As illustrated, the holographic object volume 255 is a volume in which a holographic object can be presented by the LF display system so that the holographic object can be perceived by a viewer (e.g., viewer 280) in a viewing volume 285. In this way, the viewing volume 285 is an example of an ideal viewing volume, and the holographic object volume 255 is an example of an ideal object volume. However, in various configurations, a viewer can perceive a holographic object presented by the LF display system 200 in other example holographic object volumes. More generally, when viewing holographic content projected from an LF display module, a "sight line guide" is applied. The sight line guide asserts that the line formed by the viewer's eye positioning and the holographic object being viewed must intersect the LF display surface.

[0059] Because the holographic content is presented according to the 4D light field function, each eye of the viewer 280 sees a different perspective of the holographic object 250 when viewing the holographic content presented by the LF display module 210. In addition, as the viewer 280 moves within the viewing volume 285, he / she will also see different perspectives of the holographic object 250, as will other viewers within the viewing volume 285. As one of ordinary skill in the art will appreciate, 4D light field functions are well known in the art and will not be described in further detail herein.

[0060] As described in more detail herein, in some embodiments, the LF display can project more than one type of energy. For example, the LF display can project two types of energy, for example, mechanical energy and electromagnetic energy. In this configuration, the energy relay layer 230 can include two separate energy repeaters that are intertwined at the energy surface 275, but separated so that the energy is relayed to two different energy device layers 220. Here, one repeater can be configured to transmit electromagnetic energy, while the other repeater can be configured to transmit mechanical energy. In some embodiments, mechanical energy can be projected from a location between electromagnetic waveguide elements on the energy waveguide layer 240, thereby helping to form a structure that inhibits light from being transmitted from one electromagnetic waveguide element to another. In some embodiments, the energy waveguide layer 240 can also include a waveguide element that transmits focused ultrasound along a specific propagation path according to display instructions from a controller.

[0061] It should be noted that in an alternative embodiment (not shown), the LF display module 210 does not include the energy relay layer 230. In this case, the energy surface 275 is an emitting surface formed using one or more adjacent electronic displays within the energy device layer 220. And in some embodiments, in the absence of the energy relay layer, the spacing between the edges of the adjacent electronic displays is less than the minimum perceptible outline defined by the visual acuity of a human eye with 20 / 40 vision, so that the energy surface is effectively seamless from the perspective of the viewer 280 within the viewing volume 285.

[0062] LF Display Module

[0063] Figure 3A 300A according to one or more embodiments. The LF display module 300A may be the LF display module 110 and / or the LF display module 210. In other embodiments, the LF display module 300A may be some other LF display module. In the illustrated embodiment, the LF display module 300A includes an energy device layer 310 and an energy relay layer 320 and an energy waveguide layer 330. The LF display module 300A is configured to present holographic content from a display surface 365, as described herein. For convenience, the display surface 365 is shown in dashed outline on the frame 390 of the LF display module 300A, but is more accurately the surface directly in front of the waveguide element defined by the inner edge of the frame 390. The display surface 365 includes a plurality of projection locations from which energy may be projected. Some embodiments of the LF display module 300A have components that differ from those described herein. For example, in some embodiments, the LF display module 300A does not include an energy relay layer 320. Similarly, functionality may be distributed among components in ways other than that described herein.

[0064] Energy device layer 310 is an embodiment of energy device layer 220. Energy device layer 310 includes four energy devices 340 (three are visible in the figure). Energy devices 340 can all be of the same type (e.g., all electronic displays) or can include one or more different types (e.g., including an electronic display and at least one acoustic energy device).

[0065] The energy relay layer 320 is an embodiment of the energy relay layer 230. The energy relay layer 320 includes four energy relay devices 350 (three are visible in the figure). The energy relay devices 350 can all relay the same type of energy (e.g., light) or can relay one or more different types (e.g., light and sound). Each of the relay devices 350 includes a first surface and a second surface, and the second surface of the energy relay device 350 is arranged to form a single seamless energy surface 360. In the illustrated embodiment, each of the energy relay devices 350 is conical so that the first surface has a smaller surface area than the second surface, which allows the mechanical envelope of the energy device 340 to be accommodated on the small end of the cone. Since the entire area can project energy, this can also make the seamless energy surface borderless. This means that this seamless energy surface can be tiled by placing multiple instances of the LF display module 300A together without dead space or borders, so that the entire combined surface is seamless. In other embodiments, the surface area of ​​the first surface and the second surface is the same.

[0066] The energy waveguide layer 330 is an embodiment of the energy waveguide layer 240. The energy waveguide layer 330 includes a plurality of waveguide elements 370. Figure 2A and Figure 2B As discussed, the energy waveguide layer 330 is configured to guide energy from the seamless energy surface 360 ​​along a specific propagation path according to a 4D light field function to form a holographic object. It should be noted that in the illustrated embodiment, the energy waveguide layer 330 is defined by a frame 390. In other embodiments, the frame 390 is absent and / or the thickness of the frame 390 is reduced. Removal or reduction of the thickness of the frame 390 can facilitate tiling the LF display module 300A with another LF display module.

[0067] It should be noted that in the illustrated embodiment, the seamless energy surface 360 ​​and the energy waveguide layer 330 are flat. In alternative embodiments not shown, the seamless energy surface 360 ​​and the energy waveguide layer 330 may be curved in one or more dimensions.

[0068] The LF display module 300A can be configured with additional energy sources residing on the surface of the seamless energy surface and allowing the projection of energy fields other than light fields. In one embodiment, the acoustic energy field can be projected from an electrostatic speaker (not shown) mounted at any number of locations on the seamless energy surface 360. In addition, the electrostatic speaker of the LF display module 300A is positioned within the light field display module 300A so that the dual energy surface simultaneously projects the acoustic field and the holographic content. For example, the electrostatic speaker can be formed with one or more diaphragm elements that transmit electromagnetic energy of some wavelengths and driven by one or more conductive elements (e.g., a plane sandwiching one or more diaphragm elements). The electrostatic speaker can be mounted on the seamless energy surface 360 ​​so that the diaphragm element covers some of the waveguide elements. The conductive electrodes of the speaker can be positioned at the same position as a structure designed to suppress light transmission between electromagnetic waveguides, and / or positioned at a position between electromagnetic waveguide elements (e.g., frame 390). In various configurations, the speaker can project audible sounds and / or many sources of focused ultrasonic energy that produce tactile surfaces.

[0069] In some configurations, the energy device 340 can sense energy. For example, the energy device can be a microphone, a light sensor, an acoustic transducer, etc. Therefore, the energy relay device can also relay energy from the seamless energy surface 360 ​​to the energy device layer 310. That is, when the energy device and the energy relay device are configured to simultaneously emit and sense energy (e.g., emit a light field and sense sound), the seamless energy surface 360 ​​of the LF display module forms a bidirectional energy surface.

[0070] More generally, the energy device 340 of the LF display module may be an energy source or an energy sensor. The LF display module 300A may include various types of energy devices that act as energy sources and / or energy sensors to facilitate projection of high-quality holographic content to a user. Other sources and / or sensors may include thermal sensors or sources, infrared sensors or sources, image sensors or sources, mechanical energy transducers that generate acoustic energy, feedback sources, etc. A plurality of other sensors or sources are possible. Furthermore, the LF display modules may be tiled such that the LF display modules may form an assembly that projects and senses multiple types of energy from a large aggregate seamless energy surface.

[0071] In various embodiments of the LF display module 300A, the seamless energy surface 360 ​​may have various surface portions, each of which is configured to project and / or emit a specific type of energy. For example, when the seamless energy surface is a dual energy surface, the seamless energy surface 360 ​​includes one or more surface portions that project electromagnetic energy and one or more other surface portions that project ultrasonic energy. The surface portions that project ultrasonic energy may be positioned on the seamless energy surface 360 ​​between electromagnetic waveguide elements and / or co-located with structures designed to inhibit light transmission between electromagnetic waveguide elements. In instances where the seamless energy surface is a dual energy surface, the energy relay layer 320 may include two types of energy relay devices interwoven at the seamless energy surface 360. In various embodiments, the seamless energy surface 360 ​​may be configured such that the portion of the surface below any particular waveguide element 370 is all energy sources, all energy sensors, or a mixture of energy sources and energy sensors.

[0072] Figure 3B 3 is a cross-sectional view of an LF display module 300B including an interlaced energy relay device according to one or more embodiments. The energy relay device 350A transmits energy between the energy relay first surface 345A connected to the energy device 340A and the seamless energy surface 360. The energy relay 350B transmits energy between the energy relay first surface 345B connected to the energy device 340B and the seamless energy surface 360. The two relay devices are interlaced at the interlaced energy relay device 352 connected to the seamless energy surface 360. In this configuration, the surface 360 ​​contains interlaced energy locations of both energy devices 340A and 340B, which can be energy sources or energy sensors. Therefore, the LF display module 300B can be configured as a dual energy projection device for projecting more than one type of energy, or configured as a bidirectional energy device for simultaneously projecting one type of energy and sensing another type of energy. The LF display module 300B can be the LF display module 110 and / or the LF display module 210. In other embodiments, the LF display module 300B can be some other LF display module.

[0073] LF display module 300B includes Figure 3A For example, in the illustrated embodiment, the LF display module 300B includes an energy device layer 310, an energy relay layer 320, a seamless energy surface 360, and an energy waveguide layer 330, including at least the same components as those of the LF display module 300A. Figure 3A In addition, the LF display module 300B can present and / or receive energy from the display surface 365. It is worth noting that Figure 3AComponents of the LF display module 300B may be connected and / or oriented alternatively compared to the components of the LF display module 300A in FIG. Some embodiments of the LF display module 300B have components that differ from those described herein. Similarly, functionality may be allocated between components in a manner that differs from that described herein. Figure 3B The design of a single LF display module 300B is illustrated that can be tiled to produce a larger area dual energy projection surface or a bidirectional energy surface.

[0074] In one embodiment, the LF display module 300B is a LF display module of a bidirectional LF display system. The bidirectional LF display system can simultaneously project energy from the display surface 365 and sense energy. The seamless energy surface 360 ​​contains both energy projection locations and energy sensing locations that are closely interwoven on the seamless energy surface 360. Figure 3B In the example of Figure 3A For convenience, the energy relay layer of the LF display module 300B will be referred to herein as an "interwoven energy relay layer".

[0075] The interwoven energy relay layer 320 includes two legs: a first energy relay device 350A and a second energy relay device 350B. Figure 3B 360, each of the legs is shown as a light shaded area. Each of the legs can be made of a flexible relay material and formed to have sufficient length for use with energy devices of various sizes and shapes. In some areas of the interwoven energy relay layer, two legs are tightly interwoven together when approaching the seamless energy surface 360. In the illustrated example, the interwoven energy relay device 352 is shown as a dark shaded area.

[0076] When interwoven at seamless energy surface 360, the energy relay device is configured to relay energy to / from different energy devices. The energy devices are located at energy device layer 310. As illustrated, energy device 340A is connected to energy relay device 350A, and energy device 340B is connected to energy relay device 350B. In various embodiments, each energy device can be an energy source or an energy sensor.

[0077] The energy waveguide layer 330 includes a waveguide element 370 to guide the energy waves from the seamless energy surface 360 ​​along the projected path toward a series of convergence points. In this example, a holographic object 380 is formed at the series of convergence points. Notably, as shown, the convergence of energy at the holographic object 380 occurs on the viewer side (i.e., the front side) of the display surface 365. However, in other examples, the convergence of energy can be anywhere in the holographic object volume, extending both in front of the display surface 365 and behind the display surface 365. The waveguide element 370 can simultaneously guide the incoming energy to an energy device (e.g., an energy sensor), as described below.

[0078] In one example embodiment of the LF display module 300B, an emissive display is used as an energy source (e.g., energy device 340A), and an imaging sensor is used as an energy sensor (e.g., energy device 340B). In this way, the LF display module 300B can simultaneously project holographic content and detect light from a volume in front of the display surface 365. In this way, this embodiment of the LF display module 300B functions as both a LF display and a LF sensor.

[0079] In one embodiment, the LF display module 300B is configured to simultaneously project a light field from a projection position on the display surface to the front of the display surface and capture a light field from the front of the display surface at the projection position. In this embodiment, the energy relay device 350A connects a first set of positions at the seamless energy surface 360 ​​that are positioned below the waveguide element 370 to the energy device 340A. In one example, the energy device 340A is an emissive display with an array of source pixels. The energy relay device 350B connects a second set of positions at the seamless energy surface 360 ​​that are positioned below the waveguide element 370 to the energy device 340B. In one example, the energy device 340B is an imaging sensor with an array of sensor pixels. The LF display module 300B can be configured so that the positions at the seamless energy surface 365 below a particular waveguide element 370 are all emissive display positions, all imaging sensor positions, or some combination of these positions. In other embodiments, the bidirectional energy surface can project and receive various other forms of energy.

[0080] In another example embodiment of the LF display module 300B, the LF display module is configured to project two different types of energy. For example, in one embodiment, the energy device 340A is an emissive display configured to emit electromagnetic energy, and the energy device 340B is an ultrasonic transducer configured to emit mechanical energy. Therefore, both light and sound can be projected from various locations at the seamless energy surface 360. In this configuration, the energy relay device 350A connects the energy device 340A to the seamless energy surface 360 ​​and relays electromagnetic energy. The energy relay device is configured to have properties that enable the energy relay device to efficiently transmit electromagnetic energy (e.g., a changing refractive index). The energy relay device 350B connects the energy device 340B to the seamless energy surface 360 ​​and relays mechanical energy. The energy relay device 350B is configured to have properties for efficient transmission of ultrasonic energy (e.g., a distribution of materials with different acoustic impedances). In some embodiments, mechanical energy can be projected from locations between waveguide elements 370 on the energy waveguide layer 330. The locations where the mechanical energy is projected can form a structure for inhibiting the transmission of light from one electromagnetic waveguide element to another electromagnetic waveguide element. In one example, an array of spatially separated locations where ultrasonic mechanical energy is projected can be configured to form a three-dimensional tactile shape and surface in midair. The surface can coincide with a projected holographic object (e.g., holographic object 380). In some examples, phase delays and amplitude variations across the array can help form tactile shapes.

[0081] In various embodiments, the LF display module 300B with interwoven energy relay devices may include multiple energy device layers, each of which includes a specific type of energy device. In these examples, the energy relay layer is configured to relay the appropriate type of energy between the seamless energy surface 360 ​​and the energy device layer 310.

[0082] Tiled LF display modules

[0083] Figure 4A 4 is a perspective view of a portion of an LF display system 400 that is tiled in two dimensions to form a single-sided seamless surface environment according to one or more embodiments. The LF display system 400 includes a plurality of LF display modules that are tiled to form an array 410. More specifically, each of the small squares in the array 410 represents a tiled LF display module 412. The LF display module 412 can be the same as the LF display module 300A or 300B. The array 410 can cover some or all of a surface (e.g., a wall) of a room, for example. The LF array can cover other surfaces, such as a tabletop, a billboard, a circular building, etc.

[0084] The array 410 may project one or more holographic objects. For example, in the illustrated embodiment, the array 410 projects holographic object 420 and holographic object 422. The tiling of the LF display modules 412 allows for a larger viewing volume and allows objects to be projected at a greater distance from the array 410. For example, in the illustrated embodiment, the viewing volume is approximately the entire area in front of and behind the array 410, rather than a partial volume in front of (and behind) the LF display modules 412.

[0085] In some embodiments, the LF display system 400 presents the holographic object 420 to the viewer 430 and the viewer 434. The viewer 430 and the viewer 434 receive different perspectives of the holographic object 420. For example, the viewer 430 is presented with a direct view of the holographic object 420, while the viewer 434 is presented with a more oblique view of the holographic object 420. As the viewer 430 and / or the viewer 434 move, different perspectives of the holographic object 420 are presented to them. This allows the viewer to visually interact with the holographic object by moving relative to the holographic object. For example, as the viewer 430 walks around the holographic object 420, the viewer 430 sees different sides of the holographic object 420 as long as the holographic object 420 remains in the holographic object volume of the array 410. Therefore, the viewer 430 and the viewer 434 can see the holographic object 420 in the real world space at the same time, as if the holographic object really existed. In addition, the viewer 430 and the viewer 434 do not need to wear an external device in order to view the holographic object 420, because the holographic object 420 is visible to the viewer in much the same way as a physical object would be visible. In addition, here, the holographic object 422 is shown behind the array because the viewing volume of the array extends behind the surface of the array. In this way, the holographic object 422 can be presented to the viewer 430 and / or the viewer 434. Because the image originates from behind the plane of the LF display, the holographic object 422 can be a virtual image 424.

[0086] In some embodiments, LF display system 400 may include a tracking system that tracks the location of viewer 430 and viewer 434. In some embodiments, the location tracked is the location of the viewer. In other embodiments, the location tracked is the location of the viewer's eyes. Eye location tracking is different from gaze tracking, which tracks where the eyes are looking (e.g., using orientation to determine gaze location). The eyes of viewer 430 and the eyes of viewer 434 are located in different locations.

[0087] In various configurations, the LF display system 400 may include one or more tracking systems. Figure 4AIn the illustrated embodiment of , the LF display system includes a tracking system 440 external to the array 410. Here, the tracking system may be a camera system coupled to the array 410. Figure 5A 4. In other example embodiments, the tracking system may be incorporated into the array 410 as described herein. For example, an energy device (e.g., energy device 340) of one or more LF display modules 412 including a bidirectional energy surface included in the array 410 may be configured to capture an image of a viewer in front of the array 410. In any case, one or more tracking systems of the LF display system 400 determine tracking information about a viewer (e.g., viewer 430 and / or viewer 434) viewing holographic content presented by the array 410.

[0088] The tracking information describes the position of the viewer or the position of a part of the viewer (e.g., one or both eyes of the viewer, or a limb of the viewer) in space (e.g., relative to the tracking system). The tracking system may use any number of depth determination techniques to determine the tracking information. The depth determination techniques may include, for example, structured light, time of flight, stereoscopic imaging, some other depth determination technique, or some combination thereof. The tracking system may include various systems configured to determine the tracking information. For example, the tracking system may include one or more infrared sources (e.g., structured light sources), one or more imaging sensors (e.g., red-blue-green-infrared cameras) that can capture infrared images, and a processor that executes a tracking algorithm. The tracking system may use depth estimation techniques to determine the position of the viewer. In some embodiments, the LF display system 400 generates a holographic object based on the tracked position, motion, or gesture of the viewer 430 and / or viewer 434 as described herein. For example, the LF display system 400 may generate a holographic object in response to the viewer entering a threshold distance and / or a specific position of the array 410.

[0089] The LF display system 400 can present one or more holographic objects customized for each viewer based in part on the tracking information. For example, the holographic object 420 can be presented to the viewer 430 instead of the holographic object 422. Similarly, the holographic object 422 can be presented to the viewer 434 instead of the holographic object 420. For example, the LF display system 400 tracks the position of each of the viewers 430 and 434. The LF display system 400 determines the viewing angle of the holographic object that should be visible to the viewer based on the position of the viewer relative to where the holographic object is to be presented. The LF display system 400 selectively projects light from specific pixels corresponding to the determined viewing angle. Therefore, the viewer 434 and the viewer 430 can have potentially completely different experiences at the same time. In other words, the LF display system 400 can present holographic content to a viewing sub-body of the viewing body (i.e., similar to Figure 2B 4. For example, as shown, because the LF display system 400 can track the position of the viewer 430, the LF display system 400 can present spatial content (e.g., holographic object 420) to the viewing sub-volumes surrounding the viewer 430, and present safari content (e.g., holographic object 422) to the viewing sub-volumes surrounding the viewer 434. In contrast, conventional systems would have to use separate headphones to provide a similar experience.

[0090] In some embodiments, the LF display system 400 may include one or more sensory feedback systems. The sensory feedback system provides other sensory stimulation (e.g., touch, audio, or smell) that enhances the holographic objects 420 and 422. For example, in Figure 4A In the illustrated embodiment of , the LF display system 400 includes a sensory feedback system 442 external to the array 410. In one example, the sensory feedback system 442 may be an electrostatic speaker coupled to the array 410. Figure 5A The external sensory feedback system is described in more detail. In other example embodiments, the sensory feedback system can be incorporated into the array 410 as described herein. For example, the energy device (e.g., Figure 3B Energy device 340A in array 410 may be configured to project ultrasound energy to a viewer in front of the array and / or receive imaging information from a viewer in front of the array. In any case, the sensory feedback system presents sensory content to and / or receives sensory content from a viewer (e.g., viewer 430 and / or viewer 434) viewing holographic content (e.g., holographic object 420 and / or holographic object 422) presented by array 410.

[0091] The LF display system 400 may include a sensory feedback system 442, which includes one or more acoustic projection devices outside the array. Alternatively or additionally, the LF display system 400 may include one or more acoustic projection devices integrated into the array 410, as described herein. The acoustic projection device may consist of an array of ultrasound sources configured to project a body tactile surface. In some embodiments, for one or more surfaces of a holographic object, if a portion of the viewer is within a threshold distance of one or more surfaces, the tactile surface may coincide with the holographic object (e.g., at the surface of the holographic object 420). The body tactile sensation may allow the user to touch and feel the surface of the holographic object. Multiple acoustic projection devices may also project audible pressure waves that provide audio content (e.g., immersive audio) to the viewer. Therefore, ultrasonic pressure waves and / or audible pressure waves may play a role in supplementing the holographic object.

[0092] In various embodiments, the LF display system 400 may provide other sensory stimuli based in part on the tracked position of the viewer. For example, Figure 4A The holographic object 422 illustrated in the figure is a lion, and the LF display system 400 can make the holographic object 422 roar both visually (i.e., the holographic object 422 seems to be roaring) and audibly (i.e., one or more acoustic projection devices project pressure waves), so that the viewer 430 perceives it as the roar of a lion emitted by the holographic object 422.

[0093] It should be noted that in the illustrated configuration, it is possible to Figure 2A and Figure 2B The holographic viewing volume is limited in a manner that minimizes the viewing volume 285 of the LF display system 200 in FIG. This can limit the perceived immersion that a viewer would experience with a single wall display unit. One way to address this problem is to use multiple LF display modules tiled along multiple sides, as described below with respect to FIG. Figures 4B to 4F described.

[0094] Figure 4B is a perspective view of a portion of an LF display system 402 in a multi-sided seamless surface environment according to one or more embodiments. The LF display system 402 is substantially similar to the LF display system 400, except that a plurality of LF display modules are tiled to create a multi-sided seamless surface environment. More specifically, the LF display modules are tiled to form an array as a six-sided aggregate seamless surface environment. Figure 4B In one embodiment, the multiple LF display modules cover all walls, ceiling, and floor of the room. In other embodiments, the multiple LF display modules may cover some but not all of the walls, floor, ceiling, or some combination thereof. In other embodiments, the multiple LF display modules are tiled to form some other aggregate seamless surface. For example, the walls may be curved so that a cylindrical aggregate energy environment is formed. In addition, as described below with respect to Figure 6-9 As described, in some embodiments, LF display modules may be tiled to form a surface in a venue (eg, a wall, etc.).

[0095] The LF display system 402 can project one or more holographic objects. For example, in the illustrated embodiment, the LF display system 402 projects a holographic object 420 into an area surrounded by a six-sided polymeric seamless surface environment. In this example, the viewing volume of the LF display system is also contained within the six-sided polymeric seamless surface environment. It should be noted that in the illustrated configuration, the viewer 434 can be positioned between the holographic object 420 and the LF display module 414, which projects energy (e.g., light and / or pressure waves) used to form the holographic object 420. Therefore, the positioning of the viewer 434 can prevent the viewer 430 from perceiving the holographic object 420 formed by the energy from the LF display module 414. However, in the illustrated configuration, there is at least one other LF display module, such as the LF display module 416, which is not obstructed (e.g., by the viewer 434) and can project energy to form the holographic object 420 and be observed by the viewer 430. In this way, being occluded in space by a viewer may cause some portions of the holographic projection to disappear, but this effect is much less than if only one side of the volume were filled with a holographic display panel. The holographic object 422 is displayed as "outside" the walls of the six-sided polymeric seamless surface environment because the holographic object volume extends behind the polymeric surface. Thus, the viewer 430 and / or the viewer 434 may perceive the holographic object 422 as "outside" the enclosed six-sided environment throughout which it can move.

[0096] As mentioned above Figure 4A As described, in some embodiments, the LF display system 402 actively tracks the position of the viewer and can dynamically instruct different LF display modules to present holographic content based on the tracked position. Thus, the multi-faceted configuration can provide a more robust environment (e.g., relative to Figure 4A ) to provide holographic objects where an unconstrained viewer can move freely throughout the area surrounded by a multi-faceted seamless surface environment.

[0097] It is worth noting that various LF display systems can have different configurations. Furthermore, each configuration can have a specific orientation of surfaces that aggregate to form a seamless display surface ("aggregate surface"). That is, the LF display modules of a LF display system can be tiled to form various aggregate surfaces. For example, in Figure 4B In , LF display system 402 includes LF display modules that are tiled to form a six-sided converging surface that approximates a wall of a room. In some other examples, the converging surface may appear on only a portion of the surface (e.g., half of a wall) rather than the entire surface (e.g., the entire wall). Some examples are described herein.

[0098] In some configurations, the converging surface of the LF display system may include a converging surface configured to project energy toward a local viewing volume. Projecting energy to a local viewing volume allows for a higher quality viewing experience by, for example, increasing the density of projected energy in a particular viewing volume, increasing the FOV of a viewer in the viewing volume, and bringing the viewing volume closer to the display surface.

[0099] For example, Figure 4C A top view of an LF display system 450A having a converging surface in a "wing-like" configuration is shown. In this example, the LF display system 450A is positioned in a room having a front wall 452, a rear wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown). The first side wall 456, the second side wall 458, the rear wall 454, the floor, and the ceiling are all orthogonal. The LF display system 450A includes LF display modules tiled to form a converging surface 460 covering the front wall. The front wall 452, and therefore the converging surface 460, comprises three sections: (i) a first section 462 that is substantially parallel to the rear wall 454 (i.e., the center surface), (ii) a second section 464 that connects the first section 462 to the first side wall 456 and is positioned at an angle to project energy toward the center of the room (i.e., the first side surface), and (iii) a third section 466 that connects the first section 462 to the second side wall 458 and is positioned at an angle to project energy toward the center of the room (i.e., the second side surface). The first section is a vertical plane in the room and has a horizontal axis and a vertical axis. The second and third sections are angled along the horizontal axis toward the center of the room.

[0100] In this example, the viewing volume 468A of the LF display system 450A is located in the center of the room and is partially surrounded by three portions of the converging surface 460. The converging surface that at least partially surrounds the viewer (the "surrounding surface") increases the immersive experience of the viewer.

[0101] For illustration, consider, for example, a polymeric surface having only a central surface. Figure 2A , as described above, rays projected from either end of the display surface create an ideal holographic volume and an ideal viewing volume. Now consider, for example, if the central surface includes two side surfaces that are angled toward the viewer. In this case, rays 256 and 257 will be projected at a larger angle from the normal to the central surface. Therefore, the field of view of the viewing volume will increase. Similarly, the holographic viewing volume will be closer to the display surface. In addition, since the two second and third parts are tilted closer to the viewing volume, the holographic object projected at a fixed distance from the display surface is closer to the viewing volume.

[0102] For simplicity, a display surface with only a center surface has a planar field of view, a planar threshold spacing between the (center) display surface and the viewing volume, and a planar proximity between the holographic object and the viewing volume. Adding one or more side surfaces angled toward the viewer increases the field of view relative to the planar field of view, decreases the spacing between the display surface and the viewing volume relative to the planar spacing, and increases the proximity between the display surface and the holographic object relative to the planar proximity. Angling the side surfaces further toward the viewer further increases the field of view, decreases the spacing, and increases the proximity. In other words, the angled placement of the side surfaces increases the immersive experience of the viewer.

[0103] In addition, as mentioned below Figure 6 As described, deflection optics can be used to optimize the size and positioning of the viewing volume with respect to LF display parameters (eg, size and FOV).

[0104] Return to Figure 4D , in similar instances, Figure 4D A side view of an LF display system 450B having a converging surface in a "tilted" configuration is shown. In this example, the LF display system 450B is positioned in a room having a front wall 452, a rear wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. The first side wall, the second side wall, the rear wall 454, the floor 474, and the ceiling 472 are all orthogonal. The LF display system 450B includes LF display modules tiled to form a converging surface 460 covering the front wall. The front wall 452, and therefore the converging surface 460, comprises three sections: (i) a first section 462 that is substantially parallel to the rear wall 454 (i.e., the center surface), (ii) a second section 464 that connects the first section 462 to the ceiling 472 and is angled to project energy toward the center of the room (i.e., the first side surface), and (iii) a third section 464 that connects the first section 462 to the floor 474 and is angled to project energy toward the center of the room (i.e., the second side surface). The first section is a vertical plane in the room and has a horizontal axis and a vertical axis. The second and third sections are angled along the vertical axis toward the center of the room.

[0105] In this example, the viewing volume 468B of the LF display system 450B is located in the center of the room and is partially surrounded by three portions of the converging surface 460. Figure 4CSimilar to the configuration shown, the two side portions (e.g., second portion 464 and third portion 466) are angled to surround the viewer and form a surrounding surface. The surrounding surface increases the viewing FOV from the perspective of any viewer in the holographic viewing volume 468B. In addition, the surrounding surface allows the viewing volume 468B to be closer to the surface of the display, making the projected objects appear closer. In other words, the angled placement of the side surfaces increases the field of view, reduces the separation, and increases the proximity of the converging surfaces, thereby increasing the immersive experience of the viewer. In addition, as will be discussed below, deflection optics can be used to optimize the size and positioning of the viewing volume 468B.

[0106] The tilted configuration of the side portions of the converging surface 460 enables the holographic content to be presented closer to the viewing volume 468B than if the third portion 466 were not tilted. For example, the lower extremities (e.g., legs) of a character presented from a LF display system in a tilted configuration may appear closer and more realistic than if a LF display system with a flat front wall were used.

[0107] Additionally, the configuration of the LF display system and the environment in which it is located can inform the shape and position of the viewing volume and viewing sub-volumes.

[0108] Figure 4E For example, a top view of a LF display system 450C is shown having a converging surface 460 on a front wall 452 of a room. In this example, a LF display system 450D is positioned in a room having a front wall 452, a back wall 454, a first side wall 456, a second side wall 458, a ceiling (not shown), and a floor (not shown).

[0109] LF display system 450C projects various rays from converging surface 460. Light rays are projected from each location on the display surface into a range of angles centered on the viewing volume. Rays projected from the left side of converging surface 460 have a horizontal angle range 481, rays projected from the right side of converging surface 460 have a horizontal angle range 482, and rays projected from the center of converging surface 460 have a horizontal angle range 483. Between these points, the projected rays may take the following forms as described below with respect to Figure 6 The midpoint of the described angular range. In this way, having a gradient deflection angle in the projection rays across the display surface creates viewing volume 468C. In addition, this configuration avoids wasting the resolution of the display when projecting rays into sidewalls 456 and 458.

[0110] Figure 4FA side view of an LF display system 450D having a converging surface 460 on a front wall 452 of a room is shown. In this example, the LF display system 450E is positioned in a room having a front wall 452, a back wall 454, a first side wall (not shown), a second side wall (not shown), a ceiling 472, and a floor 474. In this example, the floor is layered so that each layer rises in a stepwise motion from the front wall to the back wall. Here, each layer of the floor contains viewing sub-volumes (e.g., viewing sub-volumes 470A and 470B). The layered floor allows for non-overlapping viewing sub-volumes. In other words, each viewing sub-volume has a line of sight from the viewing sub-volume to the converging surface 460 that does not pass through another viewing sub-volume. In other words, this orientation creates a "stadium seating" effect, where the vertical offset between the layers allows for an unobstructed line of sight, allowing each layer to "see" the viewing sub-volumes of the other layers. A LF display system containing non-overlapping viewing sub-volumes can provide a more enhanced viewing experience than a LF display system having viewing sub-volumes that do overlap. For example, in Figure 4F In the configuration shown, different holographic content may be projected to viewers in viewing sub-volumes 470A and 470B.

[0111] LF display system control

[0112] Figure 5A is a block diagram of an LF display system 500 according to one or more embodiments. The LF display system 500 includes an LF display component 510 and a controller 520. The LF display component 510 includes one or more LF display modules 512 that project a light field. The LF display module 512 may include a source / sensor system 514, which includes one or more integrated energy sources and / or one or more energy sensors that project and / or sense other types of energy. The controller 520 includes a data storage device 522, a network interface 524, and a LF processing engine 530. The controller 520 may also include a tracking module 526 and a viewer profile analysis module 528. In some embodiments, the LF display system 500 also includes a sensory feedback system 570 and a tracking system 580. Figure 1 The LF display system described in the context of FIG. 1 , 2, 3, and 4 is an embodiment of the LF display system 500. In other embodiments, the LF display system 500 includes additional or fewer modules than those described herein. Similarly, functionality may be allocated between modules and / or different entities in a manner different from that described herein. Applications of the LF display system 500 will also be described below with respect to FIG. Figure 2A , Figure 2B , Figure 3A , Figure 3B , Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E, Figure 4F , Figure 5A and Figure 5B Discuss in detail.

[0113] The LF display component 510 provides holographic content in a holographic object volume that can be visible to a viewer positioned within the viewing volume. The LF display component 510 can provide the holographic content by executing display instructions received from the controller 520. The holographic content can include one or more holographic objects projected in front of a polymeric surface of the LF display component 510, behind a polymeric surface of the LF display component 510, or some combination thereof. Generating display instructions with the controller 520 is described in more detail below.

[0114] The LF display assembly 510 provides holographic content using one or more LF display modules (e.g., any one of the LF display module 110, the LF display system 200, and the LF display module 300) included in the LF display assembly 510. For convenience, one or more LF display modules may be described herein as LF display modules 512. The LF display modules 512 may be tiled to form the LF display assembly 510. The LF display modules 512 may be structured into various seamless surface environments (e.g., single-sided, multi-sided, walls of a venue, curved surfaces, etc.). That is, the tiled LF display modules form a polymeric surface. As previously described, the LF display module 512 includes an energy device layer (e.g., the energy device layer 220) and an energy waveguide layer (e.g., the energy waveguide layer 240) that presents holographic content. The LF display module 512 may also include an energy relay layer (e.g., the energy relay layer 230) that transfers energy between the energy device layer and the energy waveguide layer when presenting holographic content.

[0115] The LF display module 512 may also include other integrated systems that are configured for energy projection and / or energy sensing as previously described. For example, the LF display module 512 may include any number of energy devices (e.g., energy device 340) configured to project and / or sense energy. For convenience, the integrated energy projection system and the integrated energy sensing system of the LF display module 512 may be aggregated and described herein as a source / sensor system 514. The source / sensor system 514 is integrated within the LF display module 512 so that the source / sensor system 514 shares the same seamless energy surface with the LF display module 512. In other words, the aggregate surface of the LF display assembly 510 includes the functions of both the LF display module 512 and the source / sensor system 514. In other words, the LF display assembly 510 including the LF display module 512 with the source / sensor system 514 can project energy and / or sense energy while simultaneously projecting a light field. For example, the LF display assembly 510 may include a LF display module 512 and a source / sensor system 514 configured as a dual energy surface or a bidirectional energy surface as previously described.

[0116] In some embodiments, the LF display system 500 uses a sensory feedback system 570 to enhance the generated holographic content with other sensory content (e.g., coordinated touch, audio, or smell). The sensory feedback system 570 can enhance the projection of the holographic content by executing display instructions received from the controller 520. Typically, the sensory feedback system 570 includes any number of sensory feedback devices (e.g., sensory feedback system 442) outside the LF display component 510. Some example sensory feedback devices may include coordinated acoustic projection and receiving devices, aroma projection devices, temperature adjustment devices, force actuation devices, pressure sensors, transducers, etc. In some cases, the sensory feedback system 570 may have functionality similar to the LF display component 510, and vice versa. For example, both the sensory feedback system 570 and the LF display component 510 may be configured to generate a sound field. As another example, the sensory feedback system 570 may be configured to generate a tactile surface without the LF display component 510.

[0117] For illustration, in an example embodiment of the light field display system 500, the sensory feedback system 570 may include one or more acoustic projection devices. One or more acoustic projection devices are configured to generate one or more pressure waves complementary to the holographic content when executing the display instructions received from the controller 520. The generated pressure waves may be, for example, audible (for sound), ultrasonic (for touch), or some combination thereof. Similarly, the sensory feedback system 570 may include an aroma projection device. The aroma projection device may be configured to provide fragrance to some or all of the target area when executing the display instructions received from the controller. The aroma device may be connected to an air circulation system (e.g., a duct, fan, or vent) to coordinate the air flow in the target area. In addition, the sensory feedback system 570 may include a temperature adjustment device. The temperature adjustment device is configured to increase or decrease the temperature in some or all of the target area when executing the display instructions received from the controller 520.

[0118] In some embodiments, the sensory feedback system 570 is configured to receive input from a viewer of the LF display system 500. In this case, the sensory feedback system 570 includes various sensory feedback devices for receiving input from the viewer. The sensory feedback device may include devices such as acoustic receiving devices (e.g., microphones), pressure sensors, joysticks, motion detectors, transducers, etc. The sensory feedback system may transmit the detected input to the controller 520 to coordinate the generation of holographic content and / or sensory feedback.

[0119] For illustration, in an example embodiment of a light field display assembly, the sensory feedback system 570 includes a microphone. The microphone is configured to record audio (e.g., gasps, screams, laughter, etc.) generated by one or more viewers. The sensory feedback system 570 provides the recorded audio as viewer input to the controller 520. The controller 520 can use the viewer input to generate holographic content. Similarly, the sensory feedback system 570 can include a pressure sensor. The pressure sensor is configured to measure the force applied to the pressure sensor by the viewer. The sensory feedback system 570 can provide the measured force as viewer input to the controller 520.

[0120] In some embodiments, the LF display system 500 includes a tracking system 580. The tracking system 580 includes any number of tracking devices configured to determine the location, movement, and / or characteristics of a viewer in a target area. Typically, the tracking device is external to the LF display assembly 510. Some example tracking devices include a camera assembly ("camera"), a depth sensor, a structured light, a LIDAR system, a card scanning system, or any other tracking device that can track a viewer in a target area.

[0121] The tracking system 580 may include one or more energy sources that illuminate some or all of the target area with light. However, in some cases, when presenting holographic content, the target area is illuminated by natural light and / or ambient light from the LF display assembly 510. The energy source projects light when executing instructions received from the controller 520. The light may be, for example, a structured light pattern, a light pulse (e.g., an IR flash), or some combination thereof. The tracking system may project light in the visible band (approximately 380 nm to 750 nm), the infrared (IR) band (approximately 750 nm to 1700 nm), the ultraviolet band (10 nm to 380 nm), some other part of the electromagnetic spectrum, or some combination thereof. The source may include, for example, a light emitting diode (LED), a micro-LED, a laser diode, a TOF depth sensor, an adjustable laser, and the like.

[0122] When executing instructions received from controller 520, tracking system 580 can adjust one or more emission parameters. Emission parameters are parameters that affect how light is projected from the source of tracking system 580. Emission parameters can include, for example, brightness, pulse rate (including continuous illumination), wavelength, pulse length, some other parameter that affects how light is projected from the source component, or some combination thereof. In one embodiment, the source projects light pulses in a time-of-flight operation.

[0123] The camera of the tracking system 580 captures an image of light (e.g., a structured light pattern) reflected from the target area. The camera captures the image when executing the tracking instructions received from the controller 520. As previously described, the light can be projected by the source of the tracking system 580. The camera can include one or more cameras. That is, the camera can be, for example, an array of photodiodes (1D or 2D), a CCD sensor, a CMOS sensor, some other device that detects some or all of the light projected by the tracking system 580, or some combination thereof. In one embodiment, the tracking system 580 can contain a light field camera external to the LF display component 510. In other embodiments, the camera is included as part of the LF display source / sensor system 514 included in the LF display component 510. For example, as previously described, if the energy relay element of the LF display module 512 is a bidirectional energy layer that interweaves both the emission display and the imaging sensor at the energy device layer 220, the LF display component 510 can be configured to simultaneously project a light field and record imaging information from the viewing area in front of the display. In one embodiment, the image captured from the bidirectional energy surface forms a light field camera. The camera provides the captured image to the controller 520.

[0124] When executing tracking instructions received from controller 520, the camera of tracking system 580 can adjust one or more imaging parameters. Imaging parameters are parameters that affect how the camera captures an image. Imaging parameters can include, for example, frame rate, aperture, gain, exposure length, frame timing, rolling shutter or global shutter capture mode, some other parameter that affects how the camera captures an image, or some combination thereof.

[0125] Controller 520 controls LF display assembly 510 and any other components of LF display system 500. Controller 520 includes data storage 522, network interface 524, tracking module 526, viewer profiling module 528, and LF processing engine 530. In other embodiments, controller 520 includes more or fewer modules than described herein. Similarly, functionality may be allocated between modules and / or different entities in a manner different from that described herein. For example, tracking module 526 may be part of LF display assembly 510 or tracking system 580.

[0126] The data storage device 522 is a memory that stores information for the LF display system 500. The stored information may include display instructions, tracking instructions, transmission parameters, imaging parameters, a virtual model of the target area, tracking information, images captured by a camera, one or more viewer profiles, calibration data for the LF display component 510, configuration data for the LF display component 510 (including the resolution and orientation of the LF display module 512), the desired viewing volume geometry, content for graphic creation including 3D models, scenes and environments, materials and textures, and other information that may be used by the LF display system 500, or some combination thereof. The data storage device 522 is a memory such as a read-only memory (ROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), or some combination thereof.

[0127] The network interface 524 allows the light field display system to communicate with other systems or environments over a network. In one example, the LF display system 500 receives holographic content from a remote light field display system via the network interface 524. In another example, the LF display system 500 transmits holographic content to a remote data storage device using the network interface 524.

[0128] The tracking module 526 tracks viewers viewing content presented by the LF display system 500. To this end, the tracking module 526 generates tracking instructions that control the operation of one or more sources and / or one or more cameras of the tracking system 580 and provides the tracking instructions to the tracking system 580. The tracking system 580 executes the tracking instructions and provides tracking input to the tracking module 526.

[0129] The tracking module 526 can determine the location of one or more viewers within the target area (e.g., sitting in a seat in the venue). The determined location can be relative to, for example, a reference point (e.g., a display surface). In other embodiments, the determined location can be within a virtual model of the target area. The tracked location can be, for example, the tracked location of the viewer and / or the tracked location of a portion of the viewer (e.g., eye position, hand position, etc.). The tracking module 526 uses one or more captured images from the camera of the tracking system 580 to determine the location. The cameras of the tracking system 580 can be distributed around the LF display system 500 and can capture stereo images, allowing the tracking module 526 to passively track the viewer. In other embodiments, the tracking module 526 actively tracks the viewer. That is, the tracking system 580 illuminates a portion of the target area, images the target area, and the tracking module 526 uses flight time and / or structured light depth determination technology to determine the location. The tracking module 526 uses the determined location to generate tracking information.

[0130] The tracking module 526 may also receive tracking information as input from a viewer of the LF display system 500. The tracking information may include body movements corresponding to various input options provided to the viewer by the LF display system 500. For example, the tracking module 526 may track the body movements of the viewer and distribute any of the various movements as input to the LF processing engine 530. The tracking module 526 may provide the tracking information to the data storage device 522, the LF processing engine 530, the viewer profiling module 528, any other component of the LF display system 500, or some combination thereof.

[0131] To provide context for the tracking module 526, consider an example embodiment of the LF display system 500 showing a play in which the performers successfully defeat an enemy character. In response to the scene, the viewer pumps his fist in the air to show his excitement. The tracking system 580 can record the movement of the viewer's hand and transmit the record to the tracking module 526. As previously described, this can be achieved by a tracking system 580 that includes a camera, depth sensor, or other device outside the LF display component 510, or a display surface that simultaneously projects a light field image and records the image, where the image recorded from the display surface can be a light field image or any combination of these devices. The tracking module 526 tracks the movement of the viewer's hand in the record and sends the input to the LF processing engine 530. As described below, the viewer profile analysis module 528 determines that the information in the image indicates that the movement of the viewer's hand is associated with a positive response. Therefore, if enough viewers are identified to have a positive response, the LF processing engine 530 generates appropriate holographic content to celebrate the victory of the winning character over the enemy character. For example, the LF processing engine 530 can project confetti in the scene.

[0132] The LF display system 500 includes a viewer profiling module 528 configured to identify and profile a viewer. The viewer profiling module 528 generates a profile of a viewer (or multiple viewers) viewing the holographic content displayed by the LF display system 500. The viewer profiling module 528 generates a viewer profile based in part on viewer input and monitored viewer behavior, actions, and reactions. The viewer profiling module 528 can access information obtained from the tracking system 580 (e.g., recorded images, videos, sounds, etc.) and process the information to determine various information. In various instances, the viewer profiling module 528 can use any number of machine vision or machine hearing algorithms to determine viewer behavior, actions, and reactions. The monitored viewer behavior can include, for example, smiling, cheering, clapping, laughing, fright, screaming, excitement level, retreating, other changes in posture, or movement of the viewer, etc.

[0133] More generally, a viewer profile may include any information received and / or determined about a viewer viewing holographic content from a LF display system. For example, each viewer profile may record the viewer's actions or responses to content displayed by the LF display system 500. Some example information that may be included in a viewer profile is provided below.

[0134] In some embodiments, the viewer profile may describe the viewer's response to displayed performers (e.g., actors, singers, etc.), venues (e.g., stage, concert hall, etc.), etc. For example, the viewer profile may indicate that viewers generally respond positively to a performance by a group of handsome male singers with frosted blond hair between the ages of 18 and 25.

[0135] In some embodiments, the viewer profile may indicate characteristics of a viewer viewing a performance. For example, a viewer in a venue is wearing a jersey displaying a university logo. In this case, the viewer profile may indicate that the viewer is wearing a jersey and may prefer holographic content associated with the university logo on the jersey. More generally, viewer characteristics that may be indicated in a viewer profile may include, for example, age, gender, clothing, viewing location in the venue, etc.

[0136] In some embodiments, the viewer profile may indicate the viewer's preferences regarding desired performances and / or venue characteristics. For example, the viewer profile may indicate that the viewer only likes to view holographic content that is appropriate for the age of each person in his or her family. In another example, the viewer profile may indicate that the holographic object body displays holographic content (e.g., on a wall) and indicates that the holographic object body does not display holographic content (e.g., above its head). The viewer profile may also indicate that the viewer prefers to have a haptic interface presented near him or her, or prefers to avoid the haptic interface.

[0137] In another example, the viewer profile indicates a history of performances viewed by a particular viewer. For example, the viewer profiling module 528 determines that a viewer or a group of viewers has previously viewed a performance. As a result, the LF display system 500 may display different holographic content than the viewer's previous viewing of the performance. As an example, a performance containing holographic content may have three different endings, and the LF display system 500 may display different endings based on the viewers in attendance. In another example, each of the three endings may be presented to a different viewing entity in the same venue.

[0138] In some embodiments, the viewer profile can also describe the characteristics and preferences of a group of viewers rather than a specific viewer. For example, the viewer profile analysis module 528 can generate a viewer profile for the audience watching the performance in the venue. In one example, the viewer profile analysis module 528 creates a viewer profile for viewers who watch a show about a teenager falling in love with a certain type of show. The profile indicates that 86.3% of the viewers are women between 20 and 35 years old and have a positive response to the performance. The profile also indicates that the remaining 23.7% of the viewers are men between 20 and 35 years old and have a negative reaction to the performance. Any of the previously described information and characteristics can be applied to a group of viewers.

[0139] The viewer profiling module 528 may also access profiles associated with a particular viewer (or multiple viewers) from one or more third-party systems to build a viewer profile. For example, a viewer purchases a ticket to a show using a third-party vendor that is linked to the viewer's social media account. Thus, the viewer's ticket is linked to his social media account. When the viewer uses his ticket to enter the show, the viewer profiling module 528 may access information from his social media account to build (or enhance) the viewer profile.

[0140] In some embodiments, the data storage device 522 includes a viewer profile storage that stores viewer profiles generated, updated, and / or maintained by the viewer profile analysis module 528. The viewer profile can be updated in the data storage device at any time by the viewer profile analysis module 528. For example, in one embodiment, when a specific viewer views the holographic content provided by the LF display system 500, the viewer profile storage receives and stores information about the specific viewer in its viewer profile. In this example, the viewer profile analysis module 528 includes a facial recognition algorithm that can recognize the viewer and positively identify the viewer when it views the presented holographic content. For illustration, when the viewer enters the target area of ​​the LF display system 500, the tracking system 580 obtains an image of the viewer. The viewer profile analysis module 528 inputs the captured image and uses the facial recognition algorithm to identify the viewer's face. The identified face is associated with the viewer profile in the profile storage device, and in this way, all the input information obtained about the viewer can be stored in its profile. The viewer profiling module may also utilize a card identification scanner, voice identifier, radio frequency identification (RFID) chip scanner, bar code scanner, or the like to positively identify the viewer.

[0141] In embodiments where the viewer profiling module 528 can positively identify a viewer, the viewer profiling module 528 can determine each visit of each viewer to the LF display system 500. The viewer profiling module 528 can then store the time and date of each visit in the viewer profile of each viewer. Similarly, the viewer profiling module 528 can store input received from a viewer from any combination of the sensory feedback system 570, the tracking system 580, and / or the LF display component 510 each time it occurs. The viewer profiling module 528 can additionally receive other information about the viewer from other modules or components of the controller 520, which can then be stored with the viewer profile. Other components of the controller 520 can then also access the stored viewer profile to determine subsequent content provided to the viewer.

[0142] The LF processing engine 530 generates holographic content including light field data and data of all sensory domains supported by the LF display system 500. For example, the LF processing engine 530 can generate 4D coordinates in a rasterized format ("rasterized data"), which, when executed by the LF display component 510, causes the LF display component 510 to present holographic content. The LF processing engine 530 can access the rasterized data from the data storage device 522. In addition, the LF processing engine 530 can construct the rasterized data from the vectorized data set. The vectorized data is described below. The LF processing engine 530 can also generate the sensory instructions required to provide the sensory content of the enhanced holographic object. As described above, when executed by the LF display system 500, the sensory instructions can generate tactile surfaces, sound fields, and other forms of sensory energy supported by the LF display system 500. The LF processing engine 530 can access the sensory instructions from the data storage device 522 and construct the sensory instructions from the vectorized data set. The 4D coordinates and sensory data collectively represent the holographic content as display instructions that can be executed by the LF display system to generate holographic and sensory content. More generally, holographic content can take the form of CG content having the following: ideal light field coordinates, live action content, rasterized data, vectorized data, electromagnetic energy transmitted by a set of repeaters, instructions sent to a set of energy devices, energy locations on one or more energy surfaces, a set of energy propagation paths projected from a display surface, holographic objects visible to a viewer or audience, and many other similar forms.

[0143] The amount of rasterized data describing the energy flow through the various energy sources in the LF display system 500 is very large. Although the rasterized data can be displayed on the LF display system 500 when accessed from the data storage device 522, the rasterized data cannot be efficiently transmitted, received (e.g., via the network interface 524), and then displayed on the LF display system 500. For example, take the rasterized data representing a short performance holographically projected by the LF display system 500. In this example, the LF display system 500 includes a display containing several billion pixels, and the rasterized data contains information for each pixel position of the display. The corresponding size of the rasterized data is huge (e.g., several gigabytes per second of performance display time) and is unmanageable for efficient transmission over a commercial network via the network interface 524. For applications involving real-time streaming of holographic content, the efficient delivery problem can be magnified. When an interactive experience is required using input from the sensory feedback system 570 or the tracking module 526, the additional problem of storing the rasterized data only on the data storage device 522 arises. To enable an interactive experience, the light field content generated by the LF processing engine 530 may be modified in real time in response to sensory or tracking input. In other words, in some cases, the LF content cannot simply be read from the data storage 522.

[0144] Thus, in some configurations, data representing holographic content displayed by the LF display system 500 may be passed to the LF processing engine 530 in a vectorized data format ("vectorized data"). Vectorized data may be several orders of magnitude smaller than rasterized data. In addition, vectorized data provides high image quality while having a data set size that enables efficient sharing of data. For example, the vectorized data may be a sparse data set derived from a denser data set. Thus, based on how the sparse vectorized data is sampled from the dense rasterized data, the vectorized data may have an adjustable balance between image quality and data transfer size. The adjustable sampling used to generate the vectorized data enables optimization of image quality at a given network speed. Thus, the vectorized data enables efficient transmission of holographic content via the network interface 524. The vectorized data also enables real-time streaming of holographic content over commercial networks.

[0145] In summary, the LF processing engine 530 can generate holographic content derived from rasterized data accessed from the data storage device 522, vectorized data accessed from the data storage device 522, or vectorized data received via the network interface 524. In various configurations, the vectorized data may be encoded by an encoder before data transmission and decoded by a decoder within the LF controller 520 after reception. The encoder and decoder pair may be part of the same proprietary system codec. In some instances, the vectorized data is encoded for additional data security and performance improvements associated with data compression. For example, the vectorized data received via the network interface may be encoded vectorized data received from a holographic streaming media application. In some instances, the vectorized data may require a decoder, the LF processing engine 530, or both to access the information content encoded in the vectorized data. The encoder and / or decoder system may be available for consumer use or licensed to third-party vendors. Other instance encoding and / or decoding schemes may be used to transmit and / or present holographic content.

[0146] The vectorized data contains all the information for each sensory domain supported by the LF display system 500 in a manner that can support an interactive experience. For example, the vectorized data for an interactive holographic experience may include any vectorized features that can provide accurate physical effects for each sensory domain supported by the LF display system 500. The vectorized features may include any features that can be synthetically programmed, captured, computationally evaluated, etc. The LF processing engine 530 may be configured to convert the vectorized features in the vectorized data into rasterized data. The LF processing engine 530 may then project the holographic content converted from the vectorized data using the LF display component 510. In various configurations, vectorized features may include: one or more red / green / blue / alpha channels (RGBA) + depth images; multiple view images with or without depth information of varying resolutions, which may include a high-resolution center image and other views of lower resolution; material features such as albedo and reflectivity; surface normals; other optical effects; surface identification; geometric object coordinates; virtual camera coordinates; display plane position; lighting coordinates; tactile stiffness of the surface; tactile extensibility; tactile intensity; amplitude and coordinates of the sound field; environmental conditions; somatosensory energy vectors associated with mechanoreceptors for texture or temperature, audio; and any other sensory domain features. Many other vectorized features are also possible.

[0147] The LF display system 500 may also generate an interactive viewing experience. That is, the holographic content may respond to input stimuli containing information about the viewer's position, gestures, interactions, interactions with the holographic content, or other information from the viewer profile analysis module 528 and / or the tracking module 526. For example, in an embodiment, the LF display system 500 generates an interactive viewing experience using vectorized data for real-time performance management received via the network interface 524. In another example, if a holographic object needs to move immediately in a particular direction in response to a viewer interaction, the LF processing engine 530 may update the rendering of the scene so that the holographic object moves in the desired direction. This may require the LF processing engine 530 to use the vectorized data set to render a light field in real time based on a 3D graphics scene with appropriate object placement and movement, collision detection, occlusion, color, shadows, lighting, etc., so as to properly respond to viewer interactions. The LF processing engine 530 converts the vectorized data into rasterized data for presentation by the LF display component 510. The LF display system 500 may employ various other encoding / decoding techniques that allow the LF display system to render holographic content in near real time.

[0148] The rasterized data contains holographic content instructions and sensory instructions (display instructions) representing the real-time performance. The LF display component 510 simultaneously projects the holographic and sensory content of the real-time performance by executing the display instructions. The LF display system 500 monitors viewer interactions (e.g., voice responses, touch, etc.) with the presented real-time performance through the tracking module 526 and the viewer profiling module 528. In response to the viewer interaction, the LF processing engine can create an interactive experience by generating additional holographic and / or sensory content for display to the viewer.

[0149] To illustrate, consider an example embodiment of an LF display system 500 that includes an LF processing engine 530 that generates a plurality of holographic objects representing balloons that fall from the ceiling of a venue during a performance. A viewer can move to touch the holographic objects representing the balloons. Accordingly, a tracking system 580 tracks the movement of the viewer's hand relative to the holographic object. The viewer's movement is recorded by the tracking system 580 and sent to the controller 520. The tracking module 526 continuously determines the movement of the viewer's hand and sends the determined movement to the LF processing engine 530. The LF processing engine 530 determines the placement of the viewer's hand in the scene, adjusts the real-time rendering of the graphics to include any desired changes (such as positioning, color, or occlusion) in the holographic object. The LF processing engine 530 instructs the LF display component 510 (and / or the sensory feedback system 570) to generate a tactile surface using a body tactile projection system (e.g., using an ultrasonic speaker). The generated tactile surface corresponds to at least a portion of the holographic object and occupies substantially the same space as some or all of the external surfaces of the holographic object. The LF processing engine 530 uses the tracking information to dynamically instruct the LF display component 510 to move the position of the tactile surface together with the position of the rendered holographic object, so that the viewer is given both the visual and tactile perception of touching the balloon. More simply, when the viewer watches his hand touching the holographic balloon, the viewer simultaneously feels tactile feedback indicating that his hand has touched the holographic balloon, and the balloon changes position or movement in response to the touch. In some instances, rather than presenting the interactive balloons in a performance accessed from the data storage device 522, the interactive balloons may be received as part of the holographic content received from a live streaming application via the network interface 524. In other words, the holographic content displayed by the LF display system 500 may be a live stream of holographic content.

[0150] The LF processing engine 530 can provide holographic content displayed to viewers in the venue before, during and / or after the performance to enhance the venue experience. The holographic content can be provided by the publisher of the performance, provided by the venue, provided by the advertiser, generated by the LF processing engine 530, etc. The holographic content can be content associated with the performance, the genre of the performance, the location of the venue, advertising, etc. In any case, the holographic content can be stored in the data storage device 522, or streamed to the LF display system 500 in a vectorized format through the network interface 524. For example, the performance can be displayed on the wall in a venue enhanced with an LF display module. The publisher of the performance can provide holographic content presented on the wall display before the performance begins. The LF processing engine 530 accesses the holographic content and presents the accessed content from the display to the wall of the venue before the performance begins. In another example, the venue with the LF display system 500 is located in San Francisco. If no performance-specific content is provided, the venue stores a holographic representation of the Golden Gate Bridge presented in the venue before the performance. Here, because no show-specific holographic content is provided, the LF processing engine 530 accesses and presents the Golden Gate Bridge in the venue. In another example, an advertiser has provided holographic content of its product as an advertisement to the venue to be displayed after the show. After the show is over, the LF processing engine 530 presents the advertisement to the viewer as he or she leaves the venue. In other examples, as described below, the LF processing engine can dynamically generate holographic content displayed on the wall of the theater.

[0151] The LF processing engine 530 can also modify the holographic content to fit the venue where the holographic content is being presented. For example, not every venue has the same size, the same number of seats, or the same technical configuration. Therefore, the LF processing engine 530 can modify the holographic content so that it will be appropriately displayed in the venue. In one embodiment, the LF processing engine 530 can access a configuration file of the venue, including the layout, resolution, field of view, other technical specifications, etc. of the venue. The LF processing engine 530 can render and present the holographic content based on the information contained in the configuration file.

[0152] The LF processing engine 530 may also create holographic content for display by the LF display system 500. Importantly, here, creating holographic content for display is different from accessing or receiving holographic content for display. In other words, when creating content, the LF processing engine 530 generates brand new content for display, rather than accessing previously generated and / or received content. The LF processing engine 530 may use information from the tracking system 580, the sensory feedback system 570, the viewer profile analysis module 528, the tracking module 526, or some combination thereof to create holographic content for display. In some instances, the LF processing engine 530 may access information from elements of the LF display system 500 (e.g., tracking information and / or viewer profiles), create holographic content based on the information, and in response, display the created holographic content using the LF display system 500. When displayed by the LF display system 500, the created holographic content may be enhanced with other sensory content (e.g., touch, audio, or smell). In addition, the LF display system 500 may store the created holographic content so that it can be displayed in the future.

[0153] Dynamic content generation for LF display systems

[0154] In some embodiments, the LF processing engine 530 incorporates an artificial intelligence (AI) model to create holographic content for display by the LF display system 500. The AI ​​model may include a supervised or unsupervised learning algorithm, including but not limited to a regression model, a neural network, a classifier, or any other AI algorithm. The AI ​​model may be used to determine viewer preferences based on viewer information recorded by the LF display system 500 (e.g., by the tracking system 580), which may include information about the viewer's behavior.

[0155] The AI ​​model can access information from the data storage device 522 to create holographic content. For example, the AI ​​model can access viewer information from one or more viewer profiles in the data storage device 522, or can receive viewer information from various components of the LF display system 500. To illustrate, the AI ​​model can determine that viewers like to watch holographic content in which performers wear bow ties. The AI ​​model can determine the preference based on a group of viewers' positive reactions or responses to previously viewed holographic content containing actors wearing bow ties. In other words, the AI ​​model can create holographic content personalized for a group of viewers based on the learned preferences of those viewers. Thus, for example, the AI ​​model can use the LF display system 500 to incorporate bow ties into actors displayed in holographic content viewed by a group of viewers. The AI ​​model can also store the learned preferences of each viewer in the viewer profile storage of the data storage device 522. In some instances, the AI ​​model can create holographic content for a single viewer rather than a group of viewers.

[0156] An example of an AI model that can be used to identify characteristics of a viewer, identify reactions, and / or generate holographic content based on the identified information is a convolutional neural network model with node layers, where the value at a node of the current layer is a transformation of the value at a node of the previous layer. The transformation in the model is determined by a set of weights and parameters connecting the current layer and the previous layer. For example, the AI ​​model may include five node layers: layers A, B, C, D, and E. The transformation from layer A to layer B is given by the function W 1 Given, the transformation from layer B to layer C is given by W 2 Given, the transformation from layer C to layer D is given by W 3 is given by , and the transformation from layer D to layer E is given by W 4 In some instances, the transformation can also be determined by a set of weights and parameters used to transform between previous layers in the model. For example, the transformation W from layer D to layer E is 4 The transformation W from layer A to B can be completed based on 1 Parameters.

[0157] The input to the model can be an image encoded onto convolutional layer A acquired by the tracking system 580, and the output of the model is holographic content decoded from output layer E. Alternatively or in addition, the output can be determined characteristics of a viewer in the image. In this instance, the AI ​​model identifies latent information in the image that represents viewer characteristics in identification layer C. The AI ​​model reduces the dimensionality of convolutional layer A to the dimensionality of identification layer C to identify any features, actions, responses, etc. in the image. In some instances, the AI ​​model then increases the dimensionality of identification layer C to generate holographic content.

[0158] The image from the tracking system 580 is encoded into the convolutional layer A. The image input in the convolutional layer A can be correlated with various characteristics and / or reaction information in the identification layer C, etc. The relevant information between these elements can be retrieved by applying a set of transformations between the corresponding layers. That is, the convolutional layer A of the AI ​​model represents the encoded image, and the identification layer C of the model represents a smiling viewer. The transformation W can be applied to the image input in the convolutional layer A. 1 and W 2 The pixel values ​​of the image in the space of the convolutional layer A are applied to identify smiling viewers in a given image. The weights and parameters used for the transformation may indicate the relationship between the information contained in the image and the identification of the smiling viewer. For example, the weights and parameters may be quantifications of the shape, color, size, etc. contained in the information representing the smiling viewer in the image. The weights and parameters may be based on historical data (e.g., previously tracked viewers).

[0159] The smiling viewers in the image are identified in the identification layer C. The identification layer C represents the smiling viewers identified based on the underlying information about the smiling viewers in the image.

[0160] The identified smiling viewer in the image can be used to generate holographic content. To generate holographic content, the AI ​​model starts at the identification layer C and transforms W 2 and W 3 The value applied to a given identified smiling viewer in the identification layer C. The transformation produces a set of nodes in the output layer E. The weights and parameters used for the transformation can indicate the relationship between the identified smiling viewer and specific holographic content and / or preferences. In some cases, the holographic content is output directly from the nodes of the output layer E, while in other cases, the content generation system decodes the nodes of the output layer E into the holographic content. For example, if the output is a set of identified characteristics, the LF processing engine can use the characteristics to generate holographic content.

[0161] In addition, the AI ​​model may include layers referred to as intermediate layers. An intermediate layer is a layer that does not correspond to an image, does not identify features / reactions, etc., or does not generate holographic content. For example, in a given instance, layer B is an intermediate layer between convolution layer A and identification layer C. Layer D is an intermediate layer between identification layer C and output layer E. The hidden layer is a potential representation of different aspects of the identification that cannot be observed in the data, but can control the relationship between image elements when identifying features and generating holographic content. For example, a node in a hidden layer may have a strong connection (e.g., a large weight value) with an input value and an identification value that share the commonality of "happy people smiling". As another example, another node in a hidden layer may have a strong connection with an input value and an identification value that share the commonality of "scared people screaming". Of course, there are any number of connections in a neural network. In addition, each intermediate layer is a combination of functions, such as residual blocks, convolution layers, pooling operations, skip connections, concatenation, etc. Any number of intermediate layers B can be used to reduce the convolution layer to the identification layer, and any number of intermediate layers D can be used to increase the identification layer to the output layer.

[0162] In one embodiment, the AI ​​model comprises a deterministic method that has been trained with reinforcement learning (thereby creating a reinforcement learning model). The model is trained to improve the quality of performance using measurements from the tracking system 580 as input and changes to the created holographic content as output.

[0163] Reinforcement learning is a machine learning system in which the machine learns "what to do" - how to map situations to actions - so as to maximize a digital reward signal. The learner (e.g., LF processing engine 530) is not told which actions to take (e.g., generate a prescribed holographic content), but rather discovers which actions produce the highest reward by trying the actions (e.g., improving the quality of the holographic content by getting more people to cheer). In some cases, an action can affect not only the immediate reward, but also the next situation, and therefore all subsequent rewards. These two features - trial and error search and delayed rewards - are two notable features of reinforcement learning.

[0164] Reinforcement learning is defined not by characterizing the learning method, but by characterizing the learning problem. Basically, the reinforcement learning system captures those important aspects of the problem faced by the learning agent in interacting with its environment to achieve a goal. That is, in the example of generating a song for a performer, the reinforcement learning system captures information about the viewers in the venue (e.g., age, personality, etc.). This agent senses the state of the environment and takes actions that affect the state to achieve one or more goals (e.g., create a pop song that will make the viewers cheer). In its most basic form, the formulation of reinforcement learning includes three aspects of the learner: feelings, actions, and goals. Continuing with the song example, the LF processing engine 530 senses the state of the environment through the sensors of the tracking system 580, displays holographic content to the viewers in the environment, and achieves a goal that is a measure of the viewer's reception of the song.

[0165] One of the challenges that arise in reinforcement learning is the trade-off between exploration and exploitation. To increase the reward in the system, a reinforcement learning agent prefers actions that have been tried in the past and found to be effective in producing rewards. However, to discover actions that produce rewards, the learning agent chooses actions that were not previously chosen. The agent "exploits" information that it already knows to obtain rewards, but it also "explores" information to make better action choices in the future. The learning agent tries various actions and gradually prefers those that appear to be best as it continues to try new actions. On a stochastic task, each action is typically tried multiple times to get a reliable estimate of its expected reward. For example, if the LF processing engine creates holographic content that the LF processing engine knows will cause a viewer to laugh after a long period of time, the LF processing engine can change the holographic content so that the time until the viewer laughs is reduced.

[0166] Furthermore, reinforcement learning considers the entire problem of goal-directed agents interacting with uncertain environments. Reinforcement learning agents have clear goals, can sense aspects of their environment, and can choose actions that receive high rewards (i.e., roaring at the crowd). Furthermore, agents typically operate despite significant uncertainty in the environment they face. When reinforcement learning involves planning, the system addresses the interaction between planning and real-time action selection, as well as how to acquire and improve elements of the environment. In order for reinforcement learning to make progress, important subproblems must be separated and studied, and subproblems play a clear role in the complete interactive goal-seeking agent.

[0167] The reinforcement learning problem is a framework for machine learning problems in which interactions are processed and actions are performed to achieve a goal. Learners and decision makers are called agents (e.g., LF processing engine 530). The things that the agent interacts with (including everything outside the agent) are called the environment (e.g., viewers in the venue, etc.). The two interact continuously, with the agent choosing actions (e.g., creating holographic content) and the environment responding to these actions and presenting new situations to the agent. The environment also brings rewards, which are special numerical values ​​that the agent tries to maximize over time. In one context, the reward plays the role of maximizing the viewer's positive response to the holographic content. The complete specification of the environment defines a task, which is an instance of a reinforcement learning problem.

[0168] To provide more context, an agent (e.g., LF processing engine 530) interacts with the environment at each discrete time step in a series of discrete time steps (i.e., t = 0, 1, 2, 3, etc.). At each time step t, the agent receives some environment state s t A representation of (eg, measurements from tracking system 580). State s t In S, where S is the set of possible states. Based on state s t At time step t, the agent chooses an action at (e.g., causing the executor to split). t ), where A(s t ) is the set of possible actions. One time state later (partially as a result of its actions), the agent receives a numerical reward r t+1 . Status t+1 Within R, where R is the set of possible rewards. Once the agent receives a reward, it chooses a new state s t+1 .

[0169] At each time step, the agent implements a mapping from states to the probability of choosing each possible action. This mapping is called the agent’s policy and is denoted by π t , where π t (s,a) is if s t=s, then a t = a probability. Reinforcement learning methods can determine how an agent changes its policy as a result of the states and rewards produced by the agent's actions. The agent's goal is to maximize the total number of rewards received over time.

[0170] This reinforcement learning framework is very flexible and can be applied to many different problems in many different ways (e.g., generating holographic content). The framework proposes that, regardless of the details of the sensory, memory, and control devices, any problem (or purpose) of learning goal-directed behavior can be reduced to three signals passed back and forth between an agent and its environment: a signal representing the choice made by the agent (the action), a signal representing the rationale for making the choice (the state), and a signal defining the agent's goal (the reward).

[0171] Of course, the AI ​​model can include any number of machine learning algorithms. Some other AI models that can be employed are linear and / or logistic regression, classification and regression trees, k-means clustering, vector quantization, etc. In any case, the LF processing engine 530 typically takes input from the tracking module 526 and / or the viewer profile analysis module 528 and in response, the machine learning model creates holographic content. Similarly, the AI ​​model can guide the rendering of the holographic content.

[0172] In an example, the LF processing engine 530 creates a virtual pop singer. The LF processing engine 530 can create a virtual pop singer using information contained in the viewer profile stored in the data storage device 522. For example, the information contained in the stored viewer profile indicates that a large number of viewers like pop music sung by avant-garde female singer in the mid-20s. Thus, the LF processing engine 530 creates a virtual pop singer displayed by the LF display system 500 as avant-garde female singer in the mid-20s. More specifically, the LF processing engine 530 accesses the viewer profiles of the viewers in the venue. The LF processing engine 530 parameterizes (e.g., quantifies) the information in each viewer profile. For example, the LF processing engine 530 can quantify characteristics such as the age, location, gender, etc. of the viewer. In addition, the LF processing engine 530 can parameterize other information contained in the viewer profile. For example, if the viewer profile indicates that the viewer watched four performances of a female singer, the content creation module can quantify this trend (e.g., generate a score indicating the viewer's interest in the female singer). The LF processing engine 530 inputs the parameterized user profile into an AI model (e.g., a neural network) configured to generate characteristics of a virtual performer based on the input parameters, and receives the characteristics of the performer in response. The LF processing engine 530 then inputs the characteristics of the virtual performer into an AI model (e.g., a procedural generation algorithm) configured to generate a performer given a set of characteristics, and generates a virtual performer that is an avant-garde female singer in the mid-20s in response. In addition, the LF processing engine 530 can create holographic content (e.g., songs, performances, etc.) that will follow the persona of the virtual pop singer. For example, the content generation module 530 can create a song about a jealous ex-boyfriend for a virtual female singer in the mid-20s. More specifically, the LF processing engine 530 can access the characteristics of the virtual performer and information about the viewer, and input the information into an AI model (e.g., a recurrent neural network "RNN"). Similarly, the characteristics and information can be parameterized (e.g., using classification and regression trees) and input into the RNN. Here, the RNN can be trained using real-world songs with similar input parameters. Thus, the RNN generates a virtual song for the virtual singer to viewers in the venue, wherein the virtual song has similar characteristics to one or more songs written by other avant-garde female singers in the mid-1920s.

[0173] The LF processing engine 530 can create holographic content based on the performance shown in the venue. For example, the performance shown in the venue can be associated with a set of metadata describing the performance characteristics. The metadata may include, for example, background, genre, performer, performance type, theme, title, running time, etc. The LF processing engine 530 can access any of the metadata describing the performance and generate holographic content present in the venue in response. For example, a movie titled "The Last Merman" will be played in a venue enhanced with the LF display system 500. The LF processing engine 530 accesses the metadata of the performance to create holographic content for the wall of the venue before the performance begins. Here, the metadata includes that the background is underwater and the genre is romantic. The LF processing engine 530 inputs the metadata into the AI ​​model and receives holographic content displayed on the wall of the venue in response. In this example, the LF processing engine 530 creates a seaside sunset displayed on the wall of the venue before the performance starts playing.

[0174] In an example, the LF processing engine 530 creates holographic content based on viewers present at a venue containing the LF display system 500. For example, a group of viewers enter the venue to watch a performance enhanced by holographic content displayed by the LF display system 500. The viewer profile analysis module 528 generates viewer profiles for the viewers in the venue, as well as an aggregate viewer profile representing all viewers in the venue. The LF processing engine 530 accesses the aggregate viewer profile and creates holographic content to display to the viewers in the venue. For example, the viewers in the venue are a group of couples watching a performer known for romantic ballads, and therefore, the aggregate viewer profile contains information indicating that they may like holographic content that is commensurate with a couple on a date (e.g., through parameterization and input to an AI model). Therefore, the LF processing engine 530 generates holographic content so that the venue has a more romantic atmosphere (e.g., candles, dim lights, music by Marvin Gaye, etc.).

[0175] In some instances, the LF processing engine 530 can create holographic content based on pre-existing content. Here, the pre-existing content can be previously recorded songs, existing real-world performers, etc. For example, the viewer's favorite band is "The Town People", and they want to watch a concert of their favorite Town People songs. Therefore, the LF processing engine 530 creates a performance of Town People singing "DMCA" and "In the Gravy". In this example, the LF processing engine 530 can access existing songs and models from the Town People singer (e.g., stored in the data storage device 522) and use the content to create a new performance for display by the LF display system 500. More specifically, the content creation system can access the songs and models from the data storage device 522. The LF processing engine 530 inputs the songs and models into an AI model (e.g., a procedural generation algorithm) configured to create a performance of, for example, a rock band, and in response, the AI ​​model outputs holographic content of the performance. In some cases, because the LF processing engine 530 creates holographic content based on copyrighted content, the viewer and / or the LF processing engine 530 may pay a fee to the copyright holder.

[0176] In an example, the LF processing engine 530 creates holographic content based on the response of viewers watching the performance. For example, viewers in the venue are watching a performance in a venue enhanced by the LF display system 500. The tracking module 526 and the viewer profile analysis module 528 monitor the viewer's reaction to watching the performance. For example, the tracking module 526 can obtain an image of the viewer while the viewer is watching the performance. The tracking module 526 identifies the viewer, and the viewer profile analysis module 528 can use a machine vision algorithm to determine the viewer's reaction based on the information contained in the image. For example, an AI model can be used to identify whether a viewer watching the performance is smiling, and therefore, the viewer profile analysis module 528 can indicate in the viewer profile whether the viewer has a positive or negative response to the performance based on the smile. Other reactions can also be determined. The tracking module can determine information about the viewer, the information includes the viewer's positioning, the viewer's movement, the viewer's gestures, the viewer's expression, the viewer's age, the viewer's gender, or the clothes worn by the viewer. This information can be shared with the viewer profile analysis module 528 to generate a viewer profile. By way of illustration, the LF processing engine 530 generates a comedy routine performed by a virtual comedian. While the LF display system 500 displays the performance, the tracking system 580 and the viewer profile analysis module 528 monitor the reactions of the viewers. In this case, the viewers do not laugh at the "bad jokes" told by the virtual comedian. In response, the LF processing engine 530 modifies the comedy performance so that the virtual comedian begins to tease the specific viewer. Here, the LF processing engine 530 can input the reaction into an AI model (e.g., a reinforcement learning model) that is configured to induce laughter in viewers watching the holographic content. Based on the responses and characteristics of the viewers in the audience, the LF processing engine 530 changes the way the jokes are told. For example, the virtual comedian may start making fun of the leopard print pants worn by a specific viewer. Here, the model can be trained using previously performed comedy performances and recorded user responses to these performances.

[0177] In some instances, the LF processing engine 530 can create holographic content based on pre-existing content. Here, the pre-existing content can be previously recorded songs, existing real-world performers, etc. For example, the viewer's favorite band is "The Town People", and they want to watch a concert of their favorite Town People songs. Therefore, the LF processing engine 530 creates a performance of Town People singing "DMCA" and "In the Gravy". In this example, the LF processing engine 530 can access existing songs and models from the Town People singer (e.g., stored in the data storage device 522) and use the content to create a new performance for display by the LF display system 500. More specifically, the content creation system can access the songs and models from the data storage device 522. The LF processing engine 530 inputs the songs and models into an AI model (e.g., a procedural generation algorithm) configured to create a performance of, for example, a rock band, and in response, the AI ​​model outputs holographic content of the performance. In some cases, because the LF processing engine 530 creates holographic content based on copyrighted content, the viewer and / or the LF processing engine 530 may pay a fee to the copyright holder.

[0178] In a similar instance, the LF processing engine 530 can create holographic content based on previously existing or provided advertising content. In other words, for example, the LF processing engine 530 can request an advertisement from a network system via the network interface 524, in response, the network system provides holographic content, and the LF processing engine 530 creates a holographic content for display containing the advertisement. Some instances of advertisements may include products, text, videos, etc. Advertisements can be presented to specific viewing bodies based on viewers in the viewing body. Similarly, holographic content can enhance performances with advertisements (e.g., product placement). Most generally, as previously described, the LF processing engine 530 can create advertising content based on any of the characteristics and / or reactions of viewers in the venue.

[0179] The foregoing examples of creating content are not limiting. In the broadest sense, the LF processing engine 530 creates holographic content for display to a viewer of the LF display system 500. The holographic content may be created based on any of the information contained in the LF display system 500.

[0180] Holographic Content Distribution Network

[0181] Figure 5BAn example LF performance network 550 is shown according to one or more embodiments. One or more LF display systems may be included in the LF performance network 550. The LF performance network 550 includes any number of LF display systems (e.g., 500A, 500B, and 500C) coupled to each other via a network 552, an LF generation system 554, and a network system 556. In other embodiments, the LF performance network 550 includes additional or fewer entities than those described herein. Similarly, functionality may be distributed between different entities in a manner different from that described herein.

[0182] In the illustrated embodiment, the LF performance network 550 includes LF display systems 500A, 500B, and 500C, which can receive holographic content via a network 552 and display the holographic content to a viewer. The LF display systems 500A, 500B, and 500C are collectively referred to as LF display system 500.

[0183] The LF generation system 554 is a system for generating holographic content for display in a performance venue including an LF display system. In other words, the LF generation system is configured to generate any of the LF content described herein. The holographic content may be a performance or may be a holographic content that enhances a traditional performance. The LF generation system 554 may include a light field recording component composed of any number of sensors and / or processors to record energy data of an event, and a processing engine configured to convert this recorded energy data into holographic content. For example, the sensor of the light field recording component may include a camera for recording images, a microphone for recording audio, a pressure sensor for recording interactions with an object, and the like. In some instances, the light field recording component of the LF generation system 554 includes one or more recording modules (e.g., an LF display module configured to record energy data from an event, or a simple 2D camera for capturing event images) positioned around an area (e.g., a performance venue) to record an event from multiple viewpoints. In this case, the processing engine of the LF generation system 554 is configured to convert energy from multiple viewpoints into holographic content. In some examples, the light field recording component includes two or more two-dimensional recording systems that are used by the processing engine to convert multiple viewpoints of an event into three-dimensional holographic content. The light field recording component may also include other sensors, such as a depth sensor and / or a plenoptic camera.

[0184] The processing engine of the LF generation system 554 may also generate synthetic light field data from computer generated image (CGI) animations. The synthetic light field data may come from, for example, a virtual world or an animated film, and may be used to enhance the sensory data from the light field recording component. The processing engine of the LF generation system 554 combines the recorded sensory information, the synthetic data, and encodes the information into holographic content and sensory content. The LF generation system 554 may transmit the encoded holographic content to one or more of the LF display systems 500 for display to a viewer. As previously discussed, to achieve efficient transmission speeds, the data of the LF display systems 500A, 500B, 500C, etc. may be transmitted as vectorized data over the network 552.

[0185] More generally, the LF generation system 554 generates holographic content for display in the venue by using any recorded sensory data or synthetic data of an event that may be projected by the LF display system when presenting a performance. For example, the sensory data may include recorded audio, recorded images, recorded interactions with objects, etc. Many other types of sensory data may be used. To illustrate, the recorded visual content may include: 3D graphics scenes, 3D models, object placement, textures, colors, shading, and lighting; 2D performance data that may be converted to holographic form using an AI model and a similar performance conversion of a larger data set; multi-view camera data from a camera rig of many cameras with or without depth channels; plenoptic camera data; CG content; or other types of recorded sensory data of an event as described herein.

[0186] In various examples, the event recorded by one or more sensors on one or more energy fields can be a concert, a show, a scene, a single performance, a band performance, an actor performance, a dancer performance, a magician performance, or another type of event that can be presented in a performance venue. The performance venue can include one or more of a performance hall, an event space, a theater, a concert hall, a studio, or a stage.

[0187] In some configurations, the LF generation system 554 may use a dedicated encoder to perform encoding operations that reduce the sensory data recorded for the performance to a vectorized data format as described above. That is, encoding the data into vectorized data may include image processing, audio processing, or any other calculations that may produce a reduced data set that is more easily transmitted over the network 552. The encoder may support formats used by performance production industry professionals. In other configurations, the LF generation system may transmit the performance content to the network system 556 and / or LF display system without encoding the content.

[0188] Each LF display system (e.g., 500A, 500B, 500C) may receive encoded data from a network 552 via a network interface 524. In this example, each LF display system includes a decoder to decode the encoded LF display data. More specifically, the LF processing engine 530 generates rasterized data for the LF display component 510 by applying a decoding algorithm provided by the decoder to the received encoded data. In some examples, the LF processing engine may additionally generate rasterized data for the LF display component using inputs from a tracking module 526, a viewer profile analysis module 528, and a sensory feedback system 570 as described herein. The rasterized data generated for the LF display component 510 reproduces the holographic content recorded by the LF generation system 554. Importantly, each LF display system 500A, 500B, and 500C generates rasterized data suitable for a specific configuration of the LF display component in terms of geometry, resolution, etc. In some configurations, the encoding and decoding processes are part of a proprietary encoding / decoding system pair (or 'codec') that may be provided to display consumers or licensed by a third party. In some cases, the encoding / decoding system pair may be implemented as a proprietary API that may provide a common programming interface to content creators.

[0189] In some configurations, various systems in the LF performance network 550 (e.g., LF display system 500, LF generation system 554, etc.) may have different hardware configurations. The hardware configuration may include an arrangement of physical systems, energy sources, energy sensors, tactile interfaces, sensory capabilities, resolutions, LF display module configurations, or any other hardware description of the systems in the LF performance network 550. Each hardware configuration may generate or use sensory data in different data formats. Therefore, the decoder system may be configured to decode encoded data for the LF display system to be presented on the LF display system. For example, an LF display system (e.g., LF display system 500A) having a first hardware configuration receives encoded data from an LF generation system (e.g., LF generation system 554) having a second hardware configuration. The decoding system accesses information describing the first hardware configuration of the LF display system 500A. The decoding system decodes the encoded data using the accessed hardware configuration so that the decoded data can be processed by the LF processing engine 530 of the receiving LF display system 500A. Although recorded in the second hardware configuration, the LF processing engine 530 generates and presents rasterized content for the first hardware configuration. In a similar manner, regardless of the hardware configuration, the holographic content recorded by the LF generation system 554 can be presented by any LF display system (e.g., LF display system 500B, LF display system 500C). Various other aspects that may be included in the hardware configuration may include: resolution, number of rays per degree of projection, field of view, deflection angle on the display surface, and dimensions of the display surface, etc. In addition, the hardware configuration may also include: multiple display panels of the LF display assembly, relative orientation of the display panels, height of the display panels, width of the display panels, and layout of the display panels.

[0190] Similarly, the various systems in the LF performance network 550 may have different geometric orientations. The geometric orientation reflects the physical size, layout, and arrangement of the various modules and systems included in the LF display system. Therefore, the decoder system may be configured to decode the encoded data for the LF display system in the geometric configuration to be presented on the LF display system. For example, an LF display system having a first geometric configuration (e.g., LF display system 500A) receives encoded data from an LF generation system having a second geometric configuration (e.g., LF generation system 554). The decoding system accesses information describing the first geometric configuration of the LF display system 500A. The decoding system decodes the encoded data using the accessed geometric configuration so that the decoded data can be processed by the LF processing engine 530 of the receiving LF display system 500A. Although recorded in the second geometric configuration, the LF processing engine 530 generates and presents rasterized content for the first geometric configuration. In a similar manner, regardless of the geometric configuration, the holographic content recorded by the LF generation system 554 can be presented by any LF display system (e.g., LF display system 500B, LF display system 500C). Various other aspects that may be included in the geometric configuration may include: multiple display panels (or surfaces) of the LF display assembly, relative orientation of the display panels.

[0191] Similarly, various venues in the LF performance network 550 may have different configurations. The venue configuration reflects any one of the number and / or positioning of holographic object bodies, the number and / or positioning of viewing bodies, and the number and / or positioning of viewing positions relative to the LF display system. Therefore, the decoder system may be configured to decode the encoded data for the LF display system in the venue to be presented in the LF display system. For example, an LF display system (e.g., LF display system 500A) in a first venue receives encoded data from an LF generation system (e.g., LF generation system 554) recorded in a different venue (or some other space). The decoding system accesses information describing the venue. The decoding system decodes the encoded data using the accessed venue configuration so that the decoded data can be processed by the LF processing engine 530 installed in the venue. Although recorded in different locations, the LF processing engine 530 still generates and presents content for the venue.

[0192] The network system 556 is any system configured to manage the transmission of holographic content between systems in the LF performance network 550. For example, the network system 556 may receive a request for holographic content from the LF display system 500A and facilitate the transmission of holographic content from the LF generation system 554 to the LF display system 500A. The network system 556 may also store holographic content for transmission to and / or storage by other LF display systems 500 in the LF performance network 550, viewer profiles, holographic content, etc. The network system 556 may also include an LF processing engine 530 that may create holographic content as previously described.

[0193] The network system 556 may include a digital rights management (DRM) module to manage digital rights of the holographic content. For example, the LF generation system 554 may transmit the holographic content to the network system 556, and the DRM module may encrypt the holographic content using a digital encryption format. In other examples, the LF generation system 554 encodes the recorded light field data into a holographic content format that can be managed by the DRM module. The network system 556 may provide the digitally encrypted key to the LF display system so that each LF display system 500 can decrypt and then display the holographic content to the viewer. Most generally, the network system 556 and / or the LF generation system 554 encode the holographic content, and the LF display system can decode the holographic content.

[0194] The network system 556 may act as a repository for previously recorded and / or created holographic content. Each piece of holographic content may be associated with a transaction fee, which, when received, causes the network system 556 to transmit the holographic content to the LF display system 500 that provides the transaction fee. For example, the LF display system 500A may request access to holographic content via the network 552. The request includes a transaction fee for the holographic content. In response, the network system 556 transmits the holographic content to the LF display system for display to the viewer. In other instances, the network system 556 may also act as a subscription service for holographic content stored in the network system. In another instance, the LF generation system 554 records light field data of a performance in real time and generates holographic content representing the performance. The LF display system 500 transmits a request for holographic content to the LF generation system 554. The request includes a transaction fee for the holographic content. In response, the LF generation system 554 transmits the holographic content for simultaneous display on the LF display system 500. The network system 556 may act as an intermediary for exchanging transaction fees and / or managing the flow of holographic content data across the network 552 .

[0195] Network 552 represents the communication paths between systems in LF performance network 550. In one embodiment, the network is the Internet, but may be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a cloud computing network, a private network or a virtual private network, and any combination thereof. In addition, all or some links may be encrypted using conventional encryption techniques, such as secure socket layer (SSL), encrypted HTTP and / or a virtual private network (VPN). In another embodiment, the entities may use custom and / or dedicated data communication techniques instead of or in addition to the techniques described above.

[0196] Example site

[0197] Figure 6-8 Several example venues are shown in which holographic content can be displayed using an LF display system (e.g., LF display system 500). The holographic content of a performance displayed in the venue ("performance content") can be any holographic content described herein. The performance content can be displayed in place of a live performance by a real performer. Within the venue, any number of viewers are located within any number of viewing volumes. The LF display system is configured to display the performance content in a holographic object volume ("performance volume") so that viewers in the viewing volume perceive the performance content. Typically, the LF display system in the venue includes an array of LF display modules 210 surrounding the performance volume, the array of light field modules generating a seamless multilateral LF display. Therefore, the performance volume can be an aggregated holographic object volume of the array of LF display modules 210.

[0198] Figure 6 A side view of a venue 600 is shown, which is a conventional theater that has been enhanced with an LF display system, according to one or more embodiments. In the theater, a stage 602 and a wall 604 surrounding the stage are arrayed with an array 640 of LF display modules, such that the area above the stage is a performance volume 610 of the LF display system. For clarity, the performance volume 610 (e.g., holographic object volume 255) is shown as a bounded square, but the performance volume 610 shown is only a portion of the actual performance volume. For example, the performance volume can extend into the wall 604 or the stage 602. In Figure 6 In the embodiment of the present invention, the LF display system is an embodiment of the LF display system 500. In addition, the LF display modules in the array 640 are embodiments of the LF display assembly 510.

[0199] Here, the theater house is three-story, but it can include any number of floors. Each floor includes multiple viewing positions (e.g., viewing positions 622A, 622B) for viewers to view the performance content 630 in the performance body 610. As shown in the figure, the performance content is a real image 632, but it can also be a virtual image. The viewing position 622 in each floor is included in the viewing body (e.g., 620A, 620B, and 620C) of the LF display system. The LF display system can display the same or different performance content 630 to the viewers in each viewing body 620. For example, as described below, the viewer in the viewing position 622A in the viewing body 620A located at the bottom layer can see the performance content 630 that is different from the content seen by the viewer in the viewing position 622B in the viewing body 620B located at the middle layer.

[0200] In other embodiments, the theater may be configured differently. For example, the stage may have a slope, the theater may contain an orchestra or standing room area, etc. Any surface in the theater (e.g., proscenium arch, ceiling, backdrop, curtains, etc.) may be included in the array 640 of LF display modules. Furthermore, the theater may have additional layers and viewing volumes, and these layers and viewing volumes may be arranged in any number of configurations.

[0201] As a background example, the venue 600 shown is a theater in San Francisco that charges a fee to show live performance content of a famous performer "NJ". However, it is worth noting that NJ is performing at a venue in New York City. The venue in New York City includes an LF generation system 554 for recording and transmitting NJ's performance as performance content 630 to other venues via a network (e.g., network 552). The venue 600 in San Francisco includes an LF display system 500, which is configured to display the received performance content 630 to the viewer at a viewing position 622 in a viewing body 620. A transaction fee is paid (e.g., to the network system 556 or the LF generation system 554) to receive the performance content. In various examples, the owner of the venue 600, the attendee of the venue 600, the performance manager, or any other agent can pay the fee. The performance content 630 allows viewers in San Francisco to perceive NJ as if NJ is performing live in the performance body 610 in front of them. This allows viewers in San Francisco to watch live performances without having to travel to New York City.

[0202] In some embodiments, the venue 600 charges an admission fee to view NJ's live show through the LF display system 500. Each viewing location 622 may have a different admission fee, and typically the admission fee to the bottom viewing volume 620A is more expensive than the admission fee to the top viewing volume 620C. The LF display system 500 may display different show content 630 to each viewing volume 620 based on the admission fee of the viewing volume 620. For example, it may cost more (e.g., processing power, energy, etc.) to fully render NJ's show in San Francisco than to partially render NJ's show. Therefore, the LF processing engine 530 may display only a portion of the show content 630 to the viewing volume 620C on the top floor of the theater, while playing all of the show content 630 to the viewing volume 620A on the bottom floor of the theater. For example, when only a portion of performance content 630 is displayed, LF processing engine 530 may display performance content 630 in only a portion of performance volume 610 rather than the entire performance volume 610, LF processing engine 530 may remove various aspects of performance content 630 (e.g., backup dancers, props, etc.), LF processing engine 530 may present performance content 630 for top viewing volume 620C at a lower resolution than for bottom viewing volume 620A, and so on.

[0203] Alternatively or additionally, the LF processing engine 530 may create holographic content to enhance the performance content 630 based on the admission fee of the viewing body 620. For example, the LF processing engine 530 may create (or access from the network system 556) advertisements to be displayed simultaneously with the performance content 630. The advertisements may be based on information obtained by the viewer profiling system 590 or the tracking system 580. For example, the LF processing engine 530 may access a viewer profile containing viewer characteristics and responses to the holographic content. The LF processing engine 530 accesses the advertisement from the data storage device 522 associated with the viewer's characteristics and responses and displays the advertisement. When the performance content 630 is displayed, the venue may charge a fee for displaying the advertisement from the advertiser. In another example, the LF processing engine 530 may enhance the performance content 630 with sponsored holographic content (e.g., product placement) instead of publicly displayed advertisements. For example, the LF processing engine 530 may replace the clothing worn by NJ during the New York City performance with clothing from an advertiser. Thus, a viewer watching the performance content 630 in a theater in San Francisco would perceive NJ as wearing clothing from the advertiser rather than clothing he would wear in New York City.

[0204] Fig. 7AA cross section of another venue 700A including a LF display system for displaying performance content to viewers at viewing positions in a viewing volume in accordance with one or more embodiments is shown. The venue allows the LF display system to display real-time holographic content and / or recorded or some other additional holographic content. The holographic content may include content representing a concert, a performance, a show, a scene, an event, etc. The holographic content may also include a singer, a band, an actor, a dancer, a comedian, a performer, etc. The holographic content may also include a representation of any background, a stage, a prop, an item of clothing, a costume, a musical instrument, etc. All venues described herein may be configured to display similar holographic content.

[0205] Here, venue 700A is designed and constructed to display performance content 730 rather than to increase the existing venue. As shown in the figure, Fig. 7A is a cross-section of a venue 600 in the shape of a cylindrical ring, wherein the LF display system 740 covers the outer surface of the inner cylinder of the ring. Fig. 7A , LF display system 740 is an embodiment of LF display assembly 510. Additionally, LF display system 740 is an embodiment of LF display system 500.

[0206] Venue 700A includes a plurality of viewing positions arranged in layers. Here, each layer is located on the inner surface of an outer cylinder having a radius greater than the radius of the inner cylinder. Viewing positions 722 in each layer are within a viewing volume (e.g., 720A, 720B, 720C, 720D, 720E, and 720F). Here, performance bodies (e.g., 710A, 710B, 710C, 710D, 710E, and 710F) are located between the inner wall of the outer cylinder and the outer wall of the inner cylinder. The performance body is positioned so that a viewer in viewing position 722 can perceive performance content 730 displayed in performance body 710. Again, for clarity, performance body 710 (e.g., holographic object body 255) is shown as a bounded square, but the performance body 710 shown is only a portion of the actual performance body. For example, the performance body can extend into the inner cylinder of venue 700A.

[0207] In the example shown, each performing volume 710 and a portion of the viewing volume 720 overlap in space. Although shown as partially overlapping, the performing volume and the viewing volume can completely overlap in space. As previously described, spatial overlap is an area where viewers can interact with performance content 730. In addition, users interacting with performance content 730 in the spatial overlap area can be monitored by the sensory feedback system 570 and the performance content 730 can be changed in response. In some cases, the venue 700 may charge a higher admission fee for viewing positions 722 in the area where viewers can interact with performance content 730.

[0208] A venue designed to display performance content using an array of LF display modules has several advantages over a venue enhanced with an array of LF display modules. For example, viewers in different viewing volumes of venue 700A view different performances (or different sub-portions of a performance), while viewers in viewing volume 620 of venue 600 view the same performance. Thus, the LF display system in venue 700 can generate a better viewing experience for each viewing volume 720. For example, a viewer in viewing volume 720A of venue 700 may be provided with the same view of performance content 730 as a viewer in viewing volume 720C. However, in venue 600, viewers in viewing volume 620C view performance content 730 farther away and below than viewers in viewing volume 620A. In this way, each viewer in venue 700A can have the best seat in the house.

[0209] In addition, considering the LF display parameters, for managing the rendering of performance content, the display to Fig. 7A and Figure 7B The performance content 730 of the viewing body 720 is displayed to Figure 6 Some LF display parameters may include, for example, resolution, field of view, projection distance, and viewing sub-volume. For example, Fig. 7A and Figure 7B The viewing volumes 720 in the venue can be designed so that their shapes are roughly equal and are spaced roughly similarly to the performance volumes 710. Therefore, since each viewing volume 720 is the same, the rendering of the performance content 730 to each viewing volume 720 is reduced. Figure 6 In the venue, each viewing body 620 is different and the performance content 630 needs to be rendered independently for each viewing body 620.

[0210] The venue may take other shapes so that the array of light field display modules can display the performance content in the performance volume to viewers in viewing positions within the viewing volume 720. Some example venues with different shapes may include domes, awnings, tunnels, spherical theaters, etc.

[0211] For example, Figure 7B A cross section of another venue 700B including an LF display system for displaying performance content to viewers at viewing positions in a viewing volume according to one or more embodiments is shown. The venue 700B is designed and constructed to display performance content 730 rather than augmenting an existing venue. As shown, Figure 7B is a cross section of site 700B. Figure 7B , LF display system 740 is an embodiment of LF display assembly 510. Additionally, LF display system 740 is an embodiment of LF display system 500.

[0212] In the example shown, venue 700B resembles a circular theater, with viewing positions 722 surrounding a stage. The floor of stage 702 is covered with an array of LF display modules, so that the area above the stage forms a performance volume (e.g., performance volume 710G). The LF display presents performance content in performance volume 710G so that viewers in venue 700B can perceive the performance content. In venue 700B, viewing positions 722 are positioned at an incline so that the line of sight of each viewing position allows unobstructed viewing of performance content 730 from a viewing volume (e.g., viewing volume 720G). Here, the venue includes one viewing volume 720G, so that the same performance is presented to all viewers. In other configurations, there may be more than one viewing volume.

[0213] More generally, the LF display system can have a substantially horizontal or approximately horizontal display surface. In several examples, the LF display system can include a display surface that is: (i) at least some portion of a venue floor, (ii) at least some portion of a stage (e.g., stage 702) in the venue, and / or (iii) at least some portion of an elevated viewing platform in the venue. Other types of horizontal surfaces are also possible. For these configurations, viewers can be elevated relative to the display surface and look down to view holographic performance content projected from the display surface, and viewers can partially or completely surround the display surface. There are many other configurations of light field display surfaces, including vertically mounted display surfaces whose viewing position is arranged approximately in front of the LF display surface and are described elsewhere in this disclosure ( Figure 4C The 450A shown in Figure 4D The 450B shown in the Figure 4E 450C) shown in, and a curved LF display surface.

[0214] For clarity, the performing volume 710G is shown as a bounded square, but the performing volume 710G may be only a portion of the actual performing volume 710G. For example, the performing volume 710G may extend further toward the top of the venue 700B. In addition, a portion of the performing volume 710G and the viewing volume 720G overlap in space. Although shown as partially overlapping, the performing volume and the viewing volume may completely overlap in space. As previously described, the spatial overlap is the area where the viewer can interact with the performance content 730.

[0215] A venue can also be a much smaller location. For example, Figure 8 A venue 800 is shown in a viewer's living room 802 that also serves as a home theater according to one or more embodiments. Here, the home theater includes a LF display system 810 consisting of an array of LF display modules on one wall. Figure 8, LF display system 810 is an embodiment of LF display system 500 .

[0216] The LF display system 810 may be configured so that the performance volume (e.g., holographic object volume 255) and the viewing volume completely overlap within the living room 802. That is, at any viewing position (e.g., viewing position 822) within the living room 802, the viewer may view and interact with the performance content 830. Although the viewing position 822 is shown as a chair, the viewer may walk through the living room and view the performance content 830. That is, the viewing position 822 may be the viewer's position in the living room 802.

[0217] In some embodiments, the LF display system 810 creates (or modifies) performance content 830 based on the viewer's interaction with the performance content 830. For example, the viewer may fist bump the performer in the performance content 830. In this case, in the performance content 830, the viewer may say "Great, NJ" and move his hand as if he is going to fist bump the performer. The tracking system 580 monitors the viewer and identifies (e.g., via machine hearing, machine vision, neural network, etc.) that the viewer wants to fist bump the performer in the performance content 830. The content generation system (e.g., using machine learning, neural network, or some combination thereof) creates performance content that represents the performer shaking his fist back and forth based on using the monitored viewer information. The tracking system 580 monitors the positioning of the viewer's hand, and when the viewer's fist and the performer's fist overlap in space, the sensory feedback system 570 creates a feeling of fist bumping with the performer in the performance content for the viewer.

[0218] In some cases, viewers can interact with the network system 556 to obtain performance content 830 for their living room 802. Figure 5B In the described example, the viewer may send a transaction fee to the network system 556 and the LF generation system 554 sends the holographic content of NJ's performance in New York City to the venue 800.

[0219] Display performance content to viewers in the venue

[0220] Fig. 9 1 is a flow chart of a method 900 for displaying performance content to viewers in a venue (e.g., venue 600) in the context of an LF performance network (e.g., LF performance network 550). Method 900 may include additional or fewer steps, and the steps may be performed in a different order. Furthermore, various steps or combinations of steps may be repeated any number of times during performance of the method.

[0221] First, a venue including an LF display system (e.g., LF display system 500) transmits 910 a request for a live stream (or previously recorded) of performance content (e.g., performance content 630) to a network system (e.g., network system 556) via a network (e.g., network 552). The request includes a transaction fee sufficient to pay royalties to the copyright owner of the performance content.

[0222] The LF generation system (e.g., LF generation system 554) records LF data of a live performance and transmits the corresponding performance content to the network system. The network system transmits the performance content to the LF display system so that the performance content can be displayed at approximately the same time as the LF generation system is recording.

[0223] The LF display system receives 920 holographic performance content 630 from the network system via the network.

[0224] The LF display system determines 930 a configuration of the LF display system and / or the performance space. For example, the LF display system may access a configuration file that contains a plurality of parameters describing the HW configuration of the LF display, including resolution, rays per degree of projection, field of view, deflection angle on the display surface, or dimensions of the LF display surface. The configuration file may also contain information about the geometric orientation of the LF display assembly, including the number, relative orientation, width, height, and layout of the LF display panels. In addition, the configuration file may contain configuration parameters of the performance venue, including the position of the holographic object volume, the viewing volume, and the audience relative to the display panels.

[0225] To illustrate by example, the LF display system 500 determines 930 viewing volumes (e.g., viewing volumes 620A, 620B, and 620C) for displaying performance content. For example, the LF display system 500 may access information in the LF display system that describes the layout, geometric configuration, and / or hardware configuration of the venue. To illustrate, the layout may include the locations, spacing, and sizes of viewing locations (e.g., viewing locations 622) in the venue. Thus, the LF display system may determine that viewing locations in a first level of the venue are in a first viewing volume, viewing locations in a second level of the venue are in a second viewing volume, and viewing locations in a third level of the venue are in a third viewing volume. In various other embodiments, the LF display system may determine any number and configuration of viewing volumes at any location within the venue.

[0226] The LF display system generates 940 holographic content (and other sensed content) for presentation on the LF display system based on the hardware configuration of the LF display system within the performance venue and the specific layout and configuration of the performance venue. Determining the presentation content for display may include rendering the presentation content appropriately for the venue or viewing body. For example, the LF display system may: (i) augment the performance content for display to the viewing body in the third layer with advertisements and remove aspects of the live performance from the performance content, (ii) reduce the fidelity of the performance content for display to the viewing body in the second layer of the venue, and (iii) fully render the performance content for display to the viewing body in the third layer of the venue.

[0227] The LF display system presents 950 performance content in the performance volume in the venue so that viewers at the viewing position in each viewing volume perceive appropriate performance content. That is, viewers in the top viewing volume perceive the performance content with advertisements, viewers in the middle viewing volume perceive the performance content at a lower resolution, and viewers in the bottom viewing volume perceive the fully rendered performance content.

[0228] The LF display system can determine information about viewers in the viewing volume at any time while the viewers in the viewing volume are viewing the performance content. For example, the tracking system can monitor the facial responses of viewers in the viewing volume, and the viewer profiling system can access information about the characteristics of viewers in the viewing volume.

[0229] The LF display system may create (or modify) performance content for simultaneous display based on the determined information. For example, the LF processing engine creates a light show for simultaneous display by the LF display system based on the determined information about the viewer.

[0230] Additional configuration information

[0231] The foregoing description of the embodiments of the present disclosure is presented for illustrative purposes; it is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Those skilled in the relevant art will appreciate that many modifications and variations are possible based on the above disclosure.

[0232] Some parts of this description describe embodiments of the present disclosure according to algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are usually used by technical personnel in the field of data processing to effectively convey the essence of their work to other technical personnel in the field. Although these operations are described functionally, computationally or logically, it is understood that they are implemented by computer programs or equivalent circuits, microcodes, etc. In addition, without loss of generality, it is sometimes proven to be convenient to refer to the arrangement of these operations as modules. The described operations and their related modules can be embodied in software, firmware, hardware or any combination thereof.

[0233] One or more hardware or software modules may be used alone or in combination with other devices to perform or implement any of the steps, operations or processes described herein. In one embodiment, the software modules are implemented using a computer program product that includes a computer-readable medium containing computer program code that can be executed by a computer processor to perform any or all of the steps, operations or processes described.

[0234] Embodiments of the present disclosure may also relate to devices for performing the operations herein. This device may be specially constructed for the desired purpose, and / or it may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in a computer. This computer program may be stored in a non-transitory tangible computer-readable storage medium or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. In addition, any computing system mentioned in the specification may include a single processor, or may be an architecture that uses a multi-processor design to obtain increased computing power.

[0235] Embodiments of the present disclosure may also relate to products produced by the computing process described herein. This product may include information generated by the computing process, wherein the information is stored on a non-transitory tangible computer-readable storage medium and may include any embodiment of the computer program product or other data combination described herein.

[0236] Finally, the language used in the specification is selected primarily for readability and instructional purposes, and may not be selected to describe or limit the subject matter of the present invention. Therefore, it is intended that the scope of the present disclosure is not limited by this specific embodiment, but rather by any claims based on the application attached hereto. Therefore, the disclosure of the embodiments is intended to illustrate, but not to limit, the scope of the present disclosure set forth in the following claims.

Claims

1. A light field display system, comprising: a network interface configured to receive encoded holographic content via a network; a decoder configured to decode the encoded holographic content into a format that can be presented by a light field display component; as well as a light field display assembly configured to present the decoded holographic content to an audience in the venue; The encoded holographic content is in a vectorized data format.

2. The light field display system of claim 1, wherein the encoder decodes the encoded holographic content in near real time.

3. The light field display system of claim 1, wherein the holographic content is a live stream of an event, and the holographic content is presented in near real time. 4 . The light field display system of claim 1 , wherein the encoded holographic content is in a first format and the decoded holographic content is in a second format. 5 . The light field display system of claim 4 , wherein the first format is a vectorized data format and the second format is a rasterized data format.

6. The light field display system according to claim 1, wherein the decoder uses a proprietary codec to decode the encoded streaming holographic content.

7. The light field display system of claim 1, wherein the holographic content is encoded in a format decodable by a proprietary codec.

8. The light field display system according to claim 1, further comprising: a light field processing engine configured to determine a hardware configuration of the light field display system; and The decoder decodes the encoded holographic content based on the determined hardware configuration of the light field display system.

9. The light field display system according to claim 8, wherein the hardware configuration comprises any one of the following: Resolution, The number of rays per degree of projection, Field of view, The deflection angle on the display surface, and The dimensions of the display surface.

10. The light field display system according to claim 1, further comprising: a light field processing engine configured to determine a geometric orientation of the light field display system; and Wherein the decoder decodes the encoded holographic content based on the determined geometric orientation of the light field display system.

11. The light field display system of claim 10, wherein the geometric orientation comprises any one of the following: The plurality of display panels of the light field display assembly, the relative orientation of the display panels, The height of the display panel, the width of the display panel, and The layout of the display panel.

12. The light field display system of claim 1, further comprising: a light field processing engine configured to determine a configuration of the field; and The decoder decodes the encoded holographic content based on the determined configuration of the venue.

13. The light field display system of claim 12, wherein the configuration of the venue comprises any one of the following: one or more holographic objects, One or more viewers, and The position of the viewer relative to the light field display assembly.

14. The light field display system of claim 1, further comprising: A digital rights management system configured to manage digital rights of the received encoded holographic content, the digital rights management system allowing the light field display assembly to project the decoded holographic content, wherein the digital rights management system provides a digital key for the decoded holographic content.

15. The light field display system of claim 1, wherein the holographic content is encrypted and the light field display system is configured to decrypt the holographic content.

16. The light field display system of claim 1, wherein the encoded holographic content is received via the network from a holographic content repository connected to the light field display system.

17. The light field display system of claim 1, wherein the holographic content is received in response to the light field display system transmitting a transaction fee for the holographic content to a holographic content repository.

18. The light field display system of claim 1, wherein the holographic content is received from a holographic content generation system, the holographic content generation system being configured to: Recording live performances, encoding a recording of the performance into encoded holographic content, and The encoded holographic content is transmitted to a light field display system via the network.

19. The light field display system of claim 1, wherein the network is a public network.

20. The light field display system of claim 1, wherein the network is a dedicated network configured for transmitting holographic content.

21. The light field display system of claim 1, wherein The light field display assembly is configured to present acoustic data, and The encoded holographic content contains acoustic data such that when the holographic content is decoded the light field display system presents the decoded holographic content and the decoded acoustic data.

22. The light field display system of claim 1, wherein the presented holographic content comprises a first type of energy and a second type of energy.

23. The light field display system of claim 22, wherein the first type of energy is electromagnetic energy and the second type of energy is ultrasonic energy.

24. The light field display system of claim 23, wherein the first type of energy and the second type of energy are presented at the same location such that the light field display component presents a volumetric tactile surface.

25. The light field display system of claim 1, wherein The light field display assembly is configured to record image data, and The light field display assembly simultaneously presents holographic content and records image data.

26. The light field display system of claim 1, wherein The light field display assembly is configured to record holographic content, and The light field display assembly simultaneously presents and records holographic content.

27. The light field display system of claim 1, further comprising: A tracking system is configured to obtain information about one or more viewers in the audience viewing the holographic content.

28. The light field display system of claim 27, wherein the light field display assembly presents holographic content to the viewer based on information obtained by the tracking system.

29. The light field display system of claim 27, wherein the information obtained by the tracking system comprises: a response of the one or more viewers in the audience to the presented holographic content, and Characteristics of the one or more viewers in the audience.

30. The light field display system of claim 27, wherein the information about the viewer includes any one of the viewer's position, the viewer's movement, the viewer's gesture, the viewer's expression, the viewer's age, the viewer's gender, and clothing worn by the viewer.

31. The light field display system of claim 1, further comprising: A viewer profiling system configured to identifying viewers in the audience viewing the holographic content, and A viewer profile is generated for each of the identified viewers.

32. The light field display system of claim 31 , wherein the light field display component presents holographic content to the audience based on identified viewers in the audience.

33. The light field display system of claim 31 , wherein the light field display system presents holographic content to the viewer based on the identified viewer's profile.

34. The light field display system of claim 31, wherein the viewer profiling system is configured to identify a viewer response to the holographic content or a characteristic of a viewer viewing the holographic content, and include the identified response or characteristic in a viewer profile.

35. The light field display system of claim 27, wherein the light field viewer profiling system identifies characteristics of the viewer, and The characteristics include any of the following: the location of the viewer, The viewer's actions, The viewer's gesture, The viewer's facial expression, The gender of the viewer, the age of said viewer, and Clothing of the viewer.

36. The light field display system of claim 27, wherein the viewer profiling system accesses social media accounts of one or more identified viewers to generate a viewer profile.

37. The light field display system of claim 1, wherein the venue is a theater.

38. The light field display system of claim 1, wherein the venue comprises a plurality of viewing positions generally surrounding a substantially horizontal display surface of the light field display system.

39. The light field display system of claim 38, wherein the display surface is at least a portion of a floor of the venue.

40. The light field display system of claim 38, wherein the display surface is at least a portion of a stage in the venue.

41. The light field display system of claim 38, wherein the display surface is at least a portion of an elevated viewing platform in the venue.

42. The light field display system of claim 1, wherein the venue comprises a plurality of viewing positions arranged substantially in front of the light field display component.

43. The light field display system of claim 1, wherein the venue comprises a plurality of viewing positions positioned so as to provide an unobstructed view of the presented holographic content.

44. The light field display system of claim 1, wherein the venue has a slope and one or more viewing positions are located along the slope.

45. The light field display system of claim 1, wherein the field comprises: One or more viewing layers, each viewing layer configured to contain a portion of the audience.

46. ​​The light field display system of claim 45, wherein the light field display system presents the same decoded holographic content to each floor of the venue.

47. The light field display system of claim 45, wherein the light field display system presents different decoded holographic content to one or more layers of the venue.

48. The light field display system of claim 45, wherein the layer is substantially circular and is positioned such that it surrounds the light field display assembly.

49. The light field display system of claim 1, wherein The venue includes a first wall, and The light field display assembly is positioned on the first wall such that the holographic content is presented to the audience from the first wall.

50. The light field display system of claim 49, wherein The venue includes a second wall, and The light field display assembly is positioned on the first wall and the second wall such that the holographic content is presented to the audience from the first wall and the second wall.

51. The light field display system of claim 1, wherein The venue includes a stage, and The light field display assembly is positioned on the stage such that the holographic content is presented from the stage to the audience.

52. The light field display system of claim 1, wherein The venue includes a floor, and The light field display assembly is positioned on the floor such that the holographic content is presented from the floor to the audience.

53. The light field display system of claim 1, wherein the light field display component is a substantially flat surface.

54. The light field display system of claim 1, wherein the light field display system is a curved surface.

55. A light field display system, comprising: a network interface configured to receive a live stream including real-time holographic content via a network; a light field processing engine configured to generate additional holographic content; as well as a light field display assembly configured to present the real-time holographic content and the additional holographic content to an audience in the venue; The live stream including real-time holographic content is received in a vectorized data format.

56. The light field display system of claim 55, wherein the live stream comprises simulcast real-time holographic content representing a performance.

57. The light field display system of claim 55, wherein the live stream includes additional holographic content representing a performance.

58. The light field display system of claim 57, wherein the additional holographic content is generated by a procedural generation algorithm.

59. The light field display system of claim 55, wherein the additional holographic content comprises one or more of: Computer-generated renderings of objects, events, or scenes, and Previously recorded live holographic content.

60. The light field display system of claim 55, wherein the live stream includes holographic content representing a performer.

61. The light field display system of claim 55, wherein the additional holographic content enhancement comprises the live stream of real-time holographic content.

62. The light field display system of claim 55, wherein the additional holographic content replaces real-time holographic content contained in the live stream.

63. The light field display system of claim 55, wherein the additional holographic content represents one or more of: stage, props, Costumes, and Musical instrument.

64. The light field display system of claim 55, wherein the additional holographic content represents one or more of: A person, object, event, or scene from another time or place, Computer-generated renderings of people, objects, events, or scenes, and Previously recorded holographic content.

65. The light field display system of claim 55, wherein the additional holographic content represents one or more of advertisements.

66. The light field display system of claim 55, wherein the additional holographic content is generated by a procedural generation algorithm.

67. The light field display system of claim 55, wherein viewers in the audience are able to interact with the additional holographic content.

68. The light field display system of claim 55, wherein viewers in the audience are able to interact with the additional holographic content instead of the real-time holographic content.

69. The light field display system of claim 55, wherein a viewer is able to interact with the real-time holographic content.

70. The light field display system of claim 55, wherein the processing engine is configured to generate additional holographic content based on interactions of viewers in the audience with the holographic content presented by the light field display component.

71. The light field display system of claim 55, wherein a viewer in the audience interacts with the presented holographic content, wherein the interaction comprises any of: Auditory interaction, and Visual interaction.

72. The light field display system of claim 55, further comprising: A tracking system is configured to obtain information about one or more viewers in the audience viewing the holographic content.

73. The light field display system of claim 72, wherein the additional holographic content generated by the processing engine is based on the information about the one or more viewers in the audience obtained by the tracking system.

74. The light field display system of claim 72, wherein the information obtained by the tracking system comprises: a response by the one or more viewers in the audience to the presented holographic content, and Characteristics of the one or more viewers in the audience.

75. The light field display system of claim 72, wherein the information about the viewer includes any one of the viewer's position, the viewer's movement, the viewer's gesture, the viewer's expression, the viewer's age, the viewer's gender, and clothing worn by the viewer.

76. The light field display system of claim 72, wherein The tracking system is configured to determine aggregate information about the one or more viewers in the audience, and the additional holographic content generated by the processing engine is based on the aggregate information.

77. The light field display system of claim 55, further comprising: A viewer profiling system configured to: identifying one or more viewers among the audience viewing the holographic content, and A viewer profile is generated for the one or more identified viewers.

78. The light field display system of claim 77, wherein the additional holographic content generated by the processing engine is based on the one or more viewers identified by the viewer profiling system.

79. The light field display system of claim 77, wherein the additional holographic content generated by the processing engine is based on the viewer profile generated by the viewer profiling system.

80. The light field display system of claim 77, wherein the viewer profiling system is configured to identify a viewer response to the holographic content or a characteristic of a viewer viewing the holographic content, and to include the identified response or characteristic in a viewer profile.

81. The light field display system of claim 77, wherein the viewer profiling system identifies characteristics comprising any of the following: the location of the viewer, The viewer's actions, The viewer's gesture, The viewer's facial expression, The gender of the viewer, the age of said viewer, and Clothing of the viewer.

82. The light field display system of claim 77, wherein The viewer profiling system is configured to generate an aggregate profile about the one or more viewers in the audience, and The additional holographic content generated by the processing engine is based on the aggregated profile.

83. The light field display system of claim 55, further comprising: a light field processing engine configured to determine a hardware configuration of the light field display system; and The light field display component presents the holographic content based on the determined hardware configuration of the light field display system.

84. The light field display system of claim 83, wherein the hardware configuration comprises any one of the following: Resolution, The number of rays per degree of projection, Field of view, The deflection angle on the display surface, and The dimensions of the display surface.

85. The light field display system of claim 55, further comprising: a light field processing engine configured to determine a geometric orientation of the light field display system; and Wherein the holographic content is presented to the audience based on the determined geometric orientation of the light field display system.

86. The light field display system of claim 85, wherein the geometric orientation comprises any of the following: The plurality of display panels of the light field display assembly, the relative orientation of the display panels, The height of the display panel, the width of the display panel, and The layout of the display panel.

87. The light field display system of claim 55, further comprising: a light field processing engine configured to determine a configuration of the field; and Wherein the light field display component presents holographic content based on the determined configuration of the venue.

88. The light field display system of claim 87, wherein the configuration of the venue comprises any of the following: one or more holographic objects, One or more viewers, and The position of the viewer relative to the light field display assembly.

89. The light field display system of claim 55, wherein The light field display assembly is configured to present acoustic data, and The holographic content includes acoustic data, such that the light field display system presents the holographic content and the acoustic data.

90. The light field display system of claim 55, wherein the presented holographic content comprises a first type of energy and a second type of energy.

91. The light field display system of claim 90, wherein the first type of energy is electromagnetic energy and the second type of energy is ultrasonic energy.

92. The light field display system of claim 91, wherein the first type of energy and the second type of energy are presented at a same location such that the light field display component presents a volumetric tactile surface.

93. The light field display system of claim 55, wherein The light field display assembly is configured to record image data, and The light field display assembly simultaneously presents holographic content and records image data.

94. The light field display system of claim 55, wherein The light field display assembly is configured to record holographic content, and The light field display assembly simultaneously presents and records holographic content.

95. The light field display system of claim 55, wherein the live stream comprising real-time holographic content is received in a first format and the light field processing engine decodes the holographic content into a second format.

96. The light field display system of claim 95, wherein the first format is a vectorized data format and the second format is a rasterized data format.

97. The light field display system of claim 55, wherein the processing engine employs a dedicated codec to decode streaming holographic content encoded by the same codec.

98. A light field generation system, comprising: a light field recording component configured to record one or more types of energy representative of events in the venue; a processing engine configured to convert the recorded energy into holographic content representing a performance; as well as a network interface configured to transmit the holographic content to one or more light field display systems via a network; The holographic content is encoded in a vectorized format.

99. The light field generating system of claim 98, wherein the event is a performance.

100. The light field generating system of claim 98, wherein the event includes a performer.

101. The light field generation system of claim 98, wherein the venue is a performance hall.

102. The light field generating system of claim 98, wherein the network is a public network.

103. The light field generation system of claim 98, wherein the network is a dedicated network configured for transmitting holographic content.

104. The light field generation system of claim 98, wherein the processing engine employs a dedicated codec to encode the recorded energy into holographic content.

105. The light field generation system of claim 104, wherein the holographic content is encoded in a format decodable by a light field display system using the dedicated codec.

106. The light field generation system of claim 98, wherein the light field recording component comprises: one or more light field modules comprising one or more energy sensors configured to record electromagnetic energy as light field content, and The holographic content includes the light field content.

107. The light field generation system of claim 98, wherein the light field recording component comprises: one or more acoustic recording devices configured to record the acoustic energy as audio content, and The holographic content includes the audio content.

108. The light field generation system of claim 98, wherein the light field recording component comprises: one or more pressure sensors configured to record mechanical energy, and Wherein the holographic content comprises instructions for generating mechanical energy to produce one or more tactile surfaces.

109. The light field generation system of claim 98, wherein the light field recording component comprises: One or more recording modules positioned around the venue such that the light field recording modules capture energy from the performance at multiple viewpoints.

110. The light field generation system of claim 109, wherein the processing engine converts the energy from multiple viewpoints into the holographic content.

111. The light field generation system of claim 98, wherein the light field recording component comprises: One or more two-dimensional cameras, each two-dimensional camera recording energy from the performance as one or more two-dimensional images.

112. The light field generation system of claim 111, wherein the processing engine converts the one or more two-dimensional images into holographic content.

113. The light field generation system of claim 112, wherein the processing engine employs a machine learning algorithm to generate the holographic content.

114. The light field generation system of claim 98, wherein the light field recording component comprises: One or more depth sensors for determining the depth of an object.

115. The light field generation system of claim 98, wherein the light field recording component comprises: One or more plenoptic camera systems for recording light field data.

116. The light field generation system of claim 98, wherein the light field generation system transmits the holographic content to a light field display system via the network in near real time.

117. The light field generation system of claim 98, wherein the light field generation system stores the holographic content on a local storage device.

118. The light field generation system of claim 98, wherein the light field generation system transmits the holographic content to a network system for storage on a network storage system.

119. The light field generation system of claim 98, wherein the light field generation system transmits the holographic content to a light field display system in response to receiving a transaction fee.

Images