Head Tracking Multi-View Display and Method

Abstract

A head-tracking multi-view display includes a multi-beam backlight that provides a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image. A processor receives a plurality of provisional views of the multi-view image. The provisional views correspond to different view directions. The processor receives information regarding a tracking position of a user. The processor shifts the provisional views with respect to the view directions to form a plurality of shifted views corresponding to different view directions. The shifted views vary as a function of the tracking position of the user and are shifted by a shift value that varies across the field of view of the multi-view image. A light valve array modulates the plurality of light beams to provide the plurality of shifted views of the multi-view image in the view directions as the multi-view image.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 340,361, filed May 10, 2022, which is hereby incorporated by reference in its entirety.

[0002] Description of Research and Development Sponsored by the Federal Government Not applicable

Background Art

[0003] Electronic displays are an almost ubiquitous medium for communicating information to users of a wide variety of devices and products. The most commonly used electronic displays include cathode ray tubes (CRTs), plasma display panels (PDPs), liquid crystal displays (LCDs), electroluminescent displays (ELs), organic light - emitting diodes (OLEDs) and active - matrix OLED (AMOLED) displays, electrophoretic displays (EPs), and various displays that utilize electro - mechanical or electro - fluidic light modulation (e.g., digital micromirror devices, electro - wetting displays, etc.). Generally, electronic displays can be classified into either active displays (i.e., displays that emit light) or passive displays (i.e., displays that modulate light supplied by another source). Examples of active displays include CRTs, PDPs, and OLEDs / AMOLEDs. Displays that are typically classified as passive when considering the emitted light are LCDs and EP displays. Passive displays often exhibit attractive performance characteristics, including but not limited to being inherently low - power, but due to their inability to emit light, they can sometimes seem somewhat limited in use in many practical applications.

Brief Description of the Drawings

[0004] The various features of the examples and embodiments according to the principles described herein can be more readily understood by reference to the following detailed description in conjunction with the accompanying drawings, where like reference numerals refer to like structural elements.

[0005]

Figure 1

[0006]

Figure 2

[0007]

Figure 3

[0008]

Figure 4

[0009]

Figure 5A

[0010]

Figure 5B

[0011]

Figure 5C

[0012]

Figure 6A

[0013]

Figure 6B

[0014]

Figure 6C

[0015]

Figure 7

[0016]

Figure 8

Best Mode for Carrying Out the Invention

[0017] Specific examples and embodiments have certain features shown in the above reference drawings, or other features in addition to or instead of those features. These features and other features are described in detail below with reference to the above reference drawings.

[0018] Examples and embodiments according to the principles described herein provide a multi-view or three-dimensional (3D) image display that uses information regarding a user's position over time, which may also be referred to as "head tracking". Embodiments consistent with the principles described herein can use a multi-view display to provide a set of different views of a scene represented by multi-view images according to a user's position. A head-tracking multi-view display can receive a provisional view of multi-view images corresponding to different view directions. A head-tracking (or head-trace) multi-view display can receive information regarding a user's tracked position from, for example, a head-tracking sensor. A head-tracking (or head-trace) multi-view display can shift the provisional view with respect to the view direction to form a shifted view corresponding to a different view direction. The shifted view may vary as a function of the user's tracked position and may be shifted by a shift value that varies across the field of view of the multi-view images.

[0019] It may be beneficial to use shift values that vary across the field of view of a multi-view image. For example, in a display that uses an invariant shift value across the field of view of a multi-view image, a user may perceive different views of the multi-view image at different positions on the display, which may be disorienting or uncomfortable for the user. As a specific example, a user may perceive a portion of the display as showing a first view while simultaneously perceiving another portion of the display as showing a second view. In contrast, in a display where the shift value varies across the field of view of a multi-view image, as detailed below, the user can perceive the same view across the entire field of view of the multi-view image. By presenting the multi-view image such that the user perceives the same view across the field of view of the multi-view image (e.g., avoiding situations where the user perceives a portion of the display showing a first view and another portion of the display showing a second view), a more realistic viewing experience can be provided to the user. In a particular example, one eye of the user can perceive a first view across the entire field of view of the multi-view image, and the other eye of the user can perceive a second view across the entire field of view of the multi-view image, such that the user can experience a stereoscopic effect.

[0020] In various embodiments, a head tracking sensor or algorithm can provide information regarding the position or location of a user relative to a multi-view display. That is, the position of the user can be determined or inferred by tracking the position of the user's head or other anatomical features. For the sake of ease of explanation herein, and not by way of limitation, the embodiments or components described herein may be referred to as including or using "head tracking" in, for example, multi-view displays, systems, and methods that use head tracking.

[0021] FIG. 1 shows a perspective view of a multi-view display 10 in an example according to an embodiment consistent with the principles described herein. As shown in FIG. 1, the multi-view display 10 includes a screen 12 for displaying a multi-view image to be viewed. The multi-view display 10 provides various views 14 of the multi-view image in various view directions 16 with respect to the screen 12. The view directions 16 are shown as arrows extending in various different principal angular directions from the screen 12. The different views 14 are shown as filled polygonal frames at the ends of the arrows (i.e., indicating the view directions 16). By way of example and not limitation, only four views 14 and four view directions 16 are shown. Although FIG. 1 shows the different views 14 as being above the screen, it should be noted that when the multi-view image is displayed on the multi-view display 10, the views 14 actually appear on or near the screen 12. Showing the views 14 above the screen 12 is merely for simplicity of explanation and means looking at the multi-view display 10 from each one of the respective ones of the view directions 16 corresponding to a particular view 14.

[0022] A light beam having a viewing direction, or equivalently, a direction corresponding to the viewing direction of a multi-view display, generally has a principal angular direction given by angular components {θ, φ} as defined herein. The angular component θ is referred to herein as the "elevation angle component" or "elevation angle" of the light beam. The angular component φ is referred to as the "azimuth angle component" or "azimuth angle" of the light beam. By definition, the elevation angle θ is an angle in a vertical plane (e.g., a plane perpendicular to the plane of the multi-view display screen), and the azimuth angle φ is an angle in a horizontal plane (e.g., a plane parallel to the plane of the multi-view display screen). FIG. 2 shows a graphical representation of the angular components {θ, φ} of a light beam 20 having a specific principal angular direction corresponding to the viewing direction (e.g., viewing direction 16 in FIG. 1) of a multi-view display in an example according to an embodiment consistent with the principles described herein. Further, the light beam 20 is emitted or diverges from a specific point according to the definition herein. That is, by definition, the light beam 20 has a central ray associated with a specific origin within the multi-view display. FIG. 2 also shows the origin O of the light beam (or viewing direction).

[0023] Furthermore, as used herein, the term "multi-view" as used in the terms "multi-view image" and "multi-view display" is defined as a plurality of views representing different viewpoints, or a plurality of views including an angular parallax between views among the plurality of views. Further, as used herein, the term "multi-view" explicitly includes three or more different views (i.e., at least three views, generally four or more views) according to the definition herein. Thus, the "multi-view display" adopted herein is clearly distinguished from a stereoscopic display that includes only two different views to represent a scene or an image. However, it should be noted that multi-view images and multi-view displays include three or more views according to the definition herein, while a multi-view image may be seen as a stereoscopic pair of images (e.g., on a multi-view display) by selecting to view only two of the multi-views (e.g., one view per eye) at the same time.

[0024] In this specification, "multi-view pixel" is defined as a set of sub-pixels representing "view" pixels in each of a plurality of different views of a multi-view display. Specifically, a multi-view pixel may have individual sub-pixels corresponding to or representing the view pixels of each of the different views of a multi-view image. Further, according to the definition of this specification, the sub-pixels of a multi-view pixel are so-called "direction pixels" in that each of the sub-pixels is associated with a predetermined view direction of one corresponding view among the different views. Further, according to various examples and embodiments, the different view pixels represented by the sub-pixels of a multi-view pixel may have the same or at least substantially the same position or coordinates in each of the different views. For example, a first multi-view pixel may have individual sub-pixels corresponding to the view pixels located at {x 1 , y 1} in each of the different views of a multi-view image, and a second multi-view pixel may have individual sub-pixels corresponding to the view pixels located at {x 2 , y 2} in each of the different views.

[0025] As used herein, a "light guide" is defined as a structure that guides light within its structure using total internal reflection. In particular, a light guide can include a core that is substantially transparent at the operating wavelength of the light guide. In various examples, the term "light guide" generally refers to a dielectric optical waveguide that uses total internal reflection to guide light at the interface between the dielectric material of the light guide and the material or medium surrounding the light guide. By definition, the condition for total internal reflection is that the refractive index of the light guide is greater than the refractive index of the surrounding medium adjacent to the surface of the light guide material. In some embodiments, the light guide can include a coating in addition to, or instead of, the aforementioned refractive index difference to further facilitate total internal reflection. The coating can be, for example, a reflective coating. The light guide can be any of several light guides including, but not limited to, one or both of a plate guide or slab guide and a strip guide.

[0026] Further, as used herein, the term "plate" when applied to a light guide, as in a "plate light guide", is defined as a discrete or distinct planar layer or sheet, which may also be referred to as a "slab" guide. In particular, a plate light guide is defined as a light guide configured to guide light in two substantially orthogonal directions bounded by the top and bottom surfaces (i.e., opposite surfaces) of the light guide. Further, by the definition herein, both the top and bottom surfaces are separated from each other and can be substantially parallel to each other, at least in the sense of a difference. That is, within any distinguishable small portion of the plate light guide, the top and bottom surfaces are substantially parallel or in the same plane.

[0027] In some embodiments, the plate light guide may be substantially flat (i.e., limited to a plane), and thus the plate light guide is a planar light guide. In other embodiments, the plate light guide may be curved in one or two orthogonal dimensions. For example, the plate-shaped light guide may be curved within one dimension to form a cylindrical plate-shaped light guide. However, any curvature has a radius of curvature large enough to ensure that total internal reflection is maintained within the plate-shaped light guide to guide light.

[0028] As used herein, a "light source" is defined as a source of light (e.g., an optical emitter configured to generate and emit light). For example, a light source may include a light emitter such as a light-emitting diode (LED) that emits light when activated or turned on. In particular, as used herein, a light source may be substantially any light source, or may include a substantially arbitrary light emitter including one or more of a light-emitting diode (LED), a laser, an organic light-emitting diode (OLED), a polymer light-emitting diode, a plasma-based light emitter, a fluorescent lamp, an incandescent lamp, and substantially any other light source, but is not limited thereto. The light supplied by the light source may have a color (i.e., may include light of a specific wavelength) or may have a range of wavelengths (such as white light). In some embodiments, the light source may include a plurality of light emitters. For example, the light source may include a set or group of light emitters where at least one of the light emitters generates light having a color or equivalently a wavelength that is different from the color or wavelength of the light supplied by at least one other light emitter in the set or group. Different colors may include, for example, primary colors (e.g., red, green, blue).

[0029] In this specification, a "multi-view image" is defined as a plurality of images (i.e., a number of images exceeding three), and each image of the plurality of images represents a different view corresponding to a different view direction of the multi-view image. Thus, a multi-view image, when displayed on a multi-view display, facilitates the perception of depth and is thus, for example, a collection of images (e.g., a plurality of two-dimensional images) that appears to the user to be an image of a 3D scene. A multi-view image that provides pairs of views representing different but related perspectives of a 3D scene that match the user's viewing is defined as a 3D image.

[0030] Embodiments consistent with the principles described herein may be implemented using a variety of devices and circuits including, but not limited to, integrated circuits (ICs), very large scale integration (VLSI) circuits, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), graphical processor units (GPUs), firmware, software (such as program modules or instruction sets), and one or more of combinations of two or more of the foregoing. For example, one embodiment or an element thereof may be implemented as circuit elements within an ASIC or VLSI. Implementations using ASICs or VLSIs are examples of hardware-based circuit implementations.

[0031] In another embodiment, one implementation can be implemented as software using a computer programming language (e.g., C / C++) that runs in an operating environment, or a software-based modeling environment (e.g., MATLAB® from MathWorks, Natick, Massachusetts) that is further executed by a computer (e.g., stored in memory and executed by a processor of a general-purpose computer or a graphics processor). One or more computer programs or software may constitute a computer program mechanism, and the programming language can be compiled or interpreted for execution by a processor or graphics processor of a computer, e.g., made configurable or configured (which can be used interchangeably in this description).

[0032] In yet another embodiment, blocks, modules, or elements of the apparatuses, devices, or systems described herein (e.g., image processors, cameras, etc.) can be implemented using actual, i.e., physical circuits (e.g., as ICs or ASICs), and other blocks, modules, or elements can be implemented in software or firmware. In particular, according to the definitions herein, some embodiments can be implemented using substantially hardware-based circuit techniques or devices (e.g., ICs, VLSIs, ASICs, FPGAs, DSPs, firmware, etc.), and other embodiments can also be implemented, for example, as software or firmware that uses a computer processor or graphics processor to execute software, or as a combination of software or firmware and hardware-based circuits.

[0033] Furthermore, as used herein, the article "a" is intended to have its ordinary meaning in the art of patents, i.e., "one or more than one". For example, "a lens" means one or more lenses, and thus, in this specification, "a lens" means "one or more lenses". Also, references herein to "top", "bottom", "upper", "lower", "up", "down", "front", "back", "first", "second", "left", or "right" are not intended to be limiting in this specification. As used herein, the term "about", when applied to a value, generally means within the tolerance of the equipment used to generate that value or, unless otherwise specified, can mean plus or minus 10%, plus or minus 5%, or plus or minus 1%. Further, the term "substantially" as used herein means mostly, or almost all, or all, or an amount within the range of about 51% to about 100%. Further, the examples herein are intended to be illustrative only, presented for explanatory purposes and not for limitation.

[0034] According to some embodiments of the principles described herein, a head-tracking multi-view display 100 is provided. The head-tracking multi-view display 100 may be referred to as a head-tracking multi-view display in some embodiments. FIG. 3 shows a cross-sectional view of a head-tracking multi-view display 100 in an example according to an embodiment consistent with the principles described herein. The head-tracking multi-view display 100 is configured to provide a plurality of views of a scene as a multi-view image, i.e., a displayed multi-view image. In particular, the plurality of views are provided by the head-tracking multi-view display 100 in corresponding plurality of view directions. In FIG. 3, the view direction or equivalently the views among the plurality of views are depicted as arrows 102 pointing in different angular directions extending from the head-tracking multi-view display 100. For convenience, the view direction of the plurality of views or equivalently the views are labeled with view identification numbers 104. In the particular example of FIG. 3, the head-tracking multi-view display 100 provides eight views, although other numbers of views can also be used. The views in FIG. 3 are identified as 1 through 8 with view identification numbers 104, although other identifiers such as letters A - H can also be used. The view identification numbers 104 in FIG. 3 are provided for convenience only and do not mean that the views are continuous or ordered in any way.

[0035] As shown in FIG. 3, the head-tracking multi-view display 100 includes a multi-beam backlight 106. The multi-beam backlight 106 is configured to provide a plurality of light beams having different principal angular directions corresponding to different view directions of the multi-view image. In the configuration of FIG. 3, the multi-beam backlight 106 can include a light source 108 controlled by a controller 110. The light source 108 can direct light to the edge of a light guide 112. The light guide 112 can propagate the light within the light guide 112 as guided light. A plurality of multi-beam elements 114 can direct respective portions of the guided light out of the light guide 112 to form a plurality of light beams. The configuration of the multi-beam backlight 106 shown in FIG. 3 is merely an example of the multi-beam backlight 106, and other suitable configurations can also be used.

[0036] As shown in FIG. 3, the head-tracking multi-view display 100 further includes a processor 116. In the example of FIG. 3, the processor 116 is included in the controller 110. In other configurations, the processor 116 may be separate from the controller 110. The processor 116 is configured to receive a plurality of provisional views of the multi-view image. The plurality of provisional views can correspond to different view directions. The processor 116 is further configured to receive information regarding the tracking position of the user 118, such as including information regarding the direction or distance between the user 118 and the light-emitting portion of the display. In one example, the tracking position of the user 118 can include information regarding the view direction of the user 118. The processor 116 is further configured to shift the provisional views with respect to the view direction to form a plurality of shifted views corresponding to different view directions. In various embodiments, the shifted views are shifted by or in accordance with a shift value. The shift value may vary as a function of the tracking position of the user 118. In various embodiments, the shift value may also vary across the field of view of the multi-view image. The calculation of the shift value will be described later.

[0037] As shown in FIG. 3, the head-tracking multi-view display 100 further includes a light valve array 120. The light valve array 120 is configured to modulate a light beam among a plurality of light beams in order to provide a plurality of shifted views of a multi-view image in a view direction as the multi-view image. The light valve array 120 can be controlled by the controller 110. In various embodiments, any of various different types of light valves may be used as the light valve of the light valve array 120, including but not limited to one or more of a liquid crystal light valve, an electrophoretic light valve, and an electro-wetting based light valve.

[0038] As shown in FIG. 3, the head-tracking multi-view display 100 further includes a head tracker 122. The head tracker 122 is configured to provide a tracking position of the user 118 (e.g., the head of the user 118, or one or both eyes of the user 118, or another anatomical feature of the user 118). The head tracker 122 can include a camera configured to capture an image of the user 118. The head tracker 122 can further include an image processor (or a general-purpose computer programmed as an image processor) configured to determine the position of the user 118 within the captured image in order to provide the tracking position. In some examples, the processor 116 can include the image processor of the head tracker 122, such as by executing operations on the same processing circuitry. In other examples, the processor 116 can be separate from the image processor of the head tracker 122. Other suitable head trackers can also be used, including head trackers based on lidar or other technologies (e.g., using time-of-flight to a view of reflected light on a scene to determine the distance to one or more objects in the scene, such as the head or eyes of a user). The output of the head tracker 122 can be used to modify the operation of the head-tracking multi-view display 100 (e.g., modulation of the light beam by the light valve array 120). For example, the determined position of the user 118 can be provided to one or both of the processor (such as the processor 116) and the light valve driver (e.g., a driver circuit, or the controller 110) of the head-tracking multi-view display 100, and the light emission pattern from the head-tracking multi-view display 100 can be adjusted to correspond to the position of the user. Other implementation examples of the head tracker 122 can include, but are not limited to, any of various two-dimensional (2D) and three-dimensional (3D) object tracking systems, such as the Kinect® object tracking system. Kinect® is a registered trademark of Microsoft Corporation of Redmond, Washington.

[0039] As described above and as will be explained in detail below, the head-tracking multi-view display 100 of FIG. 3 determines a shift value that varies across the field of view of the multi-view image. To demonstrate the effect of varying the shift value across the field of view of the multi-view image, FIG. 4 shows a plan view of an example of a shift-invariant head-tracking multi-view display 400 in which the shift value does not vary across the field of view of the multi-view image. Instead, the shift value is constant or invariant across the field of view of the multi-view image, such as across the active surface area of the shift-invariant head-tracking multi-view display 400.

[0040] In the example of FIG. 4, for some positions of user 402, different regions of the multi-view image can be seen in different views of the multi-view image. For example, the upper left corner of the multi-view image can be seen in view number 1, the central portion of the multi-view image can be seen in view number 2, and the lower right corner of the multi-view image can be seen in view number 3. Since the image perceived by user 402 at the user position with respect to the shift-invariant head-tracking multi-view display 400 may include regions from different views of the multi-view image, the perceived image may be distorted or may include artifacts from view misalignment or from the boundaries between different view portions.

[0041] Varying the shift value across the field of view of the multi-view image can help overcome the drawbacks of the shift-invariant head-tracking multi-view display 400. Specifically, varying the shift value across the entire field of view of the multi-view image can help ensure that all or substantially all of the multi-view image is presented to the user as a single view (or a single combination of views). Examples of determining the shift value will be described in detail below.

[0042] Processor 116 may determine a shift value to provide two general functions to the head-tracking multi-view display 100. The first general function of the shift value is to direct the specified view towards the eyes of user 118 when the user 118 changes position relative to the display. The second general function of the shift value is to make only a single specified view (or a specified combination of views) visible (such as extending across the specified active surface area of the head-tracking multi-view display) across the entire multi-view image, and to make adjacent views or spurious views invisible. In other words, by calculating the shift value as detailed below, it can be ensured that user 118 does not see a portion of the multi-view image of the first view and another portion of the multi-view image of the second view, thereby avoiding the example of the shift-invariant head-tracking multi-view display 400 of FIG. 4.

[0043] Processor 116 can include a mathematical model that predicts which field of view is visible to user 118 as a function of the position of user 118 and as a function of the surface area of the head-tracking multi-view display or the direction of the field of view of user 118 thereon. The views can be represented as real numbers. For example, in the case of a multi-view image including eight views, the views can be represented by an integer N, where N can have integer values from 1 to 8. User 118 can have a central head position represented by spatial coordinates as (x, y, z). The head-tracking multi-view display can have pixel positions within the active surface area of the head-tracking multi-view display represented by spatial coordinates as (x 0 , y 0 ). The mathematical model can determine the view number N(x, y, z, x 0 , y 0 ) that is a function of the user position and the pixel position. In other words, the user position (x, y, z) and the pixel position (x 0 , y 0)For each combination with, the mathematical model can determine which view N is visible.

[0044] The shift value represented as the quantity δN is a dimensionless quantity representing a correction or addition to the view number N. For example, a shift value δN of +1 may include incrementing the view number (e.g., view number 3) by a value of +1 (e.g., view number 4). The shift value δN may vary for each pixel position across the active surface area of the head-tracking multi-view display (e.g., across the field of view of the multi-view image). By enabling the shift value δN to vary across the field of view of the multi-view image, the user 118 can reliably view just one specified multi-view image (or one specified combination of multi-view images) across the entire multi-view image.

[0045] The processor can calculate the shift value to achieve a specific effect in the perceived view. Two such effects are hereinafter referred to as the "peeling mode" and the "sliding mode", both of which are described in more detail hereinafter.

[0046] In the peeling mode, when the user 118 changes position, the views can advance view by view, so the user 118 can view the multi-view image from many different views depending on the position of the user 118. In some embodiments, in the peeling mode, the shift value may be selected such that the movement of the user 118 allows the user to view the views in a progression that matches the progression of the provisional view. In some embodiments, in the peeling mode, the shift value may be selected such that the movement of the user 118 provides the user with a sense of parallax as the views advance.

[0047] The peeling mode can expand the view zone of the head-tracking multi-view display without changing the parallax effect. In the peeling mode, the shift value δN is the static view number N that is invariant with respect to the user position and the pixel position as follows 0, the central view number N(x, y, z, 0, 0) representing which view is visible from the user position to the center of the display, and the pixel-dependent view number N(x, y, z, x 0 , y 0 ) may be calculated as a function of. δN = N 0 + N(x, y, z, 0, 0) - N(x, y, z, x 0 , y 0 ) (1)

[0048] FIG. 5A shows a plan view of a head-tracking multi-view display 500 used in a peeling mode in an example according to an embodiment consistent with the principles described herein, where the user 502 is located at a first position. At the first position, the user 502 can view a first view (indicated by reference numeral 1) of the multi-view image across the entire multi-view image.

[0049] FIG. 5B shows a plan view of a head-tracking multi-view display 500 used in a peeling mode in an example according to an embodiment consistent with the principles described herein, where the user 502 is located at a second position. At the second position, the user 502 can view a second view (indicated by reference numeral 2) of the multi-view image across the entire multi-view image.

[0050] FIG. 5C shows a plan view of a head-tracking multi-view display 500 used in a peeling mode in an example according to an embodiment consistent with the principles described herein, where the user 502 is located at a third position. At the third position, the user 502 can view a third view (indicated by reference numeral 3) of the multi-view image across the entire multi-view image.

[0051] In the sliding mode, when user 118 changes position, a single view (or a single combination of views) follows user 118. For example, the sliding mode can effectively function as an image stabilization device that ensures that user 118 only views a specified view (or a specified combination of views) and prevents other views from being visible when user 118 changes position. In some embodiments, in the sliding mode, the shift value may be selected such that the shifted view follows the movement of user 118 and enables user 118 to view only a single view or a single combination of views when user 118 moves. The sliding mode can ensure that user 118 experiences a uniform static view represented by the amount N 0 without parallax. In the sliding mode, head tracking can effectively cancel out any movement of user 118 and provide user 118 with a 3D experience similar to using 3D glasses in a movie theater.

[0052] Using the same amount as defined above, the shift value δN can be calculated as follows. δN = N 0 -N(x, y, z, x 0 , y 0 ) (2)

[0053] FIG. 6A shows a plan view of a head-tracking multi-view display 600 used in an example in the sliding mode according to an embodiment consistent with the principles described herein, where user 602 is located at a first position. At the first position, user 602 can view a first view (indicated by reference numeral 1) of the multi-view image across the entire multi-view image.

[0054] FIG. 6B shows a plan view of a head-tracking multi-view display 600 used in a sliding mode in an example according to an embodiment consistent with the principles described herein, where user 602 is positioned at a second position. At the second position, user 602 can view a first view (indicated by reference numeral 1) of the multi-view image across the entire multi-view image.

[0055] FIG. 6C shows a plan view of a head-tracking multi-view display 600 used in a sliding mode in an example according to an embodiment consistent with the principles described herein, where user 602 is positioned at a third position. At the third position, user 602 can view a first view (indicated by reference numeral 1) of the multi-view image across the entire multi-view image.

[0056] In some embodiments, the shift value may be selected for a first three-dimensional view of the multi-view image with respect to the left eye of user 118 and a second three-dimensional view of the multi-view image with respect to the right eye of user 118, and the first and second three-dimensional views are invariant as user 118 moves.

[0057] Generally, the shift value δN may not be an integer and may be the sum of an exclusive integer k between 0 and 1 and a decimal q. For example, in some embodiments, the shift value may correspond to a non-integer number of views of a multi-view image. As another example, in some embodiments, the shift value may be able to correspond to a non-integer number of light valves on a light valve array. If it may include shifting a view with a pure integer shift value δN to the angular position of another view, the decimal (e.g., non-integer) shift value δN may include combining adjacent views to form a combined view or a fractional view. There are many possible ways to combine adjacent views to form a combined view. In some embodiments, the processor may be configured to form a shifted view by combining two adjacent provisional views. In some embodiments, the processor may be configured to form a shifted view by non-linearly combining two adjacent provisional views, i.e., by non-linearly combining two adjacent provisional views.

[0058] In some embodiments, the processor is configured to form a shifted view by converting the pixel values of two adjacent provisional views from gamma space to linear space to form linear space pixel values, linearly adding the linear space pixel values to form a linear sum value, and converting the linear sum value from linear space to gamma space. For example, the non-linear mixing function f(q) can be defined to have a value of 0 when q = 0, a value of 0.5 when q = 0.5, and a value of 1 when q = 1. The dimensionless exponent γ can convert red-green-blue (RBG) pixel values from gamma space to linear space. The pixel values of view k can be represented by the quantity V k and the pixel values of the adjacent view k + 1 can be represented by the quantity V k+1 and the pixel values of the partial view k + q can be represented by the quantity V k+q and can be calculated as follows. V k+q = [(1 - f(q))(V k ) γ + (f(q))(V k+1 ) γ1 / γ (3) In some examples where different color sub - pixels correspond to different views, the shift value δN may be calculated separately for each color.

[0059] According to some embodiments of the principles described herein, a head - tracking multi - view display system is provided. The head - tracking multi - view display system is configured to provide or “display” a 3D or multi - view image representing a scene. In particular, the multi - view image is provided as a plurality of different “views” associated with the multi - view image. Different views may provide, for example, a “glasses - free” (e.g., autostereoscopic) representation of the information within the displayed multi - view image. Further, according to various embodiments, different sets of views may be provided for different positions or locations (e.g., head position) of a user of the head - tracking multi - view display system.

[0060] FIG. 7 shows a block diagram of a head - tracking multi - view display system 200 in an example according to an embodiment consistent with the principles described herein. The head - tracking multi - view display system 200 is configured to display a multi - view image according to different views in different view directions. In particular, the light beams emitted by the head - tracking multi - view display system 200 are used to display the multi - view image and may correspond to the pixels of different views (i.e., view pixels). Different views or equivalently different view directions are shown as arrows 202 emanating from the head - tracking multi - view display system 200 in FIG. 7. As provided below, the arrows 202 also represent the light beams emitted by the head - tracking multi - view display system 200.

[0061] ​The head-tracking multi-view display system 200 includes a head-tracking multi-view display 210. The head-tracking multi-view display 210 includes a multi-beam backlight configured to provide a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image. The head-tracking multi-view display 210 receives a plurality of provisional views of the multi-view image, the plurality of provisional views corresponding to different view directions, receives information regarding the tracking position of the user 230, and includes a processor configured to shift the provisional views with respect to the view directions to form a plurality of shifted views corresponding to different view directions, the shifted views varying as a function of the tracking position of the user 230 and being shifted by a shift value that varies across the field of view of the multi-view image. The head-tracking multi-view display 210 includes a light valve array configured to modulate light beams among the plurality of light beams to provide the plurality of shifted views of the multi-view image in the view directions as the multi-view image.

[0062] The head-tracking multi-view display system 200 includes a head tracker 220 configured to provide the tracking position of the user 230 to the processor of the head-tracking multi-view display 210. The head tracker 220 can include a camera having a field of view 222 that includes the user 230. The camera is configured to capture an image of the user 230. The head tracker 220 can further include an image processor configured to determine the position of the user 230 within the captured image to provide the tracking position.

[0063] According to other embodiments of the principles described herein, an operating method for a head-tracking multi-view display is provided. FIG. 8 shows a flowchart of an operating method 300 for a head-tracking multi-view display in an example according to an embodiment consistent with the principles described herein.

[0064] As shown in FIG. 8, the operation method 300 of the head-tracking multi-view display includes a step 310 of providing a plurality of light beams having different main angular directions corresponding to different view directions of the multi-view image by using a multi-beam backlight. The multi-beam backlight may have the same structure and function as the multi-beam backlight 106.

[0065] As shown in FIG. 8, the operation method 300 of the head-tracking multi-view display further includes a step 320 of receiving a plurality of provisional views of the multi-view image by using a processor. The plurality of provisional views correspond to different view directions. The provisional view can represent an original, unchanged, or unshifted portion of the 3D image to be displayed to the user using the multi-view display. The processor may have the same structure and function as the processor 116.

[0066] As shown in FIG. 8, the operation method 300 of the head-tracking multi-view display further includes a step 330 of shifting the provisional view with respect to the view direction by using a processor to form a plurality of shifted views corresponding to different view directions. The shifted view changes as a function of the user's tracking position and is shifted by a shift value that changes across the field of view of the multi-view image.

[0067] As shown in FIG. 8, the operation method 300 of the head-tracking multi-view display further includes a step 340 of modulating the light beams among the plurality of light beams by using a light valve array to provide the plurality of shifted views of the multi-view image as the multi-view image in the view direction. The light valve array may have the same structure and function as the light valve array 120.

[0068] In some embodiments, the method 300 of operating a head-tracking multi-view display further includes combining two adjacent provisional views with a processor to form a shifted view. In some embodiments, the method 300 of operating a head-tracking multi-view display further includes non-linearly combining two adjacent provisional views using a processor to form a shifted view. In some embodiments, the processor forms linear space pixel values by converting the pixel values of two adjacent provisional views from a gamma space to a linear space, linearly adds the linear space pixel values to form a linear sum value, and forms a shifted view by converting the linear sum value from the linear space to the gamma space. In some embodiments, the shift value corresponds to a non-integer number of views of the multi-view image. In some embodiments such as the peeling mode described above, the processor selects a shift value such that the user's movement enables the user to view the views in a progression that matches the progression of the provisional views. In some embodiments such as the peeling mode described above, the processor selects a shift value such that the user's movement provides the user with a sense of parallax as the views progress.

[0069] In some embodiments such as the sliding mode described above, the processor selects a shift value such that the shifted view follows the user's movement and enables the user to view only a single view or a single combination of multiple views as the user moves. In some embodiments such as the sliding mode described above, the processor selects a shift value to provide the user with a first three-dimensional view of the multi-view image to the user's left eye and a second three-dimensional view of the multi-view image to the user's right eye. The first and second three-dimensional views may remain unchanged as the user moves.

[0070] Thus, examples and embodiments of a head-tracking multi-view display, a head-tracking multi-view display system, and an operating method for a head-tracking multi-view display are described that shift a provisional view relative to a view direction to form a plurality of shifted views corresponding to different view directions, where the shifted views change as a function of a user's tracked position and are shifted by a shift value that varies across the field of view of the multi-view image. The examples above are merely illustrative of some of the many specific examples that represent the principles described herein. Clearly, those skilled in the art can readily devise many other configurations without departing from the scope defined by the following claims.

Claims

1. A head-tracking multi-view display, comprising: a multi-beam backlight configured to provide a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image; a processor, receiving a plurality of provisional views of the multi-view image, the plurality of provisional views corresponding to the different view directions, receiving information regarding a tracking position of a user, configured to shift the provisional views with respect to the view directions to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracking position of the user and that varies across a field of view of the multi-view image; a light valve array configured to modulate a light beam of the plurality of light beams to provide the plurality of shifted views of the multi-view image as the multi-view image in the view directions; A head-tracking multi-view display comprising the above.

2. The head-tracking multi-view display according to claim 1, wherein the processor is further configured to shift the provisional views with respect to the view directions and form the shifted views by combining two adjacent provisional views.

3. The head-tracking multi-view display according to claim 2, wherein the processor is further configured to form the shifted views by non-linearly combining the two adjacent provisional views.

4. The processor is further configured to: convert pixel values of the two adjacent provisional views from a gamma space to a linear space to form linear space pixel values; linearly add the linear space pixel values to form a linear sum value; convert the linear sum value from the linear space to the gamma space; to form the shifted views. The head-tracking multi-view display according to claim 3.

5. The head-tracking multi-view display according to claim 1, wherein the shift value corresponds to a non-integer number of views of the multi-view image.

6. The head-tracking multi-view display according to claim 1, wherein the shift value is selected so that the movement of the user enables the user to view the view in a progression that matches the progression of the provisional view.

7. The head-tracking multi-view display according to claim 6, wherein the shift value is selected so that the movement of the user gives a sense of parallax as the view progresses.

8. The head-tracking multi-view display according to claim 1, wherein the shifted view follows the movement of the user, and the shift value is selected so that the user can view only a single view or a single combination of multiple views as the user moves.

9. The head-tracking multi-view display according to claim 8, wherein the shift value is selected to provide a first three-dimensional view of the multi-view image to the left eye of the user and a second three-dimensional view of the multi-view image to the right eye of the user, and the first and second three-dimensional views are invariant when the user moves.

10. The head-tracking multi-view display according to claim 1, further comprising a head tracker configured to provide the tracking position of the user, the head tracker including a camera configured to capture an image of the user, and the head tracker further including an image processor configured to determine the position of the user in the captured image to provide the tracking position.

11. A method of operation for a head-tracking multi-view display, comprising: providing, using a multi-beam backlight, a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image; receiving, using a processor, a plurality of provisional views of the multi-view image, the plurality of provisional views corresponding to the different view directions; receiving, using the processor, information regarding a tracking position of a user; A step of using the processor to shift the provisional view with respect to the view direction to form a plurality of shifted views corresponding to different view directions, wherein the shifted views are shifted by a shift value that changes as a function of the tracking position of the user and that changes across the field of view of the multi-view image, step; A step of modulating a light beam among the plurality of light beams in order to provide the plurality of shifted views of the multi-view image as the multi-view image in the view direction using a light valve array; A method comprising: **Claim 12** The method according to claim 11, further comprising a step of combining two adjacent provisional views in order to form the shifted view using the processor. The method according to claim 11. **Claim 13** The method according to claim 11, further comprising a step of non-linearly combining two adjacent provisional views in order to form the shifted view using the processor. The method according to claim 11. **Claim 14** The processor A step of converting pixel values of the two adjacent provisional views from a gamma space to a linear space to form linear space pixel values; A step of linearly adding the linear space pixel values to form a linear sum value; A step of converting the linear sum value from the linear space to the gamma space; The method according to claim 13, wherein the shifted view is formed by the above. **Claim 15** The method according to claim 11, wherein the shift value corresponds to a non-integer number of views of the multi-view image. **Claim 16** The method according to claim 11, wherein the processor selects the shift value such that the movement of the user enables the user to view the view in a progression that matches the progression of the provisional view. **Claim 17** The method according to claim 16, wherein the processor selects the shift value such that the movement of the user gives a sense of parallax as the view progresses. **Claim 18** The method according to claim 11, wherein the processor selects the shift value such that the shifted view follows the movement of the user and enables the user to view only a single view or a single combination of a plurality of views as the user moves. **Claim 19** The method according to claim 18, wherein the processor selects the shift value so as to provide a first three-dimensional view of the multi-view image to the left eye of the user and a second three-dimensional view of the multi-view image to the right eye of the user, and the first and second three-dimensional views are invariant when the user moves.

20. A head-tracking multi-view display, a head tracker configured to provide a tracking position of a user with respect to the head-tracking multi-view display, the head tracker including a camera configured to capture an image of the user, and the head tracker further including an image processor configured to determine a position of the user in the captured image to provide the tracking position, a head tracker; a multi-beam backlight configured to provide a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image; a processor, receiving a plurality of provisional views of the multi-view image, the plurality of provisional views corresponding to the different view directions, shifting the provisional views with respect to the view directions to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracking position of the user and varies across the field of view of the multi-view image, a processor configured as such; a light valve array configured to modulate a light beam among the plurality of light beams to provide the plurality of shifted views of the multi-view image in the view directions as the multi-view image; A head-tracking multi-view display including.

Images