Head tracking multi-view display and method

Abstract

A head tracking multi-view display includes a multi-beam backlight that provides a plurality of light beams having different principal angular directions corresponding to different view directions of a multi-view image. A processor receives a plurality of temporary views of a multi-view image. The temporary views correspond to different view directions. A processor receives information about a tracked location of a user. The processor shifts the temporary view relative to the view direction to form a plurality of shifted views corresponding to different view directions. The shifted view is shifted by a shift value that varies as a function of the tracking position of the user and over the field of view of the multi-view image. The light valve array modulates the plurality of light beams to provide a plurality of shifted views of the multi-view image as the multi-view image in a view direction.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 340,361, filed on May 10, 2022, the entire contents of which are incorporated herein by reference.

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0004] none Background Art

[0005] Electronic displays are almost ubiquitous media for transmitting information to users of a wide variety of devices and products. The most commonly used electronic displays include cathode ray tubes (CRTs), plasma display panels (PDPs), liquid crystal displays (LCDs), electroluminescent displays (ELs), organic light emitting diodes (OLEDs), and active matrix OLED (AMOLED) displays, electrophoretic displays (EPs), and various displays that employ electromechanical or electrofluidic light modulation (e.g., digital micromirror devices, electrowetting displays, etc.). Typically, electronic displays can be classified as active displays (i.e., displays that emit light) or passive displays (i.e., displays that modulate light provided by another source). Examples of active displays include CRTs, PDPs, and OLED / AMOLEDs. Displays that are typically classified as passive when considering emitted light are LCD and EP displays. Although passive displays typically exhibit attractive performance characteristics, including but not limited to inherent low power consumption, they may find some limitations in many practical applications due to their lack of luminous ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The various features of examples and embodiments according to the principles described herein may be more readily understood by reference to the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals represent like structural elements, and in which:

[0007] Figure 1 A perspective view of a multi-view display is shown in an example of an embodiment consistent with the principles described herein.

[0008] Figure 2 A graphical representation of angular components of light beams having a particular principal angle direction corresponding to a view direction of a multi-view display is shown, according to an example of an embodiment consistent with the principles described herein.

[0009] Figure 3 A cross-sectional view of a head-tracking multi-view display is shown in an example of an embodiment consistent with the principles described herein.

[0010] Figure 4A plan view of an example of a shift-invariant head-tracking multi-view display is shown, where the shift value does not vary across the field of view of the multi-view images.

[0011] Figure 5A A plan view of a head-tracking multi-view display used in a peel-away mode with a user at a first position is shown, according to an example of an embodiment consistent with the principles described herein.

[0012] Figure 5B A plan view of a head-tracked multi-view display used in a peel-away mode with a user at a second position is shown, according to an example of an embodiment consistent with the principles described herein.

[0013] Figure 5C A plan view of a head-tracked multi-view display used in a peel-away mode with a user at a third position is shown, according to an example of an embodiment consistent with the principles described herein.

[0014] Fig. 6A A plan view of a head tracking multi-view display used in a sliding mode with a user at a first position is shown, according to an example of an embodiment consistent with the principles described herein.

[0015] Figure 6B A plan view of a head tracking multi-view display used in a sliding mode with a user at a second position is shown, according to an example of an embodiment consistent with the principles described herein.

[0016] Figure 6C A plan view of a head tracking multi-view display used in a sliding mode with a user at a third position is shown, according to an example of an embodiment consistent with the principles described herein.

[0017] Figure 7 A block diagram of a head tracking multi-view display system is shown in an example of an embodiment consistent with the principles described herein.

[0018] Figure 8 A flowchart is shown of a method of operation for a head tracking multi-view display in an example of an embodiment consistent with the principles described herein.

[0019] Certain examples and embodiments have other features that are in addition to or in lieu of the features shown in the above drawings. These and other features are described in detail below with reference to the above drawings. DETAILED DESCRIPTION

[0020] Examples and embodiments according to the principles described herein provide a multi-view or three-dimensional (3D) image display that employs information about a user's position over time, sometimes referred to as "head tracking" or alternatively referred to as "tracking the head." Embodiments consistent with the principles described herein may employ a multi-view display to provide a different set of views of a scene represented by a multi-view image based on the user's position. A head-tracking multi-view display may receive temporary views of a multi-view image corresponding to different view directions. A head-tracking (or tracking head) multi-view display may receive information about a user's tracking position, such as from a head tracking sensor. A head-tracking (or tracking head) multi-view display may shift a temporary view relative to a view direction to form a shifted view corresponding to a different view direction. The shifted view may shift a shift value that varies as a function of the user's tracking position and varies over the field of view of the multi-view image.

[0021] It may be beneficial to use a shift value that changes over the field of view of a multi-view image. For example, in a display using a shift value that does not change over the field of view of a multi-view image, a user may perceive different views of the multi-view image at different locations on the display, which may make the user feel uneasy or uncoordinated. As a specific example, a user may simultaneously perceive a portion of the display as showing a first view and another portion of the display as showing a second view. In contrast, in a display in which a shift value changes over the field of view of a multi-view image as described in detail below, a user may perceive the same view over the full field of view of the multi-view image. Presenting a multi-view image so that the user perceives the same view over the field of view of the multi-view image (e.g., avoiding a situation in which the user perceives that a portion of the display shows a first view and another portion of the display shows a second view) can provide a more realistic viewing experience for the user. In a specific example, one eye of the user may perceive the first view over the full field of view of the multi-view image, and the other eye of the user may perceive the second view over the full field of view of the multi-view image, so that the user may experience a stereoscopic effect.

[0022] In various embodiments, a head tracking sensor or algorithm may provide information about the position or location of a user relative to a multi-view display. That is, the user location may be determined or inferred by tracking the location of the user's head or other anatomical features. For ease of discussion herein and not limitation, embodiments or components described herein may be referred to as including or using "head tracking," such as in a multi-view display, system, and method that employs head tracking.

[0023] Figure 1 1 shows a perspective view of a multi-view display 10 in an example of an embodiment consistent with the principles described herein. Figure 1As shown, the multi-view display 10 includes a screen 12 to display a multi-view image to be viewed. The multi-view display 10 provides different views 14 of the multi-view image in different view directions 16 relative to the screen 12. The view directions 16 are shown as arrows extending from the screen 12 in various different principal directions. The different views 14 are shown as shaded polygonal boxes at the ends of the arrows (i.e., depicting the view directions 16). Only four views 14 and four view directions 16 are shown, all of which are by way of example and not limitation. Note that although the different views 14 are shown in FIG. Figure 1 14 is shown as being above the screen, but when the multi-view images are displayed on the multi-view display 10, the view 14 actually appears on or near the screen 12. The view 14 is depicted above the screen 12 only to simplify the explanation and is intended to represent that the multi-view display 10 is viewed from a respective one of the view directions 16 corresponding to the particular view 14.

[0024] According to the definition herein, a view direction, or equivalently a light beam having a direction corresponding to the view direction of a multi-view display, generally has a principal angle direction given by an angular component {θ, φ}. The angular component θ is referred to herein as the "elevation component" or "elevation angle" of the light beam. The angular component φ is referred to as the "azimuth component" or "azimuth angle" of the light beam. According to the definition, the elevation angle θ is an angle in a vertical plane (e.g., perpendicular to the plane of the multi-view display screen), while the azimuth angle φ is an angle in a horizontal plane (e.g., parallel to the plane of the multi-view display screen). Figure 2 FIG. 1 shows a plurality of view directions (eg, a plurality of view directions) associated with a multi-view display according to an example of an embodiment consistent with the principles described herein. Figure 1 16 in the multi-view display). In addition, the light beam 20 is emitted or emanates from a specific point according to the definition herein. That is, the light beam 20 has a central ray associated with a specific origin within the multi-view display according to the definition. Figure 2 The beam (or view direction) origin O is also shown.

[0025] Furthermore, as used herein, the term "multi-view" as used in the terms "multi-view image" and "multi-view display" is defined as a plurality of views representing different viewing angles or angular disparity between views including a plurality of views. Additionally, as defined herein, the term "multi-view" explicitly includes more than two different views (i.e., a minimum of three views and typically more than three views) herein. Thus, a "multi-view display" as used herein is explicitly distinguished from a stereoscopic display that includes only two different views to represent a scene or image. Note, however, that although a multi-view image and a multi-view display include more than two views, as defined herein, a multi-view image may be viewed (e.g., on a multi-view display) as a stereoscopic image pair by selecting only two of the multi-view views to be viewed at a time (e.g., one view for each eye).

[0026] A "multi-view pixel" is defined herein as a set of sub-pixels representing a "view" pixel in each of a plurality of different views of a multi-view display. Specifically, a multi-view pixel may have separate sub-pixels corresponding to or representing view pixels in each different view of a multi-view image. In addition, according to the definition herein, the sub-pixels of a multi-view pixel are so-called "directional pixels" because each of the view pixels is associated with a predetermined view direction of a corresponding view in the different views. In addition, according to various examples and embodiments, different view pixels represented by the sub-pixels of a multi-view pixel may have identical or at least substantially similar positions or coordinates in each of the different views. For example, a first multi-view pixel may have separate sub-pixels corresponding to view pixels located at {x1, y1} in each of the different views of the multi-view image, while a second multi-view pixel may have separate sub-pixels corresponding to view pixels located at {x2, y2} in each of the different views, and so on.

[0027] In this document, a "lightguide" is defined as a structure that guides light within the structure using total internal reflection. Specifically, a lightguide may include a core that is substantially transparent at the operating wavelength of the lightguide. In various examples, the term "lightguide" generally refers to a dielectric optical waveguide that uses total internal reflection to guide light at the interface between the dielectric material of the lightguide and the material or medium surrounding the lightguide. By definition, the condition for total internal reflection is that the refractive index of the lightguide is greater than the refractive index of the surrounding medium adjacent to the surface of the lightguide material. In some embodiments, in addition to or in lieu of the above-mentioned refractive index difference, the lightguide may include a coating to further promote total internal reflection. The coating may be, for example, a reflective coating. The lightguide may be any of several lightguides, including but not limited to one or both of a plate or slab lightguide and a strip lightguide.

[0028] Furthermore, in this document, the term "plate" when applied to a lightguide as in a "plate lightguide" is defined as a segmented or differently planar layer or sheet, which is sometimes referred to as a "flat plate" lightguide. Specifically, a plate lightguide is defined as a lightguide configured to guide light in two substantially orthogonal directions defined by a top surface and a bottom surface (i.e., opposing surfaces) of the lightguide. Furthermore, according to the definition herein, the top surface and the bottom surface are both separated from each other and may be substantially parallel to each other, at least in a differential sense. That is, within any small differential portion of the plate lightguide, the top surface and the bottom surface are substantially parallel or coplanar.

[0029] In some embodiments, the plate light guide may be substantially flat (i.e., confined to a plane), and thus, the plate light guide is a planar light guide. In other embodiments, the plate light guide may be curved in one or two orthogonal dimensions. For example, the plate light guide may be curved in a single dimension to form a cylindrical plate light guide. However, any curvature has a sufficiently large radius of curvature to ensure that total internal reflection is maintained within the plate light guide to guide light.

[0030] As used herein, a "light source" is defined as a source of light (e.g., a light emitter configured to generate and emit light). For example, a light source may include an optical emitter, such as a light emitting diode (LED) that emits light when activated or turned on. Specifically, as used herein, a light source may be substantially any light source or include substantially any optical emitter, including but not limited to one or more of a light emitting diode (LED), a laser, an organic light emitting diode (OLED), a polymer light emitting diode, a plasma-based optical emitter, a fluorescent lamp, an incandescent lamp, and virtually any other light source. The light generated by the light source may have a color (i.e., may include light of a specific wavelength), or may be a range of wavelengths (e.g., white light). In some embodiments, the light source may include a plurality of optical emitters. For example, the light source may include a collection or group of optical emitters, wherein at least one of the optical emitters generates light having a color or an equivalent wavelength that is different from the color or wavelength of light generated by at least one other optical emitter in the collection or group. For example, the different colors may include primary colors (e.g., red, green, blue).

[0031] In this document, a "multi-view image" is defined as a plurality of images (i.e., greater than three images), wherein each image of the plurality of images represents a different view corresponding to a different view direction of the multi-view image. Thus, for example, a multi-view image is a collection of images (e.g., two-dimensional images) that, when displayed on a multi-view display, can facilitate the perception of depth and, therefore, appear to be an image of a 3D scene to a user. Multi-view images that provide pairs of views representing different but related perspectives of a 3D scene consistent with viewing by a user are defined as 3D images.

[0032] Embodiments consistent with the principles described herein may be implemented using a variety of devices and circuits, including but not limited to integrated circuits (ICs), very large scale integrated circuits (VLSI) circuits, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), graphics processor units (GPUs), etc. One or more, firmware, software (such as program modules or instruction sets), and combinations of two or more of the above. For example, an embodiment or an element thereof may be implemented as a circuit element within an ASIC or VLSI circuit. An implementation using an ASIC or VLSI circuit is an example of a hardware-based circuit implementation.

[0033] In another example, an embodiment may be implemented as software using a computer programming language (e.g., C / C++) that is executed in an operating environment or a software-based modeling environment (e.g., The operating environment or software-based modeling environment is further executed by a computer (e.g., stored in a memory and executed by a processor or graphics processor of a general-purpose computer). Note that one or more computer programs or software may constitute a computer program mechanism, and a programming language may be compiled or interpreted, for example, configurable or configured (which may be used interchangeably in this discussion) to be executed by a processor or graphics processor of a computer.

[0034] In another example, a block, module or element of an apparatus, device or system (e.g., an image processor, a camera, etc.) described herein may be implemented using an actual or physical circuit (e.g., as an IC or ASIC), while another block, module or element may be implemented in software or firmware. Specifically, according to the definition herein, for example, some embodiments may be implemented using a substantially hardware-based circuit method or device (e.g., IC, VLSI, ASIC, FPGA, DSP, firmware, etc.), while other embodiments may also be implemented as software or firmware using a computer processor or graphics processor to execute software, or as a combination of software or firmware and hardware-based circuits.

[0035] In addition, as used herein, the article "a" is intended to have its ordinary meaning in the patent field, i.e., "one or more". For example, "lens" refers to one or more lenses, and therefore, "the lens" refers to "one or more lenses" in this article. In addition, any reference to "top", "bottom", "upper", "lower", "up", "lower", "front", "back", "first", "second", "left" or "right" in this article is not intended to be a limitation of this article. In this article, the term 'about' when applied to a value generally means within the tolerance range of the device used to produce the value, or may mean plus or minus 10%, or plus or minus 5%, or plus or minus 1%, unless otherwise expressly stated. In addition, the term "substantially" as used herein means most, or almost all, or all, or an amount in the range of about 51% to about 100%. In addition, the examples in this article are intended to be illustrative only and are presented for discussion purposes rather than as limitations.

[0036] According to some embodiments of the principles described herein, there is provided a head tracking multi-view display 100. In some embodiments, the head tracking multi-view display 100 may be referred to as a tracking head multi-view display. Figure 3 A cross-sectional view of a head-tracking multi-view display 100 is shown in an example of an embodiment consistent with the principles described herein. The head-tracking multi-view display 100 is configured to provide multiple views of a scene as a multi-view image, i.e., a displayed multi-view image. Specifically, the head-tracking multi-view display 100 provides multiple views in corresponding multiple view directions. Figure 3 In FIG. 1 , view directions, or equivalently views in a plurality of views, are depicted as arrows 102 extending from the head tracking multi-view display 100 pointing in different angular directions. For convenience, view directions, or equivalently views in a plurality of views, are labeled with view identification numbers 104. Figure 3 In the specific example of FIG. 1 , the head tracking multi-view display 100 provides eight views; other numbers of views may also be used. Figure 3 The views in FIG. 1 are identified by view identification numbers 104 as 1 through 8; other identifiers, such as letters A through H, may also be used. Figure 3 The view identification number 104 is for convenience only and does not imply that the views are sequential or ordered in any way.

[0037] like Figure 3 As shown, the head tracking multi-view display 100 includes a multi-beam backlight 106. The multi-beam backlight 106 is configured to provide a plurality of light beams having different principal directions corresponding to different view directions of the multi-view image. Figure 3In a configuration of , the multi-beam backlight 106 may include a light source 108 that is controlled by a controller 110. The light source 108 may direct light into an edge of a light guide 112. The light guide 112 may propagate the light as guided light within the light guide 112. A plurality of multi-beam elements 114 may direct respective portions of the guided light out of the light guide 112 to form a plurality of light beams. Figure 3 The configuration of the multi-beam backlight 106 shown in FIG. 1 is only one example of a multi-beam backlight 106 ; other suitable configurations may also be used.

[0038] like Figure 3 As shown, the head tracking multi-view display 100 also includes a processor 116. Figure 3 In an example of , the processor 116 is included in the controller 110. In other configurations, the processor 116 may be separated from the controller 110. The processor 116 is configured to receive multiple temporary views of the multi-view image. The multiple temporary views may correspond to different view directions. The processor 116 is also configured to receive information about the tracking position of the user 118, such as including information about the direction or distance between the user 118 and the light-emitting portion of the display. In an example, the tracking position of the user 118 may include information about the view direction of the user 118. The processor 116 is also configured to shift the temporary view relative to the view direction to form multiple shifted views corresponding to different view directions. In various embodiments, the shifted views are shifted by or according to a shift value. The shift value may vary as a function of the tracking position of the user 118. In various embodiments, the shift value may also vary over the field of view of the multi-view image. Details on the calculation of the shift value are provided below.

[0039] like Figure 3 As shown, the head tracking multi-view display 100 further includes a light valve array 120. The light valve array 120 is configured to modulate light beams in the plurality of light beams to provide a plurality of shifted views of a multi-view image to a view direction as a multi-view image. The light valve array 120 may be controlled by a controller 110. In various embodiments, any of a variety of different types of light valves may be used as the light valves of the light valve array 120, including but not limited to one or more of a liquid crystal light valve, an electrophoretic light valve, and a light valve based on electrowetting.

[0040] like Figure 3As shown, the head tracking multi-view display 100 also includes a head tracker 122. The head tracker 122 is configured to provide a tracked position of the user 118 (e.g., the head of the user 118, or one or both eyes of the user 118, or another anatomical feature of the user 118). The head tracker 122 may include a camera configured to capture an image of the user 118. The head tracker 122 may also include an image processor (or a general-purpose computer programmed as an image processor) configured to determine the position of the user 118 within the captured image to provide the tracked position. In some examples, the processor 116 may include an image processor of the head tracker 122, such as by performing operations using the same processing circuit. In other examples, the processor 116 may be separate from the image processor of the head tracker 122. Other suitable head trackers may also be used, including a lidar-based head tracker (e.g., using the time of flight of reflected light on the viewing scene to determine the distance to one or more objects in the scene (such as the user's head or the user's eyes)) or other technologies. The output of the head tracker 122 may be used to modify the operation of the head-tracked multi-view display 100 (e.g., the modulation of the light beams by the light valve array 120). For example, the determined position of the user 118 may be provided to one or both of a processor (such as the processor 116) and a light valve driver (e.g., the driver circuit or controller 110) of the head-tracked multi-view display 100 to adjust the emission pattern from the head-tracked multi-view display 100 to correspond to the position of the user. Other examples of head tracker 122 implementations may include any of a variety of two-dimensional (2D) and three-dimensional (3D) object tracking systems, such as, but not limited to, Object tracking system. is a registered trademark of Microsoft Corporation in Redmond, Washington.

[0041] As mentioned above and explained in detail below, Figure 3 The head tracking multi-view display 100 determines a shift value that varies across the field of view of the multi-view image. To demonstrate the effect of varying the shift value across the field of view of the multi-view image, Figure 4 A plan view of an example of a shift-invariant head-tracking multi-view display 400 is shown, in which the shift value does not vary over the field of view of the multi-view images. Instead, the shift value is constant or invariant over the field of view of the multi-view images, such as over the field of view of the active surface area of ​​the shift-invariant head-tracking multi-view display 400.

[0042] exist Figure 4In the example of FIG. 4 , different regions of the multi-view image may be visible in different views of the multi-view image for some positions of the user 402. For example, the upper left corner of the multi-view image may be visible in view number 1, the center portion of the multi-view image may be visible in view number 2, and the lower right corner of the multi-view image may be visible in view number 3. Because the image perceived by the user 402 at the user's position relative to the shift-invariant head tracking multi-view display 400 may include regions from different views of the multi-view image, the perceived image may be distorted or may contain artifacts from misalignment of the views or at the boundaries between different view portions.

[0043] Changing the shift value across the field of view of the multi-view image can help overcome the disadvantages of the shift-invariant head-tracking multi-view display 400. In particular, changing the shift value across the full field of view of the multi-view image can help ensure that all or substantially all of the multi-view image shows a single view (or a single combination of views) to the user. The following paragraphs describe in detail examples of determining the shift value.

[0044] Processor 116 may determine the shift values ​​to provide two general functions for head-tracked multi-view display 100. A first general function of the shift values ​​is to direct a specified view to the eyes of user 118 as user 118 changes position relative to the display. A second general function of the shift values ​​is to ensure that only a single specified view (or specified combination of views) is visible across the entire multi-view image (such as extending over a specified active surface area of ​​the head-tracked multi-view display), such that adjacent views or pseudo-views are not visible. In other words, the calculated shift values, as described in detail below, may ensure that user 118 cannot see a portion of the multi-view image in a first view and cannot see another portion of the multi-view image in a second view, thereby avoiding Figure 4 An example of a shift-invariant head-tracked multi-view display 400 .

[0045] The processor 116 may include a mathematical model that predicts which view is visible to the user 118 as a function of the user 118 position and as a function of the user 118 view direction at or above the surface area of ​​the head-tracked multi-view display. The view may be represented as a real number. For example, for a multi-view image including eight views, the view may be represented by an integer N, where N may have an integer value from one to eight inclusive. The user 118 may have a center head position represented by spatial coordinates (x, y, z). The head-tracked multi-view display may have a pixel position represented by spatial coordinates (x0, y0) within the effective surface area of ​​the head-tracked multi-view display. The mathematical model may determine a view number N (x, y, z, x0, y0), which is a function of the user position and the pixel position. In other words, for each combination of user position (x, y, z) and pixel position (x0, y0), the mathematical model may determine which view N is visible.

[0046] The shift value, represented as the quantity δN, is a dimensionless quantity representing a correction or addition to the view number N. For example, a shift value δN of +1 may involve incrementing the view number (e.g., view number 3) by a value of +1 (e.g., to view number 4). The shift value δN may vary from pixel position to pixel position over the active surface area of ​​the head-tracking multi-view display (e.g., over the field of view of the multi-view image). Allowing the shift value δN to vary over the field of view of the multi-view image may ensure that the user 118 sees only one specified multi-view image (or one specified combination of multi-view images) over the entire multi-view image.

[0047] The processor may calculate the shift values ​​to achieve a particular effect in the perceived view. Two such effects are referred to hereinafter as "peeling mode" and "sliding mode", both of which are explained in more detail below.

[0048] In peel-off mode, as user 118 changes positioning, views may advance from view to view, such that user 118 may view a multi-view image from a number of different views depending on the positioning of user 118. In some embodiments, in peel-off mode, the shift values ​​may be selected such that the movement of user 118 allows user 118 to view the views in a progression that matches the progression of the temporary views. In some embodiments, in peel-off mode, the shift values ​​may be selected such that the movement of user 118 provides the user with a sense of parallax as the views advance.

[0049] The peeling mode can extend the view area of ​​the head-tracked multi-view display without changing the parallax effect. In the peeling mode, the shift value δN can be calculated as a function of a static view number N0 that is unchanged relative to the user position and the pixel position, a center view number N(x, y, z, 0, 0) indicating which view is visible from the user position at the center of the display, and a pixel-relative view number N(x, y, z, x0, y0) indicating which view is visible from the user position at a specified pixel position, as shown below:

[0050] δN=N0 + N(x,y,z,0,0)- N(x,y,z,x0,y0) (1)

[0051] Figure 5A A plan view of a head-tracked multi-view display 500 used in a peel-away mode is shown with a user 502 at a first position, according to an example of an embodiment consistent with the principles described herein. At the first position, the user 502 can view a first view (indicated by numeral 1) of a multi-view image over the entire multi-view image.

[0052] Figure 5B A plan view of a head-tracked multi-view display 500 used in a peeled-away mode is shown with a user 502 at a second position, according to an example of an embodiment consistent with the principles described herein. At the second position, the user 502 can view a second view of the multi-view image (indicated by the number 2) over the entire multi-view image.

[0053] Figure 5C A plan view of a head-tracked multi-view display 500 used in a peel-away mode is shown with a user 502 at a third position, according to an example of an embodiment consistent with the principles described herein. At the third position, the user 502 can view a third view (indicated by numeral 3) of the multi-view image over the entire multi-view image.

[0054] In sliding mode, a single view (or a single combination of views) follows the user 118 as the user 118 changes positioning. For example, the sliding mode may effectively act as an image stabilizer, which may ensure that the user 118 sees only a specified view (or a specified combination of views) and prevent other views from being visible as the user 118 changes positioning. In some embodiments, in sliding mode, the shift value may be selected so that the shifted view follows the movement of the user 118 and allows the user 118 to see only a single view or a single combination of views as the user 118 moves. The sliding mode may ensure that the user 118 experiences a uniform static view without parallax, represented by the quantity N0. In sliding mode, head tracking may effectively counteract any movement of the user 118 and may provide the user 118 with a 3D experience similar to using 3D glasses in a movie theater.

[0055] Using the same quantities defined above, the shift value δN can be calculated as follows:

[0056] δN= N0 - N(x, y, z, x0, y0) (2)

[0057] Fig. 6A A plan view of a head-tracking multi-view display 600 used in a sliding mode is shown with a user 602 at a first position, according to an example of an embodiment consistent with the principles described herein. At the first position, the user 602 can view a first view (indicated by numeral 1) of a multi-view image over the entire multi-view image.

[0058] Figure 6B A plan view of a head-tracking multi-view display 600 used in a sliding mode with a user 602 at a second position is shown, according to an example of an embodiment consistent with the principles described herein. At the second position, the user 602 can view a first view (indicated by numeral 1) of the multi-view image over the entire multi-view image.

[0059] Figure 6C A plan view of a head-tracked multi-view display 600 used in a sliding mode is shown with a user 602 at a third position, according to an example of an embodiment consistent with the principles described herein. At the third position, the user 602 can view a first view of the multi-view image (indicated by numeral 1) over the entire multi-view image.

[0060] In some embodiments, the shift values ​​may be selected to be a first three-dimensional view of the multi-view image to the left eye of user 118 and a second three-dimensional view of the multi-view image to the right eye of user 118 , the first three-dimensional view and the second three-dimensional view being constant when user 118 moves.

[0061] In general, the shift value δN may not be an integer, but may be the sum of an integer k and a fractional q between zero and one (not included). For example, in some embodiments, the shift value may correspond to a non-integer number of views of the multi-view image. As another example, in some embodiments, the shift value may correspond to a non-integer number of light valves on the light valve array. Where a pure integer shift value δN may involve shifting one view to an angular position of another view, a fractional (e.g., non-integer) shift value δN may involve combining adjacent views to form a combined view or a fractional view. There are many possible ways to combine adjacent views to form a combined view. In some embodiments, the processor may be configured to form a shifted view by combining two adjacent temporary views. In some embodiments, the processor may be configured to form a shifted view by non-linearly combining two adjacent temporary views (i.e., combining two adjacent temporary views in a non-linear manner).

[0062] In some embodiments, the processor is configured to form the shifted view by converting pixel values ​​of two adjacent temporary views from a gamma space to a linear space to form a linear space pixel value, linearly summing the linear space pixel values ​​to form a linear sum value, and converting the linear sum value from the linear space to the gamma space. For example, a nonlinear blending function f(q) may be defined to have a value of 0 for q=0, a value of 0.5 for q=0, and a value of 1 for q=1, a dimensionless exponent γ may convert a red-green-blue (RBG) pixel value from a gamma space to a linear space, and a pixel value of view k may be calculated by the quantity V k Indicates that the pixel value of the adjacent view k+1 can be expressed by the quantity V k+1 Represented by V, and the pixel value of the fractional view k+q can be represented by V k+q It is represented and calculated as follows:

[0063] V k+q = [(1 - f(q))(V k ) γ + (f(q))(V k+1 ) γ ] 1 / γ (3)

[0064] In some examples where different color sub-pixels correspond to different views, the shift value δN may be calculated separately for each color.

[0065] According to some embodiments of the principles described herein, a head tracking multi-view display system is provided. The head tracking multi-view display system is configured to provide or "display" a 3D or multi-view image representing a scene. Specifically, the multi-view image is provided as a plurality of different "views" associated with the multi-view image. For example, the different views may provide a "glasses-free" (e.g., autostereoscopic) representation of information in the multi-view image being displayed. In addition, according to various embodiments, different sets of views may be provided for different positions or locations (e.g., head positions) of a user of the head tracking multi-view display system.

[0066] Figure 7 A block diagram of a head tracking multi-view display system 200 is shown in an example of an embodiment consistent with the principles described herein. The head tracking multi-view display system 200 is configured to display a multi-view image according to different views in different view directions. Specifically, light beams emitted by the head tracking multi-view display system 200 are used to display the multi-view image and may correspond to pixels of different views (i.e., view pixels). Figure 7, different views, or equivalently different view directions, are shown as arrows 202 emitted by the head tracking multi-view display system 200. As provided below, the arrows 202 also represent light beams emitted by the head tracking multi-view display system 200.

[0067] The head tracking multi-view display system 200 includes a head tracking multi-view display 210. The head tracking multi-view display 210 includes a multi-beam backlight configured to provide a plurality of light beams having different principal directions corresponding to different view directions of a multi-view image. The head tracking multi-view display 210 includes a processor configured to: receive a plurality of temporary views of the multi-view image, the plurality of temporary views corresponding to different view directions; receive information about a tracking position of a user 230; and shift the temporary views relative to the view directions to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracking position of the user 230 and varies over a field of view of the multi-view image. The head tracking multi-view display 210 includes a light valve array configured to modulate light beams of the plurality of light beams to provide a plurality of shifted views of the multi-view image to the view directions as the multi-view image.

[0068] The head tracking multi-view display system 200 includes a head tracker 220 configured to provide a tracked position of a user 230 to a processor of the head tracking multi-view display 210. The head tracker 220 may include a camera having a field of view 222, the field of view 222 including the user 230. The camera is configured to capture an image of the user 230. The head tracker 220 may also include an image processor configured to determine a position of the user 230 within the captured image to provide the tracked position.

[0069] According to other embodiments of the principles described herein, there is provided a method of operating a head-tracking multi-view display. Figure 8 A flow chart is shown of a method 300 of operation for a head tracking multi-view display in an example of an embodiment consistent with the principles described herein.

[0070] like Figure 8 As shown, a method 300 of operating a head-tracked multi-view display includes providing 310 a plurality of light beams having different principal angle directions corresponding to different view directions of a multi-view image using a multi-beam backlight. The multi-beam backlight may be similar to the multi-beam backlight 106 in structure and function.

[0071] like Figure 8As shown, the method 300 for operating a head-tracked multi-view display also includes receiving 320 a plurality of temporary views of the multi-view image using a processor. The plurality of temporary views correspond to different view directions. The temporary views may represent original, unmodified or unshifted portions of the 3D image to be displayed to the user using the multi-view display. The processor may be similar in structure and function to the processor 116.

[0072] like Figure 8 As shown, the method 300 for operating a head-tracked multi-view display further includes shifting 330 the temporary view relative to the view direction using a processor to form a plurality of shifted views corresponding to different view directions. The shifted views are shifted by a shift value that varies as a function of the tracking position of the user and varies across the field of view of the multi-view image.

[0073] like Figure 8 As shown, the method 300 for operating a head-tracked multi-view display further includes modulating 340 a light beam of the plurality of light beams using a light valve array to provide a plurality of shifted views of the multi-view image as the multi-view image in the viewing direction. The light valve array may be similar to the light valve array 120 in structure and function.

[0074] In some embodiments, the method 300 for operating a head-tracked multi-view display further comprises combining, with a processor, two adjacent temporary views to form a shifted view. In some embodiments, the method 300 for operating a head-tracked multi-view display further comprises combining, with a processor, two adjacent temporary views nonlinearly to form a shifted view. In some embodiments, the processor forms the shifted view by converting pixel values ​​of two adjacent temporary views from a gamma space to a linear space to form a linear space pixel value, linearly summing the linear space pixel values ​​to form a linear summation value, and converting the linear summation value from the linear space to the gamma space. In some embodiments, the shift value corresponds to a non-integer number of views of the multi-view image. In some embodiments, such as in the above-described peeling mode, the processor selects the shift value so that the user's movement allows the user to view the view with a progression that matches the progression of the temporary view. In some embodiments, such as in the above-described peeling mode, the processor selects the shift value so that the user's movement provides the user with a sense of parallax as the view advances.

[0075] In some embodiments, such as in the sliding mode described above, the processor selects the shift value so that the shifted view follows the user's movement and allows the user to view only a single view or a single combination of views as the user moves. In some embodiments, such as in the sliding mode described above, the processor selects the shift value to provide the user with a first three-dimensional view of the multi-view image to the user's left eye and a second three-dimensional view of the multi-view image to the user's right eye. The first three-dimensional view and the second three-dimensional view may be constant as the user moves.

[0076] Thus, examples and embodiments of head-tracked multi-view displays, head-tracked multi-view display systems, and methods of operation for head-tracked multi-view displays have been described, in which temporary views are shifted relative to a view direction to form multiple shifted views corresponding to different view directions, the shifted views being shifted by a shift value that varies as a function of the user's tracking position and varies over the field of view of the multi-view image. The above examples are merely illustrative of some of the many specific examples that represent the principles described herein. Clearly, many other arrangements can be readily devised by those skilled in the art without departing from the scope defined by the appended claims.

Claims

1. A head tracking multi-view display, comprising: a multi-beam backlight configured to provide a plurality of light beams having different principal angle directions corresponding to different view directions of a multi-view image; A processor, the processor being configured to: receiving a plurality of temporary views of the multi-view image, the plurality of temporary views corresponding to the different view directions; receiving information about the user's tracked location; as well as shifting the temporary view relative to the view direction to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracked position of the user and that varies across a field of view of the multi-view image; and A light valve array is configured to modulate light beams of the plurality of light beams to provide the plurality of shifted views of the multi-view image as the multi-view image to the view direction.

2. The head tracking multi-view display of claim 1, wherein: The processor is further configured to shift the provisional view with respect to the view direction by combining two adjacent provisional views to form the shifted view.

3. The head tracking multi-view display of claim 2, wherein: The processor is further configured to form the shifted view by non-linearly combining the two adjacent temporary views.

4. The head tracking multi-view display of claim 3, wherein: The processor is further configured to form the shifted view by: converting pixel values ​​of the two adjacent temporary views from a gamma space to a linear space to form linear space pixel values; performing linear summation on the linear space pixel values ​​to form a linear summation value; as well as The linear sum value is converted from linear space to gamma space.

5. The head tracking multi-view display of claim 1, wherein: The shift value corresponds to a non-integer number of views of the multi-view image.

6. The head tracking multi-view display of claim 1, wherein: The shift values ​​are selected such that the user's movement allows the user to view a view with a progression that matches the progression of the temporary view.

7. The head tracking multi-view display of claim 6, wherein: The shift values ​​are selected so that the user's movement provides the user with a perception of parallax as the views advance.

8. The head tracking multi-view display of claim 1, wherein: The shift values ​​are selected so that the shifted views follow the movement of the user and allow the user to see only a single view or a single combination of views as the user moves.

9. The head tracking multi-view display of claim 8, wherein: The shift values ​​are selected to provide the user with a first three-dimensional view of the multi-view images to the user's left eye and a second three-dimensional view of the multi-view images to the user's right eye, the first three-dimensional view and the second three-dimensional view being invariant when the user moves.

10. The head-tracking multi-view display of claim 1 , further comprising a head tracker configured to provide the tracked position of the user, the head tracker comprising a camera configured to capture an image of the user, the head tracker further comprising an image processor configured to determine a position of the user within the captured image to provide the tracked position.

11. A method for operating a head-tracking multi-view display, the method comprising: providing a plurality of light beams having different principal angle directions corresponding to different view directions of a multi-view image using a multi-beam backlight; receiving, with a processor, a plurality of temporary views of the multi-view image, the plurality of temporary views corresponding to the different view directions; receiving, with the processor, information regarding a tracked location of a user; shifting, with the processor, the temporary view relative to the view direction to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracked position of the user and that varies across a field of view of the multi-view image; as well as Light beams of the plurality of light beams are modulated using a light valve array to provide the plurality of shifted views of the multi-view image as the multi-view image to the view directions.

12. The method according to claim 11, further comprising: Two adjacent temporary views are combined, using the processor, to form the shifted view.

13. The method according to claim 11, further comprising: Two adjacent temporal views are non-linearly combined using the processor to form the shifted view.

14. The method according to claim 13, wherein: The processor forms the shifted view by: converting pixel values ​​of the two adjacent temporary views from a gamma space to a linear space to form linear space pixel values; performing linear summation on the linear space pixel values ​​to form a linear summation value; and The linear sum value is converted from linear space to gamma space.

15. The method according to claim 11, wherein: The shift value corresponds to a non-integer number of views of the multi-view image.

16. The method according to claim 11, wherein: The processor selects the shift values ​​such that the user's movement allows the user to view a view with a progression that matches the progression of the temporary view.

17. The method according to claim 16, wherein: The processor selects the shift values ​​so that the user's movement provides the user with a perception of parallax as the views advance.

18. The method according to claim 11, wherein: The processor selects the shift values ​​so that the shifted views follow the movement of the user and allow the user to view only a single view or a single combination of views as the user moves.

19. The method according to claim 18, wherein: The processor selects the shift values ​​to provide the user with a first three-dimensional view of the multi-view images to the user's left eye and a second three-dimensional view of the multi-view images to the user's right eye, the first three-dimensional view and the second three-dimensional view being invariant when the user moves.

20. A head tracking multi-view display comprising: a head tracker configured to provide a tracked position of a user relative to the head-tracked multi-view display, the head tracker comprising a camera configured to capture an image of the user, the head tracker further comprising an image processor configured to determine a position of the user within the captured image to provide the tracked position; a multi-beam backlight configured to provide a plurality of light beams having different principal angle directions corresponding to different view directions of a multi-view image; A processor, the processor being configured to: receiving a plurality of temporary views of the multi-view image, the plurality of temporary views corresponding to the different view directions; as well as shifting the temporary view relative to the view direction to form a plurality of shifted views corresponding to the different view directions, the shifted views being shifted by a shift value that varies as a function of the tracked position of the user and that varies across a field of view of the multi-view image; and A light valve array is configured to modulate light beams of the plurality of light beams to provide the plurality of shifted views of the multi-view image as the multi-view image to the view direction.

Images