Processor for eye-tracked three-dimensional display

Abstract

A three-dimensional (3D) display monitor is disclosed, featuring an integrated processing chip designed to enhance real-time rendering and reduce system latency. The chip receives raw 3D image data and face tracking images, performs real-time face tracking and pixel mapping, and outputs the mapped image for display. The chip may receive direct eye center position data, bypassing face tracking images. The monitor includes functionalities including touch panel feedback, hand tracking, and automatic detection of 2D or 3D content, enabling seamless mode switching. Furthermore, the chip can convert 2D content to 3D content in real-time using a neural processing unit.

Description

PROCESSOR FOR EYE-TRACKED THREE-DIMENSIONAL DISPLAYPRIORITY CLAIM

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 685,784, filed August 22, 2024, which is incorporated herein by reference in its entirety.FIELD OF THE DISCLOSURE

[0002] This document relates generally to display systems, and more specifically relates to multiview displays, three-dimensional (3D) displays, or autostereoscopic displays and processing for real-time face tracking and pixel mapping.BACKGROUND OF THE DISCLOSURE

[0003] Three-dimensional display technology provides immersive visual experiences by delivering distinct images to each eye, either with or without auxiliary eyewear. Autostereoscopic and multiview systems aim to replicate depth perception through computational alignment of image data based on the viewer’s position. Realtime face or eye tracking, combined with precise pixel mapping, supports consistent depth effects as users move in front of a display. However, maintaining an uninterrupted sense of depth and clarity for many applications may be problematic due to issues of latency.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:

[0005] FIG. 1 shows a front-view schematic drawing of an example of a 3D display system that includes a 3D display.

[0006] FIG. 2 shows a front-view drawing of an example of a display panel that includes an array of light-emitting diodes.

[0007] FIG. 3 shows a front-view drawing of an example of a display panel that includes a multibeam backlight and a light valve array.

[0008] FIG. 4 shows a front-view drawing of an example of a display panel that includes a pentile subpixel arrangement.

[0009] FIG. 5 shows a front-view drawing of an example of a periodic optical element that includes a lenticular lens array.

[0010] FIG. 6 shows a cross-sectional view of the lenticular lens array of FIG. 5.

[0011] FIG. 7 shows a front-view drawing of an example of a periodic optical element that includes a parallax barrier having transmissive slits.

[0012] FIG. 8 shows a cross-sectional view of the parallax barrier having transmissive slits of FIG. 7.

[0013] FIG. 9 illustrates an example architecture of a 3D monitor, according to some examples.

[0014] FIG. 10 shows a flowchart of providing image data, according to some examples.

[0015] FIG. 11 shows a flowchart of a method for eye tracking and calibration, according to some examples.

[0016] FIG. 12 illustrates a block diagram of an electronic device, according to some examples.DETAILED DESCRIPTION

[0017] The following description and the drawings sufficiently illustrate specific embodiments to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. Portions and features of some embodiments may be included in or substituted for, those of other embodiments. Embodiments outlined in the claims encompass all available equivalents of those claims.

[0018] Three-dimensional (3D) systems aim to replicate depth perception using real-time face or eye tracking, combined with precise pixel mapping. Applications such as gaming and visualization (e.g., for medical or design purposes), among others, may use such systems as maintaining an uninterrupted sense of depth and clarity is desirable. For example, interactive applications use displays that can deliver depth imagery with minimal latency and high frame rates to preserve motion stability and prevent visual discomfort. Virtual reality demonstrations, hands-free gesture controls, and collaborative environments all place a premium on rapid responsiveness; delays in facial(e.g., eye) or hand tracking or in image rendering can diminish user engagement and erode the perceived realism of 3D content. However, host devices often juggle multiple processing tasks, including scene rendering and user-interface management, increasing efficient allocation of compute resources to maintain battery life and system responsiveness, particularly in mobile and embedded platforms.

[0019] In particular, 3D display architectures rely on a host device’s central processing unit (CPU) or graphics processing unit (GPU) to carry out face detection and pixel remapping tasks. Offloading these operations to the host can introduce significant computational overhead, reducing available processing capacity for primary rendering workloads and other system functions. Transmitting high-resolution camera feeds and 3D content between host and display over standard interfaces can create throughput bottlenecks and exacerbate energy consumption issues. External processing accessories may add bulk, while wireless bandwidth limitations and cross-platform compatibility issues may limit their practical adoption and fail to fully address host-load concerns.

[0020] Latency remains a persistent barrier to seamless three-dimensional viewing, as buffering and data-transfer delays contribute to motion lag that can exceed acceptable limits in interactive settings. The shuttling of both image and tracking data (position information) across connections may further increase response times and degrade visual fidelity. Moreover, the use of external modules may lack a fully integrated approach, complicating installation and limiting versatility across diverse display environments. A more cohesive system that performs viewer tracking and image remapping directly within the display hardware is desirable to minimize data-transfer latency, alleviate host-device burdens, and deliver a more responsive depth-capable viewing experience.

[0021] Turning to a description of the system, FIG. 1 shows a front-view schematic drawing of an example of a 3D display system 100 that includes an exploded view of a 3D display 102. The sign conventions shown in FIG. 1 and used below assume that the 3D display 102 extends in an (x, y) plane, and that a z-axis extends away from the 3D display 102 and generally toward a viewer, along a direction that is orthogonal to a plane of the 3D display 102. Other sign conventions may also be used.

[0022] In a 3D display, a display panel having an array of subpixels may display an image according to stereo mapping coordinates associated with a viewer. A periodic optical element may direct light from the display panel to the viewer. The periodicoptical element may be invariant along an optical axis having a slant angle relative to the display panel. A viewer tracker may determine a location of the viewer. The stereo mapping coordinate of a selected subpixel of the array of subpixels may be a function of the location of the viewer, a location of the selected subpixel, a phase function of the periodic optical element, a separation between the periodic optical element and the display panel, and a refractive index of a material disposed between the periodic optical element and the display panel.

[0023] A controller 118 may use the stereo mapping coordinate from a particular subpixel to determine whether light from the subpixel is directed to a left eye or a right eye of the viewer. The controller may use the stereo mapping coordinate of the subpixel to select which image to represent with the subpixel, such as a subpixel of a “left image” to be directed to the left eye of the viewer, a subpixel of a “right image” to be directed to the right eye of the viewer, or a weighted combination of the subpixel of “left image” and the subpixel of the “right image.”

[0024] As illustrated in FIG. 1, the 3D display 102 may include a display panel 106 that may have an array of subpixels 108 configured to display an image according to stereo mapping coordinates associated with a viewer 104. The subpixels 108 may be located at subpixel locations in a grid having grid axes. Each subpixel 108 may generate light having a specified color. For example, the subpixels 108 may include red subpixels, green subpixels, and blue subpixels, which generate red light, green light, and blue light, respectively. Other color / wavelength schemes may also be used. The subpixels 108 may be grouped into pixels, with each pixel including at least two subpixels 108 that produce light of different colors. Two possible configurations for the display panel 106 are described below and shown in FIGS. 2 and 3; other configurations may also be used.

[0025] FIG. 2 shows a front-view drawing of an example of a display panel 106A that includes an array 202 of light-emitting diodes 208, such as an array of organic lightemitting diodes. Each light-emitting diode 208 may correspond to a subpixel 108. The array 202 of light-emitting diodes 208 may include red light-emitting diodes 208R, green light-emitting diodes 208G, and blue light-emitting diodes 208B, which correspond to the red subpixels, green subpixels, and blue subpixels, respectively. A controller 118 (described below) may control the light-emitting diodes 208 individually or in one or more groups. Each light-emitting diodes 208 may controllably generate light in responseto an electrical signal provided by the controller 118 or by suitable light-emitting diode driving circuitry in communication with the controller 118. The controller 118 may cause a specified light-emitting diode 208 to be directly powered with a power that varies as a function of an intensity in a corresponding location in the image. The power delivered to a light-emitting diode 208 may optionally be pulse-width modulated at a modulation frequency that is greater than may be perceived by a human eye. Using pulse-width modulation may simplify a design of a light-emitting diode array controller, because it may generate an arbitrary average power level from a relatively small number of instantaneous power levels by varying a duty cycle of the power. In some examples, the array 202 of light-emitting diodes 208 may be arranged in a rectangular or square repeating pattern over a surface area 210 of the array 202. For example, the array 202 may have grid axes 204 that are orthogonal to each other. In some examples, the grid axes 204 may be parallel to edges 206 of the array 202 of light-emitting diodes 208.

[0026] FIG. 3 shows a front-view drawing of an example of a display panel 106B that includes a backlight 302 and a light valve array 304. Although FIG. 3 shows the backlight 302 and the light valve array 304 as being separated, in practice, the backlight 302 and the light valve array 304 may be in contact or may be located as close together as is practical. The backlight 302 may provide illumination having a uniform or substantially uniform intensity over a surface area of the backlight 302. The backlight 302 may provide illumination having a relatively broad spectrum, such as including most or all of the visible portion of the electromagnetic spectrum. The backlight 302 may provide the illumination into a continuum of propagation angles toward the light valve array 304. The backlight 302 may provide unmodulated illumination to the light valve array 304. The light valve array 304 may include light valves 308 that are individually controllable or controllable in one or more groups by a controller 118 (described below). Each light valve 308 may controllably attenuate the illumination from the backlight, such as in response to an electrical signal provided by the controller 118 or by suitable light valve driving circuitry in communication with the controller 118. The light valves 308 may have color filters that allow only a portion of the electromagnetic spectrum to pass through the light valve 308. For example, the light valves 308 may include red light valves 308R that have a red filter that allows only red light to pass through the red light valves 308R, green light valves 308G that have a green filter that allows only green light to pass through the green light valves 308G, and blue light valves 308B that have a bluefilter that allows only blue light to pass through the blue light valves 308B. Other color schemes and numbers of colors may also be used. Suitable light valve arrays 304 may include liquid crystal light valves, electrophoretic light valves, and light valves based on electrowetting, and others. In some examples, the light valves 308 of the light valve array 304 may be arranged in a rectangular or square repeating pattern over a surface area 312 of the light valve array 304. For example, the light valve array 304 may have grid axes 204 that are orthogonal to each other. In some examples, the grid axes 204 may be parallel to edges 306 of the light valve array 304.

[0027] FIG. 4 shows a front-view drawing of an example of a display panel 106C that includes a pentile subpixel arrangement. The subpixels 408 may include lightemitting diodes 208 of an array 202 of light-emitting diodes 208, as in FIG. 2, or light valves 308 of a light valve array 304, as in FIG. 3. Compared to a traditional red-green- blue subpixel arrangement, in which each pixel includes a red subpixel 408R (e.g., a light emitting diode that produces red light), a green subpixel 408G (e.g., a light emitting diode that produces green light), and a blue subpixel 408B (e.g., a light emitting diode that produces blue light), the pentile subpixel arrangement may include just two subpixels 408 (or light-emitting diodes) per pixel 402. The colors of the subpixels 408 in the display panel 106C may be arranged such that the missing color of a particular pixel 402 may be found in an adjacent pixel 404. Although some display panels may employ subpixel rendering in software, which may help smooth features in the image, the display panel 106C described herein may turn off subpixel rendering when the image is displayed. For a display panel 106C that turns off subpixel rendering when the image is displayed, a location of each subpixel 408 (e.g., each light-emitting diode) may be used for calculating the corresponding stereo mapping coordinate, rather than a center of a pixel 402 (e.g., the center of a specified group of subpixels 408 or a specified group of light-emitting diodes 208).

[0028] Referring again to FIG. 1, the 3D display 102 may include a periodic optical element 110 that may direct light 112 corresponding to the image from the display panel 106 to the viewer 104. For example, the periodic optical element 110 may include a parallax optic or a parallax-generating optic. Two possible configurations for the periodic optical element 110 are described below and shown in Figures 5 and 6 and in Figures 7 and 8. Other configurations may also be used. Each of the configurations ofFigures 5 and 6 and Figures 7 and 8 may be used in combination with any of the configurations of Figures 2 and 3.

[0029] FIG. 5 shows a front-view drawing of an example of a periodic optical element 110A (e.g., the parallax optic or parallax-generating optic) that includes a lenticular lens array 502. FIG. 6 shows a cross-sectional view of the lenticular lens array 502 of FIG. 5. The lenticular lens array 502 may include an array of thin cylindrical lenslets 604 positioned to receive light from the display panel 106 and at least partially focus the received light to direct the light to specified regions proximate the viewer’s eyes.

[0030] FIG. 7 shows a front-view drawing of an example of a periodic optical element HOB (e.g., the parallax optic or parallax-generating optic) that includes a parallax barrier 702 having transmissive slits 804. FIG. 8 shows a cross-sectional view of the parallax barrier 702 having transmissive slits 804 of FIG. 7. The parallax barrier 702 may include an array of opaque strips 806 and thin transmissive slits 804 arranged to occlude portions of a displayed image in left and right viewing regions. The transmissive slits 804 may be spatially arranged to ensure that the left / right image portions are only visible in the corresponding left / right viewing regions for which they are intended. The parallax barrier 702 may be provided by a static physical layer in which the slits are precisely positioned, or electronically generated on an adaptive intermediate liquid crystal display layer.

[0031] The periodic optical element 110, including one of the lenticular lens array 502 or the parallax barrier 702 having transmissive slits 804, may be operable with the display panel 106, including one of the array 202 of light-emitting diodes 208 or the backlight 302 and light valve array 304.

[0032] As illustrated in Figures 5 and 6, the periodic optical element 110 may be invariant along an optical axis (OA) having a slant angle, a, relative to the grid axes 204. For example, the periodic optical element 110 may have transmissive features, such as the lenslets or the transmissive slits, that are invariant along the optical axis (OA) and are periodic along an orthogonal axis that is orthogonal to the optical axis (OA). As a specific example, the periodic optical element 110 may have transmissive slits that are parallel to the optical axis (OA) and are equally spaced along the orthogonal axis. As another specific example, the periodic optical element 110 may have cylindrical lenslets that are invariant in shape along the optical axis (OA), have curvature along theorthogonal axis, and are equally spaced (e.g., with center-to-center spacing) along the orthogonal axis. The periodic optical element 110 may be angled by the slant angle, a, with respect to the grid axes 204, which may optionally be parallel to edges 206 of the array 202 of light-emitting diodes 208 or edges 306 of the light valve array 304. For example, the slant angle, a, may be within a specified angular tolerance of forty -five degrees, such as being between forty-four and forty-six degrees for a tolerance of + / - one degree, between forty-three and forty-seven degrees for a tolerance of + / - two degrees, between forty-two and forty-eight degrees for a tolerance of + / - three degrees, between forty-one and forty-nine degrees for a tolerance of + / - four degrees, between forty and fifty degrees for a tolerance of + / - five degrees, or another suitable angle or angular range.

[0033] As illustrated in FIG. 1, the 3D display 102 may include a material 114 disposed between the display panel 106 and the periodic optical element 110. In some examples, the material 114 may extend fully between the display panel 106 and the periodic optical element 110, such that a light ray originating at the display panel 106 passes only through the material 114 (and does not pass through any air or unfilled volume) before arriving at the periodic optical element 110. In other examples, the material 114 may occupy only a portion of the volume between the display panel 106 and the periodic optical element 110, such that a light ray originating at the display panel 106 passes through at least some of the material 114 and passes through a volume of air before arriving at the periodic optical element 110. The material 114 may have a refractive index denoted by n. The value of the refractive index n may be between about 1.3 and about 2, although other suitable values may also be used. Suitable materials may include glass, plastic, a transparent optical adhesive, and others. In some examples, the material 114 may be dispensed in a liquid form, then cured in place, such as by exposure to ultraviolet light or heat. In other examples, the material 114 may be manufactured as a solid unit and placed in its location in the 3D display 102. For example, the material 114 may function as a cover glass for the display panel 106. In some examples, the material 114 may function as a relatively precise spacing element. For example, the material 114 may be manufactured to have a specified thickness to within a specified thickness tolerance, and may set the spacing between the display panel 106 and periodic optical element 110 to have a value equal to the specified thickness when the 3D display 102 is assembled.

[0034] As illustrated in FIG. 1, the 3D display 102 may include a viewer tracker 116 that may determine a location of the viewer 104. The viewer tracker 116 may provide a tracked position of the viewer 104 (e.g., of a head of the viewer 104, or of one or both eyes of the viewer 104, or of another anatomical feature of the viewer 104). The viewer tracker 116 may be coupled to the controller 118 (described below), such as by providing viewer location data (shown in FIG. 1 as coordinates xv, yv, and zv) that represents a measured position or location of the viewer 104. The viewer tracker 116 may provide the viewer location data at regular or irregular intervals to the controller 118. The viewer tracker 116 may include a camera configured to capture an image of the viewer 104. The viewer tracker 116 may further include an image processor (or general- purpose computer programmed as an image processor) configured to determine a position of the viewer 104 within the captured image to provide the tracked position. In some examples, the controller 118 may include the image processor of the viewer tracker 116, such as by performing operations with the same processing circuitry. In other examples, the controller 118 may be separate from the image processor of the viewer tracker 116. Other suitable viewer trackers may also be used, including viewer trackers based on lidar (e.g., using time-of-flight of reflected light over a scene to of view to determine distances to one or more objects in the scene, such as a viewer’s head or a viewer’s eyes) or other technologies. The controller 118 may use an output of the viewer tracker 116, among other data, to calculate the stereo mapping coordinates, as described in detail below.

[0035] As illustrated in FIG. 1, the 3D display system 100 may include a controller 118. The controller 118 may include a processor 120 and memory 122 storing instructions executable by the processor 120. The instructions may be executable by the processor 120 to perform data processing activities. The data processing activities may include, for subpixels 108 of the array of subpixels 108 of the display panel 106, determining the stereo mapping coordinates of the subpixels 108, and causing the display panel 106 to display the image according to the stereo mapping coordinates. These data processing activities are described in detail below.

[0036] As noted above, drawbacks of the 3D display include that the system can create a relatively high system load and have a relatively high latency. For example, a 3D display can perform face tracking and pixel mapping, which can use significant CPU and GPU resources. Using the processing resources for face tracking and pixel mappingcan introduce bottlenecks in the processing chain for the display. Similarly, buffering and processing on a host device can introduce “motion-to-photon” latency, which can negatively affect the real-time responsiveness of the 3D display and can diminish the user experiences of 3D content.

[0037] To decrease the system load and / or decrease the system latency, a 3D display can offload face tracking and pixel mapping to the 3D display device, or monitor, itself. Doing so can free up resources on the host device, which in turn can increase an efficiency of 3D rendering and can decrease the latency of the 3D display device. An integrated monitor unit (or chip) can help with the tasks of offloading face tracking and pixel mapping to the monitor itself. The monitor can include the integrated monitor unit. The integrated monitor unit can perform real-time 3D rendering or one or more tasks or processes associated with such rendering. In other examples, the integrated monitor unit may be designed not only for face tracking and pixel remapping but also for interpreting various forms of user interaction, including touch input, hand gestures, and spatial feedback derived from head position. These inputs may be processed locally and transmitted back to the host application or rendering engine to enable real-time interaction with 3D content.

[0038] Examples of user input scenarios include that as the viewer moves laterally in front of the display, the integrated monitor unit continuously updates eye position data and feeds the eye position data to the host application, allowing a remote renderer to shift the virtual camera in the 3D scene to create a real-time look-around or parallax effect. In some examples, when the user taps the touch panel on a specific region of the screen, the chip relays both the 2D touch coordinates and the 3D face position to the host application. This can be used, for example, to shift depth-of-field focus in a 3D scene based on user attention. As above, in some cases the same camera used for face tracking can also detect hand gestures; e.g., the viewer may perform a pinch gesture to zoom into a 3D object, or swipe to rotate a model. The monitor unit extracts the relevant hand position data and forwards the hand position data to the application to drive scene interaction without the use of additional peripherals. In addition, the system can track whether a user’s face or hand moves closer to or farther from the display, enabling interactions like “push-to-selecf ’ or zoom based on proximity, again forwarding this data to the remote rendering engine for dynamic adjustment.

[0039] FIG. 9 illustrates an example architecture of a 3D monitor, according to some examples. The display monitor system 900 is designed to address the challenges of high system load and latency in conventional 3D display solutions by offloading processing tasks from the host device to the display hardware itself. This architecture enables efficient, low-latency 3D rendering and interactive experiences, suitable for applications such as gaming, visualization, and collaborative environments.

[0040] The display monitor system 900 includes a monitor 902, which serves as the primary visual output device for presenting three-dimensional content to a viewer. The term “monitor” refers to the complete hardware system that serves as the primary visual output device for a user. The monitor 902 may incorporate a variety of display technologies. The monitor 902 encompasses not only the display panel 906 itself, but also all integrated electronics, processing units, input / output interfaces, sensors, and auxiliary components used for operation. Specifically, a monitor 902 includes: a display panel 906 (e.g., LCD, LED, or other technology), an integrated processing unit 910 (e.g., system on a chip (SoC), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or others) responsible for real-time face / eye tracking, pixel mapping, and optionally 2D-to-3D conversion, a periodic optical element (e.g., lenticular lens array or parallax barrier) may be used to direct light from the display panel 906 to the appropriate viewing regions corresponding to the eyes of the viewer, a tracking camera for capturing images of the viewer for tracking purposes, input / output interfaces (e.g., HDMI, DisplayPort, USB, Wi-Fi, Bluetooth), other components such as a touch panel for user interaction, hand tracking device for gesture-based control, speakers, and housing, and power supply and control circuitry. The monitor 902 is thus the entire apparatus that receives content from an external source (such as a laptop or mobile device), processes the content, and presents the final visual output to the user, while also handling user interaction and feedback. In an example, the monitor 902 comprises an example of, or one or more components of, the 3D display system 100.

[0041] The term “display panel” (or merely “display”) refers specifically to the component within the monitor 902 that is responsible for generating the visual image. The display panel 906 is the screen or panel that emits or modulates light to form the images seen by the viewer. The display panel 906 does not include the processing electronics, cameras, or other peripheral components. The display panel 906 may be implemented using various technologies, such as: an array of LEDs (e.g., organic LEDs(OLEDs)), a backlight and light valve array (e.g., LCD), or other suitable panels capable of supporting autostereoscopic or multiview 3D effects. The display panel 906 is paired with a periodic optical element (such as a lenticular lens array or parallax barrier) to achieve the 3D effect, but the display panel 906 itself is limited to the image-forming hardware.

[0042] The term “display monitor system” refers to the entire system or arrangement that includes the monitor 902 as a central component, but also encompasses the external devices and data sources that interact with the monitor 902. The display monitor system 900 may include the host device (such as a PC, laptop, or mobile device) that provides raw image data, the network or communication infrastructure (wired or wireless), and any other peripherals or subsystems that participate in the operation of the 3D display experience. The display monitor system 900 thus describes the monitor 902 in combination with its operational environment and the flow of data between the monitor 902 and external sources.

[0043] The monitor 902 is further equipped with a tracking camera 904, which is mounted on or integrated with the monitor housing. The tracking camera 904 is configured to capture images or video streams of a viewer positioned in front of the display panel 906. The tracking camera 904 may be a visible light camera, an infrared camera (e.g., for low-light environments), and / or a depth-sensing camera for enhanced tracking accuracy, and may be used for eye tracking and / or facial tracking and, in some embodiments, hand tracking. The captured images are transmitted directly to a processing unit 910 for analysis, eliminating routing high-bandwidth camera data through a host device and thereby reducing data transfer bottlenecks.

[0044] The viewer represents the user or users interacting with the 3D display system 900. The system 900 is capable of tracking the position of the viewer’s head, eyes, or other anatomical features in real time. The tracking camera 904 captures images of the viewer, and the processing unit 910 executes eye detection algorithms to determine the three-dimensional coordinates (e.g., x, y, z) of the left and right eyes. This eye position data is used for accurate stereo mapping and pixel remapping, ensuring that the correct image content is delivered to each eye to produce the intended 3D effect. In some embodiments, the system 900 may also track hand positions for gesture-based input, using either the same tracking camera 904 or a dedicated hand tracking device.The monitor 902 may include a hand tracking device, or the chip may process hand tracking from the same images used for face tracking.

[0045] Integrated within the monitor 902 is a processing unit 910, which is an integrated monitor chip that is responsible for performing real-time data processing activities essential to 3D display operation. The processing unit 910 may be implemented, for example, as a System on Chip (SoC), Application-Specific Integrated Circuit (ASIC), or Field-Programmable Gate Array (FPGA), and may include dedicated neural processing units or digital signal processors (DSPs) for accelerated computation. Accordingly, a different processing unit 910 may be used for different applications; for example, an ARM-based SoC with integrated neural processing may be used for 2D-to- 3D conversion, a dedicated ASIC may be used for high-efficiency pixel mapping, and an FPGA may be used for flexible prototyping and customization.

[0046] The processing unit 910 executes instructions stored in memory to carry out tasks such as face or eye tracking 912, pixel mapping (weaving) 914, and, in some embodiments, 2D-to-3D content conversion. By performing these operations onboard, the processing unit 910 alleviates the computational burden on the host device and reduces motion-to-photon latency, thereby improving the responsiveness and realism of the 3D viewing experience.

[0047] The processing unit 910 receives raw image data and automatically determines whether the content is 2D or 3D, such as by using neural network-based content classification. The processing unit 910 can switch the display (or portion thereof) between 2D and 3D automatically, in response to the determination. The integrated monitor chip can receive 2D content. In some examples, the processing unit 910 can convert the 2D content to 3D content, such as by using a neural processing unit, such as the DSP of an ARM-based SoC. The processing unit 910 may also execute algorithms for face and eye detection, such as convolutional neural networks (CNNs) or other machine learning models, to determine the real-time 3D coordinates of the eyes of the viewer. Pixel mapping (weaving) algorithms may use this data, along with calibration data for the display panel and optical elements, to dynamically remap the input 3D image data so that the correct image content is delivered to each eye.

[0048] Assuming that processing unit 910 receives raw 3D image data, this data is received from an external source, such as a host computer, mobile device, or media player. The raw 3D image data may be provided in a side-by-side (SBS) format, top-and-bottom format, or other suitable 3D encoding. The raw 3D image data may be transmitted to the monitor 902 via a wired connection (e.g., HDMI, DisplayPort, USB) or a wireless connection (e.g., Wi-Fi, Wi-Fi Direct, Bluetooth). The use of Wi-Fi Direct, for example, enables direct peer-to-peer communication between the host device and the monitor 902 without the use of an external router or network infrastructure, facilitating flexible deployment in a variety of environments in which network or other connectivity may not be available. The processing unit 910 can receive the raw 3D image data directly without performing significant computations or calculations to generate the data.

[0049] In addition to receiving raw 3D image data, the processing unit 910 receives face tracking images from the tracking camera 904. The processing unit 910 processes the face tracking images to extract the real-time position of the viewer’s eyes (e.g., left x, y, and / or z coordinates and, right x, y, and / or z coordinates), which is then used to perform pixel mapping 914. The processing unit 910 can, for example, dynamically assess a viewer eye center position. Pixel mapping 914 involves dynamically remapping the input 3D image data based on the detected eye positions, the calibration data of the display panel 906 and optical elements, and the system geometry. This ensures that the left and right images are correctly aligned with the viewer’s eyes, maintaining the 3D effect as the viewer moves.

[0050] The processing unit 910 outputs the current position of each left and right eye as eye position data (e.g., left x, y, and / or z coordinates and, right x, y, and / or z coordinates). This data may be fed back to an external source (e.g., the host device or application) via the same or a different wired or wireless connection as the connection that supplies the raw 3D image data. The feedback of eye position data enables advanced interactive features, such as dynamic adjustment of a virtual camera in a 3D application to provide a look-around effect or enhanced parallax. The feedback mechanism may be selectively enabled depending on the application; for example, the feedback mechanism may be used in gaming or interactive visualization, but not for passive 3D movie playback.

[0051] Using both the raw 3D image data and the detected eye position data, the processing unit 910 performs eye tracking 912 and pixel mapping 914 in real time, generating a mapped output that is directly presented on the display panel 906 of the monitor 902. The mapped output represents the final image data that is displayed to the viewer, with each subpixel or pixel mapped according to the stereo mapping coordinatescalculated from the viewer’s position, the display geometry, and the optical characteristics of the system. This real-time remapping ensures that the 3D effect is maintained with minimal latency, even as the viewer moves or interacts with the display.

[0052] The interactions among the elements of the system 900 enable a highly responsive, low-latency 3D display system that offloads a significant amount of processing from the host device, supports a range of input and feedback modalities, and is adaptable to various hardware implementations and use cases.

[0053] The monitor hardware architecture integrates multiple hardware modules, each playing a specific role in delivering a seamless, real-time, eye-tracked 3D display experience. The system includes the display panel, which is responsible for rendering both 2D and 3D visual content in conjunction with a periodic optical element, such as a lenticular lens or parallax barrier, to achieve autostereoscopic (glasses-free 3D) effects. The display panel receives driving signals from a display driver, which converts digital image data from the processing unit into the electrical signals used by the LCD or LED array. The 3D circuit is responsible for managing the switching and control of the 3D effect, including toggling between 2D and 3D modes and synchronizing the periodic optical element with the display panel to ensure the correct delivery of left and right images to the viewer’s eyes.

[0054] Illumination for the display is provided by the backlight circuit and in some cases an inverter circuit. The backlight circuit ensures uniform lighting across the display, while the inverter circuit converts DC power to the high-voltage AC used by backlight types of display panels. The backlight circuit and inverter circuit may be tightly integrated with the display driver and 3D circuit to maintain image quality and synchronization during 3D operation.

[0055] The processing unit may be a SoC, for example a Qualcomm™ SM8550 module. The SoC is responsible for all real-time computational tasks, including face and eye tracking, pixel mapping (weaving), 2D-to-3D content conversion, and overall system control. The processing unit leverages integrated CPU, GPU, DSP, and neural processing engines to analyze images from the tracking camera, calculate the viewer’s eye positions, and dynamically remap the 3D image data for accurate stereo presentation. Power management for the processing unit and other components may be handled by dedicated modules, ensuring stable and efficient operation.

[0056] An input / output (I / O) board acts as the central hub for all input and output connections, routing video, data, and control signals between the display, processing unit, and external devices. The I / O board may support a range of video input interfaces, including HDMI, USB, and DisplayPort (DP), allowing the monitor to receive high- bandwidth video and data streams from PCs, laptops, or mobile devices. Video interface chips such as the LT6911XC and LT6911GX may be used to convert and manage the I / O signals signals, ensuring compatibility and optimal performance.

[0057] For wireless connectivity, the monitor incorporates modules such as the Qualcomm™ WCN 7850 and WCN 7851, which provide Wi-Fi and Bluetooth capabilities. These modules enable features like Wi-Fi Direct, allowing the monitor to receive video and data streams wirelessly from external devices without the need for a traditional network infrastructure. This may be particularly useful for flexible deployment scenarios and for connecting to devices that may not have high-end processing capabilities.

[0058] The camera module that contains the tracking camera 904 may be mounted on or integrated with the monitor housing and used for real-time face and hand tracking. The camera captures images of the viewer, which are processed by the SoC to determine the precise position of the eyes (and optionally hands) of the viewer in three- dimensional space. This information permits accurate pixel mapping and enables interactive features such as gesture control. In some configurations, a hand tracking device may be included as a separate module, or hand tracking may be performed using the same camera as face tracking, with the processing unit extracting both head and hand positions from the captured images.

[0059] User interaction may be further enhanced by the inclusion of a touch panel, which allows for direct manipulation of on-screen content through touch gestures. The touch panel interfaces with the processing unit, which can provide feedback to the host device or application regarding user input. Audio output may be provided by integrated speakers, with amplification handled by modules such as the Qualcomm™ WSA 8840, enabling multimedia and interactive applications that use sound.

[0060] The firmware, calibration data, and user settings may be stored in highspeed storage modules such as Samsung™ UFS3.x or UFS4.0, while high-speed memory modules (e.g., Micron™ LPDDR5.x) support data processing and buffering of real-time 3D rendering and tracking. A keyboard status LED may be used to providevisual feedback regarding the status of connected input devices, enhancing usability in professional or collaborative environments.

[0061] All of the components may be powered by a robust power supply that converts AC mains power (100 to 240V) to the DC voltages used by the internal electronics of the monitor. The entire assembly may be housed within a custom- designed housing and holder to provide structural support, thermal management, and aesthetic appeal.

[0062] The interactions between the components may be highly coordinated. For example, the processing unit receives video data from the I / O board, processes tracking images from the camera, and outputs mapped image data to the display driver, all while managing power consumption and responding to user input from the touch panel or hand tracking device. Wireless modules enable seamless connectivity with external devices, and the modular design of the system allows for flexibility in hardware configurations and future upgrades. The architecture enables the 3D monitor to function as a self- contained, high-performance system capable of delivering immersive, real-time 3D experiences with low latency and robust user interaction.

[0063] In some embodiments, the integrated processing unit 910 may use dedicated processing devices such as ASICs for different tasks. For example, one ASIC may be used for 2D: 3D image processing or conversion, another ASIC may be used for face tracking, and a third ASIC may be used for pixel mapping that dynamically remaps 3D image data based on the viewer's eye position to ensure correct delivery of left and right images to each respective eye (weaving).

[0064] Thus, the monitor 902 is the complete hardware system, including the display panel 906, integrated processing unit 910, periodic optical element, tracking camera 904, I / O interfaces, power supply, and housing. The display panel 906 is limited to the image-forming component (e.g., LCD, LED, or OLED) that emits or modulates light to form images, exclusive of processing electronics and peripheral components.

[0065] FIG. 10 shows a flowchart of a method for providing image data, according to some examples. The method 1000 may include additional steps not shown or may omit steps. The steps in the method 1000 may occur in an order other than that shown.

[0066] At step 1002, the method 1000 begins by receiving raw image data from an external source. This image data may be in a 3D format, such as side-by-side (SBS)or top-and-bottom, or in a 2D format, and is transmitted to the display monitor system via a wired or wireless connection, including options such as HDMI, DisplayPort, USB, Wi-Fi, or Wi-Fi Direct.

[0067] At step 1004, a processing unit that is integrated with the monitor system determines whether the received raw image data corresponds to 2D or 3D content. This determination may be performed automatically in real time, for example, using neural network-based content classification algorithms executed by a neural processing unit or digital signal processor within the processing unit.

[0068] At step 1006, the processing unit switches the display panel between 2D and 3D display modes based on the determination from step 1004. If the content is 2D, the display panel operates in a standard 2D mode; if the content is 3D, the processing unit may set the display panel to a 3D mode, which may involve activating a periodic optical element such as a lenticular lens array or parallax barrier.

[0069] At step 1008, viewer position information may be acquired. This may include capturing images of the viewer using a camera integrated with the display monitor system. The camera may be a visible light, infrared, or depth-sensing camera, and the images may be provided directly to the processing unit for analysis.

[0070] At step 1010, the processing unit performs real-time viewer tracking by analyzing the position information, or acquired images, to determine the 3D position of the viewer’s eyes (e.g., left x, y, and / or z coordinates and, right x, y, and / or z coordinates). In some embodiments, the processing unit may alternatively receive direct eye center position data from an external device or sensor. Eye center position data can indicate a location that is between the two eyes of the viewer, and may be easier (i.e., less computationally expensive) to identify and / or track than continuously identifying the position of each of two eyes.

[0071] At step 1012, the processing unit performs pixel mapping (i.e., weaving) based on the viewer position information and the raw 3D image data. This involves dynamically remapping the input image data according to the detected eye positions, the calibration data of the display panel and optical elements, and the system geometry, ensuring that the correct image content is delivered to each eye for an accurate 3D effect.

[0072] At step 1014, the mapped image data is provided to the display panel for presentation to the viewer. The display panel, in conjunction with the periodic opticalelement, directs the mapped light to the appropriate viewing regions corresponding to the viewer’s eyes, thereby producing a real-time, autostereoscopic 3D visual experience.

[0073] At step 1016, the processing unit may provide viewer position data, such as the real-time eye coordinates, to the external source or host device. This feedback enables advanced interactive features, such as dynamic adjustment of a virtual camera in a 3D application to provide a look-around effect or enhanced parallax and may be selectively enabled depending on the application. The feedback may also be user- controlled based on a user input, such as visual, audible, or tactile (touch) input from the viewer.

[0074] In some examples, an external 3D device, such as a PC, laptop, mobile device, or game console, can be used to augment the processing of the 3D monitor system to achieve the advanced interactive features. In this architecture, the 3D monitor’s integrated processing unit may continuously track the viewer’s eye or head position in real time using onboard camera and processing algorithms. The monitor may then generate and output viewer position data (such as the three-dimensional coordinates of the viewer’s eyes or the calculated eye center) to the external 3D device via a wired or wireless connection (for example, HDMI, USB, or Wi-Fi Direct).

[0075] The external 3D device may receive the real-time feedback and use the feedback to dynamically adjust the parameters of the virtual camera within the 3D application or content being rendered. For instance, as the viewer moves their head to the left or right, the external device can shift the virtual camera’s viewpoint accordingly, updating the rendered scene to reflect the new perspective. This creates a look-around effect, where the displayed 3D content appears to change in response to the viewer’s position, enhancing the sense of depth and immersion.

[0076] Similarly, the external device can use the viewer position data to modify the amount of parallax in the rendered images, making the 3D effect more pronounced or subtle depending on the viewer’s distance and angle relative to the display. This dynamic adjustment may be particularly valuable in interactive applications such as gaming, design visualization, or collaborative environments, where real-time responsiveness to user movement is desirable.

[0077] At step 1018, the method may further include user interaction steps, such as receiving input from a touch panel or hand position data from a hand tracking device. The processing unit can process these inputs and provide corresponding feedback to theexternal device or application, supporting gesture-based control and interactive 3D experiences.

[0078] FIG. 11 shows a flowchart of a method for eye tracking and calibration, according to some examples. The method 1100 may include additional steps not shown or may omit steps. The steps in the method 1100 may occur in an order other than that shown.

[0079] At step 1102, the system acquires detailed positional data for both the left and right eyes of the viewer. The integrated processing unit may use images captured by a tracking camera mounted on or within the monitor to obtain the positional data. The camera may be a visible light camera, an infrared camera, or a depth-sensing camera. The processing unit executes face or eye detection algorithms such as CNNs or other machine learning models to accurately determine the three-dimensional coordinates (x,y, z) of each eye. This step may be performed during initial setup, at system startup, or upon user request to ensure accurate calibration.

[0080] At step 1104, the system uses the two-eye positional data to establish a reference point for subsequent tracking. This reference may be the geometric midpoint between the two eyes (the “eye center”), or another easily identifiable facial feature. The processing unit calculates the reference position, which serves as a baseline for real-time tracking. This ensures that the system is calibrated to the viewer’s unique facial geometry and can compensate for variations in head position or orientation.

[0081] At step 1106, the system may transition to tracking a single, easily identifiable feature (e.g., the eye center, nose tip, or another facial landmark) rather than continuously tracking both eyes. This reduces computational load and latency, enabling faster and more efficient real-time adjustments. The processing unit analyzes incoming images from the tracking camera to detect and follow the chosen singular marker. In some implementations, the system may use a lightweight tracking algorithm or a simplified neural network model for this purpose.

[0082] At step 1108, using the real-time position of the singular marker, the processing unit performs pixel mapping (weaving) to dynamically remap the 3D image data. The mapped image data is generated based on the current position of the tracked feature, the calibration data from the display panel and optical elements, and the system geometry. The processing unit then outputs the mapped image data to the display panelfor presentation to the viewer, ensuring that the 3D effect is maintained as the viewer moves.

[0083] At step 1110, to maintain accuracy over time and compensate for changes in the viewer’s position or orientation, the system periodically or intermittently reacquires two-eye position data. This may occur at fixed intervals, when significant movement is detected, or upon user command. The processing unit uses the updated two-eye data to recalibrate the reference position, ensuring that the system remains aligned with the viewer’s actual eye positions. This hybrid approach balances the desire for precise calibration with the efficiency of real-time tracking using a singular feature. The tracking camera captures images of the viewer for both two-eye and singular marker tracking. In each step, the integrated processing unit (e.g., SoC, ASIC, or FPGA) executes the detection, calibration, tracking, and pixel mapping algorithms. The display panel presents the mapped image data generated by the processing unit. Optional user interface elements (e.g., touch panel or software controls) may allow the user to initiate recalibration or adjust tracking settings.

[0084] FIG. 12 illustrates a block diagram of an electronic device, according to some examples. The electronic device 1200 may be any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine, such as providing functionality related to the monitor.

[0085] Examples, as described herein, may include, or may operate on, logic or a number of components, modules, or mechanisms. Modules and components are tangible entities (e.g., hardware) capable of performing specified operations and may be configured or arranged in a certain manner. In an example, circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as a module. In an example, the whole or part of one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware processors may be configured by firmware or software (e.g., instructions, an application portion, or an application) as a module that operates to perform specified operations. In an example, the software may reside on a machine readable medium. In an example, the software, when executed by the underlying hardware of the module, causes the hardware to perform the specified operations.

[0086] Accordingly, the term “module” (and “component”) is understood to encompass a tangible entity, be that an entity that is physically constructed, specificallyconfigured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operation described herein. Considering examples in which modules are temporarily configured, each of the modules need not be instantiated at any one moment in time. For example, where the modules comprise a general-purpose hardware processor configured using software, the general-purpose hardware processor may be configured as respective different modules at different times. Software may accordingly configure a hardware processor, for example, to constitute a particular module at one instance of time and to constitute a different module at a different instance of time.

[0087] The electronic device 1200 may include a hardware processor (or equivalently processing circuitry) 1202 (e.g., a central processing unit (CPU), a GPU, a hardware processor core, or any combination thereof), a main memory 1204 and a static memory 1206, some or all of which may communicate with each other via an interlink (e.g., bus) 1208. The main memory 1204 may contain any or all of removable storage and non-removable storage, volatile memory or non-volatile memory. The electronic device 1200 may further include a display unit 1210 such as a video display, an alphanumeric input device 1212 (e.g., a keyboard), and a user interface (UI) navigation device 1214 (e.g., a mouse). In an example, the display unit 1210, input device 1212 and UI navigation device 1214 may be a touch screen display. The electronic device 1200 may additionally include a storage device (e.g., drive unit) 1216, a signal generation device 1218 (e.g., a speaker), a network interface device 1220, and one or more sensors, such as a global positioning system (GPS) sensor, compass, accelerometer, or another sensor. The electronic device 1200 may further include an output controller, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

[0088] The storage device 1216 may include a non-transitory machine readable medium 1222 (hereinafter simply referred to as machine readable medium) on which is stored one or more sets of data structures or instructions 1224 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The non- transitory machine readable medium 1222 is a tangible medium. The instructions 1224 may also reside, completely or at least partially, within the main memory 1204, within static memory 1206, and / or within the hardware processor 1202 during execution thereofby the electronic device 1200. While the machine readable medium 1222 is illustrated as a single medium, the term "machine readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store the one or more instructions 1224.

[0089] The term “machine readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by the electronic device 1200 and that cause the electronic device 1200 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Non-limiting machine- readable medium examples may include solid-state memories, and optical and magnetic media. Specific examples of machine-readable media may include non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; Random Access Memory (RAM); and CD- ROM and DVD-ROM disks.

[0090] The instructions 1224 may further be transmitted or received over a communications network using a transmission medium 1226 via the network interface device 1220 utilizing any one of a number of wireless local area network (WLAN) transfer protocols (e.g., internet protocol (IP)). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), cellular networks, and wireless data networks. Communications over the networks may include one or more different protocols, such as Institute of Electrical and Electronics Engineers (IEEE) 802.12, Long Term Evolution (LTE), peer-to-peer (P2P) networks, next generation (NG) / 5thgeneration (5G). In an example, the network interface device 1220 may include one or more physical jacks (e.g., Ethernet, coaxial, or phonejacks) or one or more antennas to connect to the transmission medium 1226.

[0091] Note that the term “circuitry” as used herein refers to, is part of, or includes hardware components such as an electronic circuit, a logic circuit, a processor (shared, dedicated, or group) and / or memory (shared, dedicated, or group), an Application Specific Integrated Circuit (ASIC), a field-programmable device (FPD) (e.g., a field-programmable gate array (FPGA), a programmable logic device (PLD), a complex PLD (CPLD), a high-capacity PLD (HCPLD), a structured ASIC, or aprogrammable SoC), digital signal processors (DSPs), etc., that are configured to provide the described functionality. In some embodiments, the circuitry may execute one or more software or firmware programs to provide at least some of the described functionality. The term “circuitry” may also refer to a combination of one or more hardware elements (or a combination of circuits used in an electrical or electronic system) with the program code used to carry out the functionality of that program code. In these embodiments, the combination of hardware elements and program code may be referred to as a particular type of circuitry.

[0092] The term “processor circuitry” or “processor” as used herein thus refers to, is part of, or includes circuitry capable of sequentially and automatically carrying out a sequence of arithmetic or logical operations, or recording, storing, and / or transferring digital data. The term “processor circuitry” or “processor” may refer to one or more application processors, one or more baseband processors, a physical central processing unit (CPU), a single- or multi-core processor, and / or any other device capable of executing or otherwise operating computer-executable instructions, such as program code, software modules, and / or functional processes.

[0093] Thus, the 3D display system reallocates specific processing tasks, including real-time viewer tracking and pixel mapping, from a host device to an integrated processing unit within the monitor itself. While 3D display systems may rely on the CPU or GPU of the host device to perform computationally intensive operations such as face or eye tracking and the subsequent pixel mapping (or weaving) to deliver accurate 3D imagery such an approach burdens the host system, leading to increased latency, reduced rendering efficiency, and diminished user experience, especially in interactive or resource-constrained environments. The system overcomes these limitations by embedding a dedicated processing unit directly within the hardware of the monitor. This processor is responsible for receiving raw 3D image data (for example, in side-by-side format) from an external source and, independently of the host, acquiring and processing viewer position information from an integrated camera.

[0094] The processing unit executes advanced algorithms for face or eye detection, such as neural network-based image analysis, to determine the real-time three- dimensional coordinates of a viewer or eyes of the viewer. This data is then used to perform pixel mapping, dynamically remapping the incoming 3D image data so that the correct image content is delivered to each eye, maintaining the 3D effect as the viewermoves. The pixel mapping operation can use calibration data for the display panel and optical elements, as well as system geometry, to ensure precise alignment and depth perception. The processing unit can also support additional algorithms, such as neural network-based content classification to automatically detect whether the incoming content is 2D or 3D, and, if equipped with a neural processing unit or DSP, can perform real-time 2D-to-3D content conversion.

[0095] The monitor has a flexible communication infrastructure and can receive raw image data and transmit feedback (such as real-time eye position data) via both wired (HDMI, DisplayPort, USB) and wireless (Wi-Fi, Wi-Fi Direct, Bluetooth) connections. The use of Wi-Fi Direct enables direct peer-to-peer communication between the host device and the monitor without the use of a router or network infrastructure, allowing for deployment in a wide range of environments. The system can also provide feedback of viewer position data to the host device, enabling advanced interactive features such as dynamic virtual camera adjustment for enhanced parallax or look-around effects in 3D applications.

[0096] The modularity of the integrated processing unit provides extensibility. For example, the same camera used for face tracking can also be leveraged for hand tracking, with the processor running gesture recognition algorithms for the latter operations to extract hand position data for gesture-based control. The system can also include a touch panel or other tactile input, with the processor aggregating and forwarding touch and position information to the host device or application.

[0097] The system is distinct from conventional architectures that merely divide processing tasks among different processors or offload tasks from one processor to another in a generic fashion. In traditional multi-processor or distributed processing systems, tasks may be allocated to specialized hardware (such as a GPU, DSP, or external accessory) to optimize performance, but the overall system architecture and data flow remain largely host-centric. Typically, the host device (e.g., PC, laptop, or mobile device) remains responsible for orchestrating the entire 3D rendering pipeline, including the acquisition and processing of position information, the execution of pixel mapping algorithms, and the final composition of image data for display. Even when certain tasks are offloaded to external modules or co-processors, the host device manages the transfer of high-bandwidth data (such as camera feeds or intermediate image data), coordinatessynchronization, and handles the integration of results back into the main rendering workflow.

[0098] In contrast, the system described herein relocates the above-mentioned real-time 3D-specific processing directly into the monitor hardware itself, via an integrated processing unit. This processing unit is not a generic co-processor, but a dedicated, self-sufficient module embedded within the monitor, designed to autonomously handle the entire 3D tracking and remapping pipeline. The role of the host device is reduced to simply providing raw 3D image data (e.g., left / right frames in side-by-side format), in some cases without processing or even accessing the tracking camera data, perform pixel mapping, or manage 3D-specific synchronization. As integrated processor receives both the raw 3D image data and the tracking camera input locally, executes detection and mapping algorithms internally, and provides the final mapped image data directly to the display panel, this may mitigate the use of high- bandwidth, low-latency data transfer between the host and monitor for tracking or mapping purposes, and remove the host from the path of 3D rendering.Furthermore, the system architecture enables unique operational modes and feedback mechanisms. For example, the monitor can autonomously determine whether the incoming content is 2D or 3D and switch display modes accordingly and perform realtime 2D-to-3D conversion using onboard neural processing. The system can also provide real-time feedback of viewer position data (such as eye coordinates) to the host device, enabling interactive features such as dynamic camera adjustment in 3D applications without the host processing or interpreting raw position information.

[0099] Examples

[0100] Example l is a three-dimensional (3D) monitor in a 3D system comprising: a display panel configured to present visual content to a viewer; and an integrated processing unit coupled to the display panel, the integrated processing unit configured to: receive raw image data from an external source; determine whether the raw image data corresponds to two-dimensional (2D) content or whether the raw image data corresponds to 3D content; switch the display panel between 2D and 3D display modes dependent on a determination of whether the raw image data corresponds to 2D content or whether the raw image data corresponds to 3D content; and for the determination that the raw image data corresponds to 3D content: receive viewer positioninformation; perform, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data, the pixel mapping providing mapped image data dependent on detected eye positions and geometry of the 3D system; and provide the mapped image data to the display panel for presentation to the viewer.

[0101] In Example 2, the subject matter of Example 1 includes, wherein the integrated processing unit includes at least one of a system on chip (SoC), applicationspecific integrated circuit (ASIC), or field-programmable gate array (FPGA).

[0102] In Example 3, the subject matter of Examples 1-2 includes, wherein the viewer position information comprises images captured by a camera integrated with the monitor.

[0103] In Example 4, the subject matter of Examples 1-3 includes, wherein the integrated processing unit is further configured to receive eye center position data of eyes of the viewer as the viewer position information.

[0104] In Example 5, the subject matter of Examples 1-4 includes, a touch panel coupled to the integrated processing unit, the integrated processing unit configured to provide touch input data and viewer position data of the viewer to an external device.

[0105] In Example 6, the subject matter of Examples 1-5 includes, wherein the integrated processing unit is configured to: extract hand position data from images used for viewer tracking, and provide the hand position data to an external device.

[0106] In Example 7, the subject matter of Examples 1-6 includes, wherein the integrated processing unit is configured to convert 2D content to 3D content in real time using a neural processing unit.

[0107] In Example 8, the subject matter of Examples 1-7 includes, wherein the raw image data is received via at least one of a wired connection or a wireless connection, the wireless connection comprising Wi-Fi Direct.

[0108] In Example 9, the subject matter of Examples 1-8 includes, wherein the integrated processing unit is further configured to output viewer position data to the external source to enable dynamic adjustment of 3D content.

[0109] In Example 10, the subject matter of Examples 1-9 includes, wherein the display panel uses a periodic optical element that includes at least one of a lenticular lens array or a parallax barrier.

[0110] In Example 11, the subject matter of Examples 1-10 includes, wherein the integrated processing unit is configured to perform pixel mapping based on calibration data of the display panel and optical elements, and on the geometry of the 3D system.

[0111] In Example 12, the subject matter of Examples 1-11 includes, wherein the integrated processing unit comprises respective instances of a neural processing unit or digital signal processor (DSP) for accelerated computation of each of viewer tracking, pixel mapping, and 2D-to-3D conversion.

[0112] Example 13 is a non-transitory computer-readable storage medium that stores instructions for execution by one or more processors of a monitor having a display panel in a three-dimensional (3D) system, the instructions, when executed, cause the one or more processors to: receive raw image data from an external source; determine whether the raw image data corresponds to two-dimensional (2D) content or whether the raw image data corresponds to 3D content; switch the display panel between 2D and 3D display modes dependent on a determination of whether the raw image data corresponds to 2D content or whether the raw image data corresponds to 3D content; and for the determination that the raw image data corresponds to 3D content: receive viewer position information; perform, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data, the pixel mapping providing mapped image data dependent on detected eye positions and geometry of the 3D system; and provide the mapped image data to the display panel for presentation to a viewer with a latency less than that of a system in which viewer tracking and pixel mapping are performed by a host device external to the 3D system.

[0113] In Example 14, the subject matter of Example 13 includes, wherein the viewer position information comprises images captured by a camera integrated with the monitor.

[0114] Example 15 is a method of displaying three-dimensional (3D) content to a viewer using a display monitor system, the method comprising: receiving, by an integrated processing unit of the display monitor system, raw image data from an external source; receiving, from a camera of the display monitor system, position information of the viewer; performing, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data to generate mapped image data; and providing the mapped image data to a display panel for presentation to the viewer.

[0115] In Example 16, the subject matter of Example 15 includes, automatically determining, by the integrated processing unit, whether the raw image data corresponds to two-dimensional (2D) or 3D content; and switching the display panel between 2D and 3D display modes based on the determining.

[0116] In Example 17, the subject matter of Examples 15-16 includes, wherein receiving viewer position information comprises capturing images of the viewer using the camera and analyzing the images to determine 3D coordinates of eyes of the viewer.

[0117] In Example 18, the subject matter of Examples 15-17 includes, further comprising providing, by the integrated processing unit, real-time viewer position data to the external source to enable dynamic adjustment of the 3D content.

[0118] In Example 19, the subject matter of Examples 15-18 includes, further comprising converting 2D content to 3D content in real time using a neural processing in the integrated processing unit.

[0119] In Example 20, the subject matter of Examples 15-19 includes, wherein the raw image data is received via a wireless connection comprising Wi-Fi Direct.

[0120] Example 21 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-20.

[0121] Example 22 is an apparatus comprising means to implement of any of Examples 1-20.

[0122] Example 23 is a system to implement of any of Examples 1-20.

[0123] Example 24 is a method to implement of any of Examples 1-20.

[0124] Although an embodiment has been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader scope of the present disclosure. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. The accompanying drawings that form a part hereof show, by way of illustration, and not of limitation, specific embodiments in which the subject matter may be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. This Detailed Description, therefore, is not to be taken in a limitingsense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

[0125] The subject matter may be referred to herein, individually and / or collectively, by the term “embodiment” merely for convenience and without intending to voluntarily limit the scope of this application to any single inventive concept if more than one is in fact disclosed. Thus, although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description.

[0126] In this document, the terms "a" or "an" are used, as is common in patent documents, to indicate one or more than one, independent of any other instances or usages of "at least one" or "one or more." In this document, the term "or" is used to refer to a nonexclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise indicated. In this document, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, UE, article, composition, formulation, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects. As indicated herein, although the term “a” is used herein, one or more of the associated elements may be used in different embodiments. For example, the term “a processor” configured to carry out specific operations includes both a single processor configured to carry out all of the operations as well as multiple processors individually configured to carry out some or all of the operations (which may overlap) such that the combination of processors carry out all of the operations. Further, the term “includes” may be considered to be interpreted as “includes at least” the elements that follow.

[0127] The Abstract of the Disclosure is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, inthe foregoing Detailed Description, it may be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A three-dimensional (3D) monitor in a 3D system comprising: a display panel configured to present visual content to a viewer; and an integrated processing unit coupled to the display panel, the integrated processing unit configured to: receive raw image data from an external source; determine whether the raw image data corresponds to two-dimensional (2D) content or whether the raw image data corresponds to 3D content; switch the display panel between 2D and 3D display modes dependent on a determination of whether the raw image data corresponds to 2D content or whether the raw image data corresponds to 3D content; and for the determination that the raw image data corresponds to 3D content: receive viewer position information; perform, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data, the pixel mapping providing mapped image data dependent on detected eye positions and geometry of the 3D system; and provide the mapped image data to the display panel for presentation to the viewer.

2. The monitor of claim 1, wherein the integrated processing unit includes at least one of a system on chip (SoC), application-specific integrated circuit (ASIC), or field- programmable gate array (FPGA).

3. The monitor of claim 1 or 2, wherein the viewer position information comprises images captured by a camera integrated with the monitor.

4. The monitor of any of claims 1-3, wherein the integrated processing unit is further configured to receive eye center position data of eyes of the viewer as the viewer position information.

5. The monitor of any of claims 1-4, further comprising a touch panel coupled to the integrated processing unit, the integrated processing unit configured to provide touch input data and viewer position data of the viewer to an external device.

6. The monitor of any of claims 1-5, wherein the integrated processing unit is configured to: extract hand position data from images used for viewer tracking, and provide the hand position data to an external device.

7. The monitor of any of claims 1-6, wherein the integrated processing unit is configured to convert 2D content to 3D content in real time using a neural processing unit.

8. The monitor of any of claims 1-7, wherein the raw image data is received via at least one of a wired connection or a wireless connection, the wireless connection comprising Wi-Fi Direct.

9. The monitor of any of claims 1-8, wherein the integrated processing unit is further configured to output viewer position data to the external source to enable dynamic adjustment of 3D content.

10. The monitor of any of claims 1-9, wherein the display panel uses a periodic optical element that includes at least one of a lenticular lens array or a parallax barrier.

11. The monitor of any of claims 1-10, wherein the integrated processing unit is configured to perform pixel mapping based on calibration data of the display panel and optical elements, and on the geometry of the 3D system.

12. The monitor of any of claims 1-11, wherein the integrated processing unit comprises respective instances of a neural processing unit or digital signal processor (DSP) for accelerated computation of each of viewer tracking, pixel mapping, and 2D-to- 3D conversion.

13. A non-transitory computer-readable storage medium that stores instructions for execution by one or more processors of a monitor having a display panel in a three- dimensional (3D) system, the instructions, when executed, cause the one or more processors to: receive raw image data from an external source; determine whether the raw image data corresponds to two-dimensional (2D) content or whether the raw image data corresponds to 3D content; switch the display panel between 2D and 3D display modes dependent on a determination of whether the raw image data corresponds to 2D content or whether the raw image data corresponds to 3D content; and for the determination that the raw image data corresponds to 3D content: receive viewer position information; perform, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data, the pixel mapping providing mapped image data dependent on detected eye positions and geometry of the 3D system; and provide the mapped image data to the display panel for presentation to a viewer with a latency less than that of a system in which viewer tracking and pixel mapping are performed by a host device external to the 3D system.

14. The non-transitory computer-readable storage medium of claim 13, wherein the viewer position information comprises images captured by a camera integrated with the monitor.

15. A method of displaying three-dimensional (3D) content to a viewer using a display monitor system, the method comprising: receiving, by an integrated processing unit of the display monitor system, raw image data from an external source; receiving, from a camera of the display monitor system, position information of the viewer; performing, in real time, viewer tracking and pixel mapping based on the viewer position information and the raw image data to generate mapped image data; andproviding the mapped image data to a display panel for presentation to the viewer.

16. The method of claim 15, further comprising: automatically determining, by the integrated processing unit, whether the raw image data corresponds to two-dimensional (2D) or 3D content; and switching the display panel between 2D and 3D display modes based on the determining.

17. The method of claim 15 or 16, wherein receiving viewer position information comprises capturing images of the viewer using the camera and analyzing the images to determine 3D coordinates of eyes of the viewer.

18. The method of any of claims 15-17, further comprising providing, by the integrated processing unit, real-time viewer position data to the external source to enable dynamic adjustment of the 3D content.

19. The method of any of claims 15-18, further comprising converting 2D content to 3D content in real time using a neural processing in the integrated processing unit.

20. The method of any of claims 15-19, wherein the raw image data is received via a wireless connection comprising Wi-Fi Direct.

Images