Display system

By optimizing image data transmission and processing between the graphics processor and the display controller, the problem of increased circuit cost and power consumption caused by high bandwidth requirements is solved, enabling high frame rate and low power consumption 3D image display, suitable for near-eye headphones.

CN111316204BActive Publication Date: 2024-11-12TEXAS INSTRUMENTS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880072398.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-12-26
Filing Date
2018-12-28
Publication Date
2024-11-12
Estimated Expiration
2038-12-28

AI Technical Summary

Technical Problem

Existing technologies face challenges in providing high display frame rate light field 3D or multi-focal plane 3D optical devices, which lead to increased circuit costs and power consumption due to high bandwidth requirements. Maximizing battery life and reducing heat generation are key challenges, especially in near-eye headphones.

Method used

By reducing the power consumption of the image data interface between the graphics processor and the display controller, reducing the bandwidth requirements and frame memory size of the display controller, sending concave image data directly from the graphics processor to the display controller, reducing the need for image reconstruction processing, using the YCrCb format to reduce the amount of data, and increasing the frame rate through an up-conversion circuit.

Benefits of technology

It achieves reduced power consumption and cost, while increasing frame rate, reducing image latency and suppressing color separation artifacts, supporting the generation of realistic 3D images at high frame rates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111316204B_ABST
    Figure CN111316204B_ABST
Patent Text Reader

Abstract

For image generation, a gaze tracking display system (100) includes a graphics processor (102) and a display circuit. The graphics processor (102) is configured to perform concave image data rendering and output concave image data. The display circuit is coupled to the graphics processor (102). The display circuit includes a display device (106) and a display controller (104). The display device (106) is configured to generate a visible image. The display controller (104) is configured to drive the display device (106). The display controller (104) includes a concave data reconstruction circuit (116) configured to generate an image at a resolution of the display device (106) based on the concave image data received from the graphics processor (102).
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] A variety of display technologies can be adapted for use by a single viewer. For example, near-eye headsets are used in virtual reality and augmented reality systems to provide a video display directed to a single user. In near-eye headsets, it is desirable to achieve high display frame rates for supporting light field three-dimensional (3D) optics or multi-focal plane 3D optics (or other non-stereoscopic 3D optics). Displaying images realistically in the optical domain may involve using light fields or multiple focal planes rather than traditional stereo methods. However, displaying 3D with stereo methods requires each eye to update at a frame rate of 60 Hertz (Hz) (both eyes together at a frame rate of 120 Hz), and using light fields or multiple focal planes requires much higher frame rates. For example, if in a multi-focal plane system, the image displayed at each focal plane is to be updated at 60 Hz (fast enough to prevent flickering at each plane) and six optical focal planes are supported, then an end-to-end frame rate of 60 Hz x 6 = 360 Hz is required. This means that the device that gets the images from somewhere else, usually a graphics processor, must generate the images at 360Hz, and then transmit those images to the display chipset at 360Hz while minimizing the power and cost of the electronics. This is challenging because the higher bandwidth due to the higher frame rate in this case usually increases the power and cost of the circuitry. Summary of the invention

[0002] Systems and methods for image generation in a gaze tracking display are disclosed herein. In one example, a gaze tracking display system includes: a graphics processor and a display circuit. The graphics processor is configured to: perform concave image data rendering; and output concave image data. The display circuit is coupled to the graphics processor. The display circuit includes a display device and a display controller. The display device is configured to generate a viewable image. The display controller is configured to drive the display device. The display controller includes a concave data reconstruction circuit configured to generate an image at a resolution of the display device based on the concave image data received from the graphics processor.

[0003] In another example, a method for generating an image display includes: performing concave image data rendering by a graphics processor to generate concave image data. The concave image data is transmitted from the graphics processor to a display controller that drives a display device. An output image is generated by the display controller from the concave image data received from the graphics processor at a resolution of the display device. The display device displays the output image.

[0004] In another example, a display system includes a display device and a display controller. The display device is configured to generate a viewable image. The display controller is configured to drive the display device. The display controller includes a frame buffer and a concave data reconstruction circuit. The frame buffer is configured to store the concave image data received from a graphics processor. The concave data reconstruction circuit is coupled to an output of the frame buffer. The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data stored in the frame buffer. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A block diagram of an image generation system according to various examples is shown.

[0006] Figure 2 Diagrams showing concave images according to various examples.

[0007] Figures 3A to 3C Spatial reconstruction of concave image data in a display controller according to various examples is shown.

[0008] Figure 4 A block diagram of a display controller that processes concave image data according to various examples is shown.

[0009] Figure 5 A flow chart of a method for generating image data in a graphics processor according to various examples is shown.

[0010] Figure 6 A flow chart of a method for processing image data in a display controller according to various examples is shown. DETAILED DESCRIPTION

[0011] In this description, the term "couple" or "couples" means an indirect or direct wired or wireless connection. Thus, if a first device is coupled to a second device, the connection may be through a direct connection or through an indirect connection via other devices and connections. Furthermore, in this description, the statement "based on" means "based at least in part on." Thus, if X is based on Y, X may be a function of Y and any number of other factors.

[0012] When it is desirable to provide high display frame rates for supporting light field three-dimensional (3D) optics or multi-focal plane 3D optics, the circuitry required to provide the bandwidth to achieve the high frame rates typically increases the cost and power consumption of the integrated circuits involved. Bandwidth drive power as well as display panel and controller power increase as pixel count and frame rate increase. Maximizing battery run time and minimizing headset heating are key issues in near-eye headsets.

[0013] Implementations of the display systems and methods of the present invention reduce the power consumed by the image data interface between a display controller and a graphics processor. Some implementations also reduce the cost of the display controller and the power consumed by the display controller by reducing the bandwidth required through a substantial portion of the circuitry of the display controller and by reducing the size of the frame memory provided in the display controller. The examples disclosed herein also reduce power and complexity of the graphics processor by allowing the graphics processor to send concave image data directly to the display controller and thereby eliminating the need for image reconstruction processing in the graphics processor.

[0014] Some embodiments allow for reducing the communication bandwidth from a graphics processor to a display controller by transmitting concave image data from the graphics processor to the display controller. The graphics processor performs the rendering of the concave image data, but rather than reconstructing the image, the graphics processor transmits the concave image data to the display controller. The display controller can reconstruct the image from the concave image data by filling in missing pixels.

[0015] By reducing the bandwidth required to transmit an image, some embodiments reduce the time required to transmit an image, thereby reducing image latency and providing increased frame rates that facilitate the generation of true three-dimensional images. Some implementations may also further increase the frame rate to the display to provide color separation artifact suppression.

[0016] Figure 1 A block diagram of a gaze tracking display system 100 is shown according to various examples. The gaze tracking display system 100 includes a graphics processor (GPU) 102, a display controller 104, a display device 106, and a gaze tracking camera 108. The display controller 104 and the display device 106 may be referred to herein as a display circuit or a display system. The display device 106 presents image data received from the display controller 104 in a visual form. The display device 106 may include a digital micromirror device (DMD), a liquid crystal display (LCD) device, a liquid crystal on silicon (LCOS) device, or other types of display technology. In some implementations of the gaze tracking display system 100, the display device 108 may implement a near-eye display used in a virtual reality or augmented reality headset. Some implementations of the display device 106 include upsampling circuitry and / or rendering.

[0017] The gaze tracking camera 108 captures an image of a user viewing the display device 106 and transmits the captured image to the GPU 102. The GPU 102 processes the image received from the gaze tracking camera 108 to determine the area of ​​the visual image produced by the display device 106 to which the user's eyes are directed. The GPU 102 is an instruction execution device (processor) that includes circuits for efficiently rendering images in memory. For example, the GPU 102 may include circuits that accelerate polygon rendering in memory. The GPU 102 performs operations specified by instructions retrieved from a computer-readable medium (e.g., a volatile or non-volatile memory device). The instructions executed by the GPU 102 may be adjusted for efficient manipulation of image data. The GPU 102 may include multiple execution units (hundreds or thousands of execution units) that allow the GPU 102 to perform many operations in parallel.

[0018] The GPU 102 includes a concave rendering circuit 110 that renders source image data 114 according to the area of ​​the visual image produced by the display device 106 to which the user's eyes are directed. That is, the concave rendering circuit 110 performs concave image data rendering. Concave rendering produces an image with a resolution that decreases with distance from the focus of the user's gaze.

[0019] Figure 2 An example of an image 200 generated using concave rendering according to various examples is shown. When generating image 200, GPU 102 processes the image received from gaze tracking camera 108 and determines that the user's eyes are focused on region 202. GPU 102 renders the portion of image 200 within region 202 at a high resolution, renders the portion of image 200 in region 204 at a resolution smaller than region 202, and renders the portion of image 200 in region 206 at a resolution smaller than region 204. In some implementations of GPU 102, region 202 may correspond to a region defined by an angle of about 12° around the center point of the user's focus. Therefore, relative to the image rendered at a high resolution as a whole, region 202 rendered at a high resolution may be relatively small and the amount of image data generated by concave rendering may be greatly reduced. For example, the image data generated by concave rendering may be 1 / 4 to 1 / 100 or less of the image data generated by rendering the entire image at a high resolution.

[0020] The GPU 102 may also include a frame buffer 112 that stores concave image data generated by the concave rendering circuit 102 for transmission to the display controller 104. Therefore, the GPU 102 does not reconstruct an image from the concave image data, but instead sends the concave image data to the display controller 104 for reconstruction. In some implementations, the GPU 102 may lack or bypass the frame buffer 112 and send the concave image data to the display controller 104 in bursts. Because the amount of data generated by concave rendering is greatly reduced relative to high-resolution image rendering, the bandwidth of the interface connecting the GPU 102 to the display controller 104 may be reduced, with a consequent reduction in data transfer power consumption. Because the GPU 102 does not reconstruct an image from the concave image data, the GPU 102 is less complex and consumes less power than a GPU that provides conventional concave image processing.

[0021] The display controller is a circuit that generates a signal (display synchronization signal) for driving a display device. For example, a first display controller generates a signal for driving an LCD, and a second display controller generates a signal for driving a DMD. The first display controller is not compatible with the DMD, and the second display controller is not compatible with the LCD. The display controller reads imaging data from a memory (frame buffer) and generates a signal to drive a display device based on the image data. The display controller 104 receives concave image data transmitted by the GPU 102, and reconstructs the image to be presented on the display device 106. The display controller 104 includes a frame buffer 114, a concave image data reconstruction circuit 116, and an up-conversion circuit 118. The frame buffer 114 may include two or more memory buffers, each of which is configured to store data of a concave image received from the GPU 102. The frame buffer 114 may implement a double buffer, in which one memory buffer is receiving concave image data from the GPU 102 and a different memory buffer is outputting concave image data of a frame previously received from the GPU 102 for reconstruction. For example, if GPU frame buffer 112 is bypassed, frame buffer 114 may receive concave image data from GPU 102 at a constant rate or in bursts, which may reduce image rendering latency.

[0022] The transfer of concave image data from the GPU 102 to the display controller 104 allows for several optimizations with respect to the frame buffer 114 and associated circuitry. Because the amount of data transferred for concave images is no more than 1 / 4 of the amount of corresponding high-resolution data, the input bandwidth of the frame buffer 114 can be reduced by at least a factor of 4, or up to 100 times or more for some images. Similarly, the size of the frame buffer 114 can be reduced by a factor of four relative to conventional systems, which reduces circuit area and cost. In addition, no image processing is required on the input side ("write side") of the frame buffer 114, which also reduces power consumption and cost of the display controller 104.

[0023] Implementations of the gaze tracking display system 100 may further reduce the amount of image data transferred from the GPU 102 to the display controller 104 by providing pixel data in a luma-chroma (YCrCb) format rather than a red-green-blue (RGB) format, where the YCrCb format pixel data may be decimated, for example, by a factor of 2 or 4. Given 30 bits per RGB format pixel and 20 bits per YCrCb format pixel, some implementations may reduce the size of the frame buffer 114 by a factor of 1.5 when using the YCrCb format. Given the 4x size reduction due to the concave image data, the additional size reduction due to the YCrCb pixel format allows the size of the frame buffer 114 to be reduced by a factor of 6 relative to conventional systems.

[0024] The display controller 104 enables the use of the YCrCb pixel format by providing image data processing after the frame buffer 114. Thus, in the display controller 104, concave image data received in the YCrCb pixel format may be converted to an RGB format on the read side of the frame buffer 114. In conventional systems employing a DMD in the display device 104, the data stored in the frame buffer must be in an RGB format.

[0025] Some implementations of display controller 104 include conversion circuitry in front of frame buffer 114 that converts concave image data received in RGB pixel format to YCrCb pixel format before storing the data in frame buffer 114 .

[0026] The concave image data reconstruction circuit 116 processes the concave image data read from the frame buffer 114. The processing provided by the concave image data reconstruction circuit 116 includes conversion from the YCrCb pixel format to the RGB pixel format and image reconstruction from the concave image data. The concave image data reconstruction circuit 116 may include a lookup table that provides conversion from the YCrCb pixel format to the RGB pixel format. In order to perform image reconstruction, the concave image data reconstruction circuit 116 includes a circuit that replicates pixel values ​​provided for one or more frames of concave image data received from the GPU 102. For example, the concave image data reconstruction circuit 116 may include a circuit that provides spatial replication of pixel data within a given frame and / or a circuit that provides temporal replication of pixel data across different frames. In the case of concave image data, the GPU 102 provides information (a concave header) that describes the frame data provided. In various implementations, the concave header may be provided for each portion of a frame of concave image data, for each frame of concave image data, for multiple frames of concave image data, or for a sequence of images. The display controller 104 receives and applies the concave header to reconstruct the image.

[0027] In some implementations of the gaze tracking display system 100, pixel processing is performed on a block basis. For example, in some implementations, blocks of 8×8 pixels are employed. Some implementations may employ different block sizes (e.g., 16×16). The 8×8 block size provides fine resolution to the GPU 102 during rendering with different concave compression factors used on a region-by-region basis.

[0028] At the beginning of each frame, the GPU 102 transmits a concave header to the display controller. The concave header provides information about each block in the image (e.g., each 8×8 block). The concave header can be sent at the beginning of each frame within the vertical blanking time. The typical amount of data in a concave header can be roughly equal to two image lines. Therefore, some embodiments transmit the concave header over a high-speed interface so that all data can be received within the vertical blanking time. Some embodiments of the gaze tracking display system 100 transmit the concave header over the same high-speed interface used to transmit image data from the GPU 102 to the display controller 104. In some embodiments, an interface that conforms to the Display Serial Interface (DSI) standard can be used to transmit concave image data and the concave header from the GPU 102 to the display controller 104.

[0029] The concave image data reconstruction circuit 116 decompresses the concave data of each block (e.g., each 8x8 block) on the read side of the frame buffer 114. The decompression restores the image represented by the concave data. To minimize bandwidth and power, the concave image data reconstruction circuit 116 may use pixel replication to fill in missing pixels. Figures 3A to 3CSpatial reconstruction of concave image data in a display controller according to various examples is shown. Figure 3A A 16x16 pixel block is shown where only a single pixel value, the shaded pixel, is provided in the concave image data received from the GPU 102. The concave image data reconstruction circuit 116 copies the provided pixel value to each pixel of the 16x16 block (each pixel in the 16x16 block that is below or to the right of the provided pixel value). Figure 3A The pattern represents a 256-fold reduction in image data. Figure 3B A 16x16 pixel block is shown where pixel values, i.e., shaded pixels, are provided for each 8x8 block of pixels in the concave image data received from the GPU 102. For each pixel value provided, the concave image data reconstruction circuitry 116 copies the provided pixel value to each pixel in the block that is below or to the right of the provided pixel value. Figure 3B The pattern represents a 64-fold reduction in image data. Figure 3C A 16x16 pixel block is shown where pixel values, i.e., shaded pixels, are provided for each 4x4 block of pixels in the concave image data received from the GPU 102. For each pixel value provided, the concave image data reconstruction circuitry 116 copies the provided pixel value to each pixel in the block that is below or to the right of the provided pixel value. Figure 3C The pattern represents a 16-fold reduction in image data.

[0030] Some embodiments of the concave image data reconstruction circuit 116 may be used with Figures 3A to 3C . For example, some embodiments may implement the replication of pixels so that the provided pixel values ​​are surrounded as equally as possible by the replicated pixels on all sides. Instead of simple replication, some embodiments may apply interpolation between the provided pixel values ​​to fill in the missing pixels.

[0031] The concave header of the image data block may include fields that specify spatial replication, temporal replication, and / or the pixel format of the block. The spatial replication field may specify the spatial decimation mode applied to the block and whether the mode is applied to adjacent blocks. For example, a seven-bit field may include six bits that specify which of 64 different decimation modes is applied to the block and a one-bit field that specifies that the same decimation mode has been applied to adjacent blocks. The temporal replication field may specify (e.g., via a single bit flag) whether the block is written to the frame buffer 114. For example, if the block is not written, the data of the previously written frame may be applied to replace the discarded pixel data. The pixel format field may specify the format and / or size of the pixel data, such as YCrCb or RGB and / or the number of pixel bits. For example, a three-bit field may specify that one of seven YCrCb or RGB pixel sizes is applied to the block.

[0032] The up-conversion circuit 118 receives the reconstructed image from the concave image data reconstruction circuit 116 and transmits the reconstructed image data to the display device 106 multiple times to increase the frame rate. The up-conversion circuit 118 may include a dual gamma circuit, a space-time multiplexing (STM) circuit, and a bit plane generation circuit. The dual gamma circuit adjusts the contrast or brightness of the image data received from the concave image data reconstruction circuit 116. The bit plane generation circuit generates a bit plane (e.g., a bit plane for each color) from the RGB pixel data received from the concave image data reconstruction circuit 116. The STM circuit applies dithering to the image data to be transmitted to the display device 106.

[0033] Each time an image is transmitted to the display device, the up-conversion circuit 118 applies a different spatial time multiplexing (STM) dithering pattern to the image data. This allows for lower dithering noise because each complete frame of data displayed has a unique set of spatial dithering patterns used. The faster temporal shifting of the spatial dithering patterns reduces the perceptibility of dithering noise. The up-conversion circuit 118 can increase the frame rate of the image data transmitted to the display device 106 to a rate of up to 1440 frames per second (or higher in some embodiments). By increasing the frame rate, the up-conversion circuit 118 can suppress or eliminate color separation artifacts.

[0034] In an implementation of the gaze tracking display system 100 in which the display device 106 includes a DMD, as the display frame rate increases, fewer bit planes need to be loaded per display frame time. This is because the temporal STM dither pattern moves at a faster rate due to showing each complete frame at a higher rate. This faster movement of the dither pattern helps improve the perceived signal-to-noise ratio for dithering. For example, 24 RGB bit planes may be required at 60 Hz, but only 9 RGB bit planes may be required at 1440 Hz to give the same temporal dither noise performance as perceived by a user viewing the image.

[0035] Figure 4 A block diagram of a display controller 104 is shown according to various examples. The display controller 104 includes a frame buffer 114, a concave image data reconstruction circuit 116, an upconversion circuit 118, and a display interface circuit 408. The frame buffer 414 includes a plurality of memory buffers 402. Each memory buffer 402 includes storage for a frame of concave image data received from the GPU 102. A multiplexer 410 or equivalent data selection circuit routes concave image data associated with a first frame from one of the memory buffers 402 to the concave image data reconstruction circuit 116, while concave image data for a second frame received from the GPU 102 is written to a different one of the memory buffers 402.

[0036] The concave image data reconstruction circuit 116 includes a color space conversion circuit 416, a concave header memory 412, a copy control circuit 414, a copy buffer / pixel position organizer 404, and a pixel space replicator 406. The multiplexer 410 routes the concave image data read from the selected one in the memory buffer 402 to the color space conversion circuit 416. The color space conversion circuit 416 converts the pixel data in the YCrCb format stored in the frame buffer 114 to the RGB format, and provides the RGB format pixel data to the copy buffer / pixel position organizer 404. The pixel format and size may vary across blocks because the GPU 102 may reduce the bit depth of blocks that are a predetermined distance from the user's focus (e.g., pixels of area 206 of the image 200 may be represented using fewer bits than pixels in areas 204 and / or 202).

[0037] For each frame of concave image data, GPU 102 transmits a concave header to display controller 104. The concave header is stored in concave header memory 412. Concave header memory 412 provides the concave header to copy control circuit 414 and copy buffer / pixel position organizer 404 for image reconstruction. The concave header specifies parameters for spatial and temporal replication of each block of concave image data.

[0038] The copy buffer / pixel position organizer 404 receives the concave image data in RGB format provided from the frame buffer 114 via the color space conversion circuit 416. If the display device 106 is loaded in a column-by-column manner (e.g., the display device 106 includes a DMD), the copy buffer / pixel position organizer 404 stores sufficient concave image data to allow a column of pixel data to be constructed within a few reads from the frame buffer 114. The pixel column data is scattered in the frame buffer 114. The copy buffer / pixel position organizer 404 sorts the data read from the frame buffer 114 into columns before performing spatial copies. Alternatively, some implementations of the display controller 104 may include circuitry that sorts the pixel data into columns before writing the data into the frame buffer 114.

[0039] The pixel space replicator 406 receives the pixel column data constructed by the copy buffer / pixel position organizer 404 and copies the pixel values ​​into the pixel positions whose values ​​are not provided in the concave image data. The pixel space replicator 406 includes a flip-flop that forms a column buffer to store the pixel data for a display column and a multiplexer that enables the routing of pixel values ​​to a given row of a column from the immediately higher row of the column and / or from the same row of the immediately previous column. In this way, the pixel space replicator 406 replicates pixel values ​​within and across columns.

[0040] The compressed RGB column data generated by the pixel space replicator 406 is provided to the up-conversion circuit 118. The up-conversion circuit 118 includes a dual gamma circuit, an STM circuit, and a bit plane generation circuit. The dual gamma circuit applies a dual gamma function to each pixel to be displayed on the display device 106. The bit plane generation circuit generates a bit plane (e.g., a bit plane for each color) from the RGB pixel data received from the pixel space replicator 406. The STM circuit applies dithering to the pixels to be displayed to the display device 106. The up-conversion circuit 118 includes a multiplexer that selects the bit plane to be transmitted to the display device 106.

[0041] Some implementations of display controller 104 may include circuitry to perform pixel replication prior to writing image data into frame buffer 114. In such implementations, a warping engine may be included to provide very high image quality during image reconstruction.

[0042] The display interface circuit 408 includes a driver circuit for driving image data to the display device and a control circuit for controlling the operation timing of the display device 106 and / or the transmission of image data to the display device.

[0043] Figure 5 A flow chart of a method 500 for generating image data in a graphics processor according to various examples is shown. Although depicted sequentially for convenience, at least some of the actions shown may be performed in a different order and / or in parallel. In addition, some implementations may only perform some of the actions shown. In some implementations, at least some of the operations of method 500 may be implemented by GPU 102.

[0044] In block 502, the gaze tracking camera 108 captures an image of an eye of a user of the gaze tracking display system 100. The image is processed by the GPU 102 or by another component of the gaze tracking display system 100 to generate gaze data that identifies the location of the display device 106 or the location of an image displayed on the display device 106 on which the user's eye is focused. The GPU 102 receives the gaze data for use in concave rendering of an image to be displayed on the display device 106.

[0045] In block 504, the GPU 102 applies the gaze data to render a concave image. The concave image provides high resolution in a relatively small area around the location where the user's gaze is focused and provides lower resolution at a location some distance from the location where the user's gaze is focused. The GPU 102 generates a concave header for each pixel block of the concave image. The concave header specifies spatial replication parameters, temporal replication parameters, and / or pixel format parameters for the pixel block. The pixels of the concave image can be represented in RGB format or in YCrCb format. Some embodiments use the YCrCb format to further reduce the amount of image data to be transmitted to the display controller 104.

[0046] In block 506, GPU 102 transmits the concave image data and the concave header to display controller 104. GPU 102 may transmit the concave image data and the concave header to display controller 104 via a DSI-compliant interface. The GPU does not reconstruct an image from the concave image data.

[0047] Figure 6 A flow chart of a method 600 for processing image data in a display controller according to various examples is shown. Although depicted sequentially for convenience, at least some of the actions shown may be performed in a different order and / or in parallel. Additionally, some implementations may perform only some of the actions shown. In some implementations, at least some of the operations of method 600 may be implemented by display controller 104. Method 500 may be performed in gaze tracking display system 100 along with method 600.

[0048] In block 602, display controller 104 receives concave image data and a concave header generated by GPU 102. Display controller 104 may receive the concave image data and the concave header generated by GPU 102 via a DSI-compliant interface.

[0049] In block 604 , the display controller 104 stores the concave image data in the frame buffer 114 and stores the concave header in the concave header memory 412 .

[0050] In block 606, the display controller 104 reads the concave image data from the frame buffer 114 and reads the concave header corresponding to the concave image data from the concave header memory 412. The concave image data and the concave header are provided to the concave image data reconstruction circuit 116.

[0051] In block 608, the concave image data reconstruction circuit 116 applies a concave header to convert the pixels of the concave image data from the YCrCb format to the RGB format. The concave header specifies the format of the pixel data.

[0052] In block 610, the concave image data reconstruction circuit 116 applies the concave header to reconstruct an image from the concave image data. Image reconstruction includes spatial replication of pixel data in a pixel block of the concave image data. The concave header specifies a pixel block decimation mode, which in turn determines how pixel replication will be performed (e.g., determines which pixels will be filled in by replication).

[0053] In block 612, a dual gamma function is applied to each pixel output by the data reconstruction circuit 116. The dual gamma function adjusts the contrast or brightness of the image data.

[0054] STM dithering is applied to the pixel data in block 614. Dithering applies a mask to the pixel information that selects a subset of pixels for display.

[0055] In block 616, the pixel data is converted into bit planes, and a bit plane is selected for display.

[0056] In block 618 , the data for the selected bit plane is driven to the display device 106 for presentation, and the display device 106 generates a visual display based on the received data.

[0057] The operations of blocks 614 through 618 may be performed at a rate that produces a high image transfer rate (high frame rate) to the display device. For example, the operations of blocks 614 through 618 may be performed multiple times for each concave image transferred from GPU 102 to display controller 104. In some implementations, the operations of blocks 614 through 618 may be performed at a rate suitable for suppressing color separation artifacts (e.g., a rate of 1440 frames per second).

[0058] Modifications are possible in the described embodiments, and other embodiments are possible within the scope of the claims.

Claims

1. A gaze tracking display system, comprising: A graphics processor configured to: Performing concave image data rendering; and outputting concave image data; A display circuit is coupled to the graphics processor, the display circuit comprising: A display device configured to produce a visible image; and a display controller configured to drive the display device, the display controller comprising a concave data reconstruction circuit and a frame buffer coupled to the concave data reconstruction circuit; The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data from the graphics processor; and The frame buffer is configured to store the concave image data from the graphics processor, provide the stored concave image data to the concave data reconstruction circuit, and store no more than one quarter of the pixel data of the image at the resolution of the display device.

2. The gaze tracking display system of claim 1, wherein the display device is configured to display only red, green, and blue encoded image data, and the graphics processor is configured to provide the concave image data encoded as luminance and chrominance values.

3. The gaze tracking display system of claim 1, wherein the display controller is configured to provide the image at the resolution of the display device to the display device at a rate of up to 1440 frames per second.

4. A gaze tracking display system, comprising: A graphics processor configured to: Performing concave image data rendering; and outputting concave image data; A display circuit is coupled to the graphics processor, the display circuit comprising: A display device configured to produce a visible image; and a display controller configured to drive the display device, the display controller comprising a concave data reconstruction circuit and a frame buffer coupled to the concave data reconstruction circuit; The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data from the graphics processor; The frame buffer is configured to store the concave image data from the graphics processor, provide the stored concave image data to the concave data reconstruction circuit, and store no more than one-sixth of the pixel data of the image at the resolution of the display device.

5. A gaze tracking display system, comprising: A graphics processor configured to: Performing concave image data rendering; and outputting concave image data; A display circuit is coupled to the graphics processor, the display circuit comprising: A display device configured to produce a visible image; and a display controller configured to drive the display device, the display controller comprising a concave data reconstruction circuit and a frame buffer coupled to the concave data reconstruction circuit; The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data from the graphics processor; The frame buffer is configured to store the concave image data from the graphics processor and provide the stored concave image data to the concave data reconstruction circuit; and The display controller is configured to: The concave image data stored in the frame buffer is converted as luma and chroma values ​​into red, green and blue encoded data before transmitting the concave image data to the concave data reconstruction circuit.

6. A gaze tracking display system, comprising: A graphics processor configured to: Performing concave image data rendering; and outputting concave image data; A display circuit is coupled to the graphics processor, the display circuit comprising: A display device configured to produce a visible image; and a display controller configured to drive the display device, the display controller comprising a concave data reconstruction circuit and a frame buffer coupled to the concave data reconstruction circuit; The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data from the graphics processor; the frame buffer being configured to store the concave image data from the graphics processor and to provide the stored concave image data to the concave data reconstruction circuit; The graphics processor is configured to: generating a concave header for the concave image data, the concave header comprising spatial replication information of blocks in the concave image data; and providing the concave header along with the concave image data to the display controller; and The display controller is configured to apply the spatial replication information to reconstruct the image from the concave image data at the resolution of the display device.

7. The gaze tracking display system of claim 6, wherein the concave header includes temporal copy information for the blocks in the concave image data, and the display controller is configured to apply the temporal copy information to control writing of the concave image data to the frame buffer.

8. A gaze tracking display system, comprising: A graphics processor configured to: Performing concave image data rendering; and outputting concave image data; A display circuit is coupled to the graphics processor, the display circuit comprising: A display device configured to produce a visible image; and a display controller configured to drive the display device, the display controller comprising a concave data reconstruction circuit and a frame buffer coupled to the concave data reconstruction circuit; The concave data reconstruction circuit is configured to generate an image at a resolution of the display device based on the concave image data from the graphics processor; The frame buffer is configured to store the concave image data from the graphics processor and provide the stored concave image data to the concave data reconstruction circuit; and The concave data reconstruction circuit is configured to: receiving the concave image data from the graphics processor; performing pixel replication to generate the image at the resolution of the display device; and The image at the resolution of the display device is stored in the frame buffer.

9. A method for generating an image display, the method comprising: The graphics processor performs concave image data rendering to generate concave image data; transmitting the concave image data from the graphics processor to a display controller driving a display device; generating, by the display controller, an output image from the concave image data from the graphics processor at a resolution of the display device; Displaying the output image by the display device; storing the concave image data in a frame buffer at an input of the display controller; and providing the concave image data to be processed from the frame buffer as input to the generating; and The concave image data stored in the frame buffer as luma and chroma values ​​is converted into red, green and blue encoded data prior to the generating.

10. The method of claim 9, wherein the displaying comprises driving the display device with red, green, and blue encoded image data; and wherein the concave image data is encoded as luma and chroma values.

11. The method of claim 9, further comprising providing the output image data to the display device at a rate of up to 1440 frames per second.

12. A method for generating an image display, the method comprising: The graphics processor performs concave image data rendering to generate concave image data; transmitting the concave image data from the graphics processor to a display controller driving a display device; generating, by the display controller, an output image from the concave image data from the graphics processor at a resolution of the display device; Displaying the output image by the display device; storing the concave image data in a frame buffer at an input of the display controller; and providing said concave image data to be processed from said frame buffer as input to said generating; generating, by the graphics processor, a concave header for the concave image data, the concave header comprising spatial replication information of pixels in the concave image data; providing the concave header along with the concave image data to the display controller via the graphics processor; and The spatial replication information is applied by the display controller to reconstruct the output image from the concave image data at the resolution of the display device.

13. The method of claim 12, wherein the concave header includes temporal replication information for the pixels in the concave image data; and wherein the method further comprises: The temporal duplication information is applied by the display controller to control writing of the concave image data to the frame buffer.

14. A display system comprising: A display device configured to produce a visible image; and A display controller is configured to drive the display device, the display controller comprising: a frame buffer configured to store concave image data from a graphics processor; and a concave data reconstruction circuit coupled to an output of the frame buffer, the concave data reconstruction circuit configured to generate an image at a resolution of the display device based on the concave image data stored in the frame buffer; The display controller is configured to convert the concave image data stored in the frame buffer as luma and chroma values ​​into red, green and blue encoded data before transmitting the concave image data to the concave data reconstruction circuit.

15. A display system comprising: A display device configured to produce a visible image; and A display controller is configured to drive the display device, the display controller comprising: a frame buffer configured to store concave image data from a graphics processor; and a concave data reconstruction circuit coupled to an output of the frame buffer, the concave data reconstruction circuit configured to generate an image at a resolution of the display device based on the concave image data stored in the frame buffer; The display controller is configured to: A concave header is received together with the concave image data, the concave header comprising: Spatial replication information for blocks in the concave image data; and temporal replication information for said blocks in said concave image data; and applying the spatial replication information to reconstruct the image from the concave image data at the resolution of the display device; and The temporal duplication information is applied to control writing of the concave image data to the frame buffer.