System and method for rendering sub-pixel tiles in physically subsampled RGB format

Abstract

The video processing system (100) includes a graphics subsystem (110) including a graphics processing unit (GPU) (112) and a frame buffer (116). The GPU (112) is configured to: obtain a physical pixel layout corresponding to a display architecture of an electronic display, the physical pixel layout being characterized by a non-uniform sub-pixel arrangement, receive image data including a matrix of logical pixel chroma values, subsample the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement, store the subsampled image data in the frame buffer (116), and enable transfer of the subsampled image data to a display processing unit (DPU) (124) of the electronic display for composition of a frame having the non-uniform sub-pixel arrangement.

Description

[Technical field]

[0001] The present disclosure relates to graphics rendering systems, and more particularly to graphics processing units that process image data using the physical sub-pixel arrangement of a display panel. [Background technology]

[0002]

[0002] Graphics rendering techniques process image data using a logical pixel array. Such an array typically includes red, green, and blue (RGB) pixels having the same size, shape, and layout density. A graphics processing unit (GPU) performs binning and rendering passes based on such logical pixel arrays to compose video frames and store the video frames in a frame buffer. A display processing unit (DPU), specific to a particular display, obtains the video frames and performs further processing to convert the pixels in the video frames into a physical pixel array corresponding to the physical pixel layout of the display. The DPU displays the video frames on the display according to the physical pixel arrays.

[0003]

[0003] The conversion between logical and physical pixel arrays is often inefficient in terms of storage size and processing performance. Frame buffers must be large enough to store all of the logical pixels, and GPUs and DPUs must be efficient enough to process all of the logical pixels, regardless of their correspondence to physical pixels. As physical pixel arrays continue to become more diverse compared to traditional logical pixel arrays, such inefficiencies continue to increase, thereby reducing the performance of graphics rendering systems. Summary of the Invention

[0004]

[0004] Graphics rendering techniques according to the present disclosure process image data using pixel arrays that correspond to the physical pixel layout of a display panel on which the image data is intended to be displayed. Such techniques therefore optimize the logical-to-physical pixel layout conversion process, reduce frame buffer storage requirements, and improve performance in graphics rendering systems.

[0005] In one aspect, a video processing system includes a graphics subsystem including a graphics processing unit (GPU) and a frame buffer. The GPU is configured to obtain a physical pixel layout corresponding to a display architecture of an electronic display, the physical pixel layout being characterized by a non-uniform sub-pixel arrangement, receive image data including a matrix of logical pixel chroma values, subsample the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement, store the subsampled image data in a frame buffer, and enable transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for composition of a frame having the non-uniform sub-pixel arrangement.

[0006]

[0006] In some implementations, the GPU is further configured to perform pixel binning on the subsampled image data having the subpixel rendered format, perform pixel binning using rectangular tiles spanned by N logical pixels, where N is an integer greater than or equal to 1 and selected to cover overlap between physical subpixels of the physical pixel layout and logical subpixels of the received image data, perform texture processing on the subsampled image data having the subpixel rendered format, and / or perform anti-aliasing on the subsampled image data having the subpixel rendered format.

[0007]

[0007] Thus, the GPU is configured to render only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display, and to store in the frame buffer only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display. [Brief description of the drawings]

[0008]

[0008] For a better understanding of the various implementation forms described, reference should be made to the following "Form for implementing the invention" in conjunction with the following drawings, in which like reference numbers refer to corresponding parts throughout the drawings. [Figure 1]

[0009] FIG. 1 is a block diagram of a video processing system, according to some implementations. [Diagram 2]

[0010] A block diagram of a GPU rendering process flow according to some implementations. [Diagram 3]

[0011] A diagram illustrating rendering techniques and sub-pixel arrangement types according to some implementations. [Figure 4]

[0012] FIG. 2 illustrates a diagram of logical and physical pixel arrangements according to some implementations. [Diagram 5]

[0013] A diagram illustrating a GPU rendering process flow according to some implementations. [Figure 6]

[0014] FIG. 1 is a flow diagram illustrating a bin / tile sub-pixel rendering method according to some implementations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009]

[0015] 1 is a block diagram of a video processing system 100 according to some implementations. The video processing system 100 includes a central processing unit (CPU) 100, a memory (such as random access memory (RAM)) 104, an external memory 106, a graphics subsystem 110 including a graphics processing unit (GPU) 112, a graphics memory (such as video random access memory (VRAM)) 114, and a frame buffer 116, and a display 122 including a display processing unit (DPU) 124.

[0010]

[0016] CPU 102 (also referred to as processor or main processor) receives input data, which is image data intended to be displayed on display 122 in the form of video frames. The input data may also be referred to as an input signal. CPU 102 includes one or more processors or any other electronic circuitry configured to execute instructions that include a computer program (e.g., a program stored in memory 104). CPU 102 performs initial processing on the input data and provides the input data to graphics subsystem 110 for rendering.

[0011]

[0017] The memories 104 and 106 include non-transitory computer-readable storage media, such as volatile memory (e.g., one or more random access memory devices) and / or non-volatile memory (e.g., one or more flash memory devices, magnetic disk storage devices, optical disk storage devices, or other non-volatile solid-state storage devices). The memories may include one or more storage devices located remotely from the processor(s). The memories store programs (described herein as modules and corresponding to instruction sets) that, when executed by the processor(s), cause the video processing system 100 to perform functions as described herein. The modules and data described herein need not be implemented as separate programs, procedures, modules, or data structures. Thus, various subsets of these modules and data may be combined or otherwise reconfigured in various implementations.

[0012]

[0018] GPU 112 (also called a graphics processor or graphics card) includes one or more processors or any other electronic circuitry configured to execute instructions that comprise a computer program (e.g., a program stored in memory 104, 106, and / or 114). GPU 112 is a special-purpose processor configured to accelerate graphics rendering. GPU 112 is configured to process multiple image data simultaneously, compose image frames, and communicate the image frames to display 122 via frame buffer 116.

[0013]

[0019] The graphics memory 114 stores image data while it is being processed by the GPU 112, and the frame buffer 116 stores image data in the form of a video frame upon completion of processing by the GPU 112. The graphics memory 114 and the frame buffer 116 each include a non-transitory computer-readable storage medium, such as a volatile memory (e.g., one or more random access memory devices) and / or a non-volatile memory (e.g., one or more flash memory devices, magnetic disk storage devices, optical disk storage devices, or other non-volatile solid-state storage devices). The graphics memory 114 and the frame buffer 116 may each include one or more storage devices that are remote from the GPU 112 or integrated with the GPU 112. The graphics memory 110 is dedicated to generating images in the form of frames (graphical representations) for display on the display panel 122. The graphics memory 110 is sometimes referred to as GDDR or DDR / G-MEM (double data rate (DDR) memory specialized for high-speed rendering on the GPU). The GPU 112 communicates frames from the frame buffer 116 to the display 122 (also called a frame buffer).

[0014]

[0020] Display 122 is an electronic display that includes a matrix of physical pixels arranged in a particular pattern. Physical pixel arrays are described in more detail below with reference to Figures 3 and 4. Display 122 may be a liquid crystal display (LCD) display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a quantum dot light emitting diode (QLED) display, an active matrix organic light emitting diode (AMOLED) display, an in-plane switching (IPS) display, a thin film transistor (TFT) display, a plane-to-live switching (PLS) display, a low temperature polysilicon (LTPS) display, a low temperature polycrystalline oxide (LTPO) display, or any other type of electronic display with a physical pixel array.

[0015]

[0021] The DPU 124 (also referred to as a display processor) includes one or more processors configured to convert display processor code corresponding to image frames communicated from the frame buffer 116 into a graphical representation (image). The DPU 124 may include one or more of a display controller, a display file memory, a display generator, and a display console. The DPU 124 converts digital information corresponding to image frames into analog data corresponding to pixel voltages. This digital-to-analog conversion depends on the physical characteristics and graphics capabilities of the display 122.

[0016]

[0022] 2 is a block diagram of a GPU rendering process flow according to some implementations. In some implementations, the graphics subsystem 110 (specifically, the GPU 112) performs a tile-based rendering architecture pass on the image data, including a binning pass and a rendering pass. In GPUs with memory bandwidth limitations (e.g., GPUs in mobile devices), the binning division works on smaller subgroups to reduce the overall memory bandwidth consumption. Thus, the tile-based rendering achieves efficient utilization of all available resources and obtains acceptable performance.

[0017]

[0023] The binning module 202 is configured to perform a binning pass, which includes generating a stream / map between frame tiles and corresponding geometry to be rendered into a particular tile. In a conventional process, the binning pass may be performed on logical pixels having a predetermined arrangement that has little or no relationship to the physical pixel arrangement corresponding to the display 122 on which the image data is to be displayed. However, the techniques described in this disclosure perform the binning pass on pixels having an arrangement that corresponds to a particular physical pixel layout for the particular display 122 on which the image data is intended to be displayed. The binned image data is passed to the rendering module 204 for rendering.

[0018]

[0024] The rendering module 204 is configured to perform a rendering pass, which includes obtaining a map between tiles and geometry and rendering the appropriate pixels for each tile. In a conventional process, a rendering pass may be performed on logical pixels having a predetermined arrangement that has little or no relationship to the physical pixel arrangement corresponding to the display 122 on which the image data is to be displayed. However, the techniques described in this disclosure perform a rendering pass on pixels having an arrangement that corresponds to a particular physical pixel layout for the particular display 122 on which the image data is intended to be displayed. The rendered image data is passed to the frame buffer 206.

[0019]

[0025] Frame buffer 206 temporarily stores image frames that are transmitted to display 122. The frame buffer corresponds to frame buffer 116 of Figure 1 described above. GPU 112 enables the transfer of image data to a display controller 208 of display 122.

[0020]

[0026] 1. Display controller 208 corresponds to DPU 124 I of FIG 1 above. Display controller 208 converts image data into analog signals that drive the physical pixels of a display panel 210 of display 122.

[0021]

[0027] The display panel 210 includes a matrix (also called an array or arrangement) of physical pixels that, when operated with various degrees of luminance (brightness) and chromaticity (color), cause an image (a graphical representation of image data) to be displayed on the display 122.

[0022]

[0028] 3 includes diagrams illustrating rendering techniques and subpixel arrangement types, according to some implementations. Each pixel includes at least two subpixels, and each subpixel corresponds to a different color.

[0023]

[0029] In pixel-based rendering, a pixel is the smallest addressable element in any point-addressable display device. It is therefore the smallest controllable element of an image represented on the display. Each pixel is accessible to the DPU and its brightness is controlled at the pixel level. Specifically, a pixel can be turned on or off at a particular brightness (luminance). Sub-pixels within a pixel are not individually accessible to the DPU.

[0024]

[0030] In sub-pixel rendering, each sub-pixel is an addressable element, the smallest controllable element of a picture represented on a display. Each sub-pixel is accessible to the DPU. Thus, the DPU can control the brightness of each individual sub-pixel. Specifically, each sub-pixel can be turned on or off at a particular brightness (luminance), regardless of the state of the other sub-pixels of a given pixel. Sub-pixel rendering provides higher quality images with greater image texture. Image texture is a set of metrics calculated in image processing designed to quantify the perceived texture of an image. Image texture is associated with the spatial arrangement of colors or intensities in an image or selected regions of an image. With the elements of the image controllable at the sub-pixel level, the various lines and shapes that make up the image appear smoother and less variable.

[0025]

[0031] The pixels and sub-pixels are arranged in a particular array (also called a pixel layout, pattern, or matrix). The pixels may be laid out in RGB stripes, which include repeating square or rectangular columns or rows of sub-pixels in an alternating pattern. Such an array may also be called a uniform sub-pixel array.

[0026]

[0032] The pixels and their corresponding subpixels may be arranged in a non-uniform array. In some implementations, each pixel in the non-uniform array may include a first subpixel (e.g., green) having a first size (as shown in the Pentile RGBG and Diamond-Shaped Pentile Matrix arrays) and a second subpixel (e.g., red or blue) having a second size larger than the first size. In some implementations, each pixel in the non-uniform array may include one subpixel having a first color (e.g., red or blue) and two subpixels having a second color (e.g., green) different from the first color. In some implementations, each pixel in the non-uniform array may include less than three subpixels (as shown in the Pentile RGBG and Diamond-Shaped Pentile Matrix arrays) or more than three subpixels (as shown in the Pentile RGBW array). A non-uniform arrangement may include any other arrangement of pixels and sub-pixels in which the color pattern does not repeat uniformly in every column (as shown in the RGB stripe arrangement of FIG. 3) and / or in every row (as shown in the RGB stripe arrangement of FIG. 4).

[0027]

[0033] 4 is a diagram illustrating logical and physical pixel arrangements according to some implementations. For purposes of illustration and clarity in comparing the logical and physical pixel arrangements, only half of the image in FIG. 4 is depicted in each format.

[0028]

[0034] As discussed above with reference to Figure 2, graphics subsystem 110 operates on logical pixels, while display 122 includes an array of physical pixels. If the logical and physical pixels have different layouts, as shown in Figure 4, graphics subsystem 110 may waste both processing bandwidth and storage resources on logical pixels that do not have corresponding physical pixels on display 122.

[0029]

[0035] As shown in Figure 4, an image composed of logical pixels with an RGB strip arrangement may contain more pixels than an image composed of physical pixels with a pentile RGBG arrangement. Conventional graphics subsystems process the extra pixels (at the expense of processing bandwidth) and store the extra pixels in a frame buffer (at the expense of storage resources) even if the extra pixels are redundant and add nothing to the final depiction of the image on the physical display panel. When communicated to the display, the DPU converts the image into a format that corresponds to the physical pixel layout, thereby discarding data that describes redundant logical pixels that do not have corresponding physical pixels.

[0030]

[0036] In order to minimize unnecessary processing bandwidth and storage overhead, the graphics processing techniques disclosed in this application operate on logical pixels arranged according to the particular non-uniform arrangement of the display architecture of the particular display 122 on which the image is to be displayed. Thus, referring to the example of Figure 4, if the physical pixels of a given display are arranged in a Pentile RGBG arrangement, the graphics subsystem 110 processes image data according to the Pentile RGBG layout (rather than according to an RGB strip layout as shown in the figure). Thus, substantially all of the image data processed in the graphics subsystem 110 corresponds to physical pixels contained in the display 122.

[0031]

[0037] 5 is a diagram illustrating a GPU rendering process flow according to some implementations. During the binning pass, a stream of image data (sometimes called a visibility stream) is generated based on a non-uniform sub-pixel layout. The tile rectangular binning may be expanded by an additional N logical pixels (e.g., 1-2 logical pixels) to cover the overlap of the physical sub-pixel layout.

[0032]

[0038] In some implementations, the GPU 112 performs anti-aliasing based on the physical sub-pixel layout and formats the image data using a priori knowledge of the display sub-pixel physical layout. The upsampled pixel RGB values ​​can be used to generate the physical sub-pixel values ​​directly. The anti-aliasing parameters can be aligned or adjusted on the physical display. Thus, textures can be provided directly in the sub-pixel rendering format.

[0033]

[0039] During the rendering pass, layers may be delivered to graphics memory and eventually resolved to system memory. Layers may be delivered in (i) a non-uniform physical sub-pixel format (e.g., pentile, etc.), or (ii) a sub-sampled RGB intermediate format (e.g., GRB4:2:2, GRB4:2:0, etc.), and the DPU can then filter and / or calibrate the delivered layers to the final physical sub-pixel values.

[0034]

[0040] In an example of a diamond-shaped pentile subpixel layout as shown in Figure 5, data associated with each subpixel may be processed and transmitted during a rendering pass according to a timing diagram associated with the physical pixel layout. For example, groups of two subpixels may be processed and transmitted every successive clock cycle. Specifically, with reference to Figure 5, green pixel G00 and red subpixel R00 may be processed during a first clock cycle, green subpixel G01 and blue subpixel B01 may be processed during a second clock cycle, and so on.

[0035]

[0041] FIG. 6 is a flow diagram illustrating a bin / tile sub-pixel rendering method according to some implementations. The method may be governed by instructions stored in a computer memory or a non-transitory computer-readable storage medium. The instructions may be included in one or more programs stored in the non-transitory computer-readable storage medium. When executed by one or more processors (e.g., 102, 112, and / or 124), the instructions cause the video processing system (e.g., 100) to perform a process. The non-transitory computer-readable storage medium may include one or more solid-state storage devices (e.g., flash memory), magnetic or optical disk storage devices, or other non-volatile memory devices. The instructions may include source code, assembly language code, object code, or any other instruction format that can be interpreted by one or more processors. Some operations in a process may be combined and the order of some operations may be changed.

[0036]

[0042] The binning module 202 of the GPU 112 in the graphics subsystem 110 of the video processing system performs a binning pass (602) to generate a visibility stream based on physical sub-pixel information associated with a display panel 210 of the video processing system. The physical sub-pixel information includes a physical pixel layout that corresponds to a display architecture of the display, where the physical pixel layout is characterized by a non-uniform sub-pixel arrangement.

[0037]

[0043] In some implementations, the binning module 202 obtains the physical sub-pixel information (including a pixel layout corresponding to the display architecture of the display) as part of the method 600. In other implementations, the physical sub-pixel information (including a pixel layout corresponding to the display architecture of the display) is stored locally in the graphics subsystem 110 before the method 600 is performed. In such implementations, the binning module 202 has access to the physical sub-pixel information when the method 600 begins.

[0038]

[0044] In performing the binning pass, the binning module 202 receives image data including a matrix of logical pixel chroma values ​​and subsamples the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement. In other words, the binning pass performs sub-pixel binning based on the non-uniform sub-pixel arrangement of the physical pixel layout of the display.

[0039]

[0045] In some implementations, binning module 202 performs pixel binning using rectangular tiles expanded by N logical pixels, where N is an integer greater than or equal to 1 and selected to cover the overlap between (i) the physical sub-pixels of the physical pixel layout and (ii) the logical sub-pixels of the received image data. In other words, for scenarios where the logical and physical sub-pixels do not map to each other on a one-to-one basis, binning module 202 may expand the rectangular tiles by a small number (e.g., 1 or 2) of logical pixels to ensure that all physical pixels corresponding to logical pixels in a given tile are covered within the binning pass for that tile.

[0040]

[0046] The rendering module 204 of the GPU 112 in the graphics subsystem 110 of the video processing system performs a rendering pass on the binned / tiled image data (604). In some implementations, the rendering pass includes texturing (providing texture) directly in the subpixel rendered format (for the subsampled image data having the subpixel rendered format). As described above, the rendering module 204 renders only those subpixels of the subsampled image data that correspond to physical subpixels located on the electronic display. In other words, the logical subpixels used as the basis for the rendering pass are arranged in the same order as the physical subpixels of the display panel.

[0041]

[0047] The rendering module 204 of the GPU 112 performs anti-aliasing (for non-uniform physical sub-pixels) on the subsampled image data having the sub-pixel rendered format (606). Performing anti-aliasing on pixels having non-uniform physical sub-pixel arrangement provides smoother edges, thereby improving the quality of the rendered image.

[0042]

[0048] The rendering module 204 stores the rendered subsampled image data in the frame buffer 206. In some implementations, the GPU is configured to store in the frame buffer only those subpixels of the subsampled image data that correspond to the physical subpixels located on the electronic display. In doing so, the GPU layers the rendered image data in the graphics memory and resolves the rendered image data to the system memory using the non-uniform physical subpixel format of the display panel. In other words, after the rendering pass, when performing compositing, the GPU puts the layers together to generate a display frame that is already composed to the display panel before being communicated to the DPU. Thus, the RGB layers are subsampled to the subpixel rendered format, compositing is performed to the physical subpixels, and the composited frame is communicated to the frame buffer in the subsampled and subpixel rendered format.

[0043]

[0049] In some implementations, the rendered subsampled image data corresponds directly to physical subpixels in the electronic display. In other implementations, the rendered subsampled image data may not correspond directly to physical pixels, but still provide the DPU with data that corresponds at least proportionally to physical pixels and can be converted to physical pixel data with relative computational simplicity. Specifically, while a traditional rendering process may store logical pixels that are independent of the physical display pixel / subpixel format of the electronic display (e.g., logical pixels having an RGB fully sampled pixel format or a YUV422 or YUV420 related pixel format), a rendering path as described herein stores logical pixels in a frame buffer that is directly equal to or otherwise proportionally linked to the physical display pixel / subpixel format of the electronic display.

[0044]

[0050] For example, logical pixels stored in a frame buffer (FB) may correspond directly to physical pixels of an electronic display. Logical pixels in a frame buffer may be referred to as having a direct physical sub-pixel format. Each byte in the frame buffer is equal to a corresponding sub-pixel value of the physical display. For example, if an electronic display contains physical pixels arranged in RGGB format (each color of a pixel is represented by, for example, 8 bits, 10 bits, or 12 bits), the following example shows a logical pixel format that may be stored in the frame buffer (as a result of a rendering pass) and passed to the DPU: R 08 bits | G 08 bits | G 08 bits | B 08 bits R 10 bits | G 10 bits | G 10 bits | B 10 bits R 12 bits | G 12 bits | G 12 bits | B 12 bits Or any other physical pixel display format now known or in the future.

[0045]

[0051] As another example, logical pixels stored in a frame buffer (FB) may correspond proportionally to physical pixels of an electronic display. Although logical pixels in a frame buffer may be referred to as having an intermediate physical sub-pixel format, every byte in the frame buffer has a simple conversion formula. For example, if an electronic display contains physical pixels arranged in RGGB format (e.g., each color of a pixel is represented by 8 bits), the following example shows a logical pixel format that may be stored in the frame buffer (as a result of a rendering pass) and passed to the DPU: R 16 bits | G 16 bits | G 16 bits | B 16 bits R 16 bits | G 08 bits | G 08 bits | B 08 bits

[0046]

[0052] In general, for intermediate physical sub-pixel formats, final physical sub-pixel value (in the DPU) = sub-pixel value (in the frame buffer) * For illustration purposes, an example is as follows: R(at DPU)=R(in FB) * 0.6 B(at DPU)=B(in FB) * 0.6 G(at DPU)=G(in FB) * 1

[0047]

[0053] The GPU enables the transfer of subsampled image data to a DPU (208) of the display 122 for synthesis of frames having non-uniform subpixel arrangements. Specifically, the frames transferred to the DPU include only data for pixels that correspond to physical pixels contained in the display panel of the display. Thus, the DPU does not need to convert logical pixels to physical pixels (or, if it does, processing is minimized because each logical pixel corresponds to a physical pixel contained in the display panel). In implementations where logical pixels correspond proportionally to physical pixels, the DPU performs a relatively simple conversion from logical pixels to physical pixels (e.g., multiplying each logical pixel value of a particular color by a predetermined ratio of that color).

[0048]

[0054] In some implementations, the DPU performs additional display panel-specific sub-pixel rendering adjustments and calibration (610). However, this additional processing is greatly simplified because the pixel data associated with the frame communicated from the frame buffer already corresponds directly or proportionally to the physical pixel layout of the display.

[0049]

[0055] By performing the method 600 described above, the following scenarios can be achieved: Since there is no need to compose a frame using traditional RGB layers, one single layer can cover the entire frame. Furthermore, direct physical subpixel rendering saves memory usage, which benefits devices that are sensitive to memory (e.g., RAM) size. Furthermore, direct physical subpixel rendering saves graphics memory size, which benefits chipsets and devices that are sensitive to graphics memory size. In addition, direct subpixel rendering provides a relatively simple rendering context, which benefits power-sensitive and performance-sensitive devices. Furthermore, future display techniques may use physical subpixel formats that are even different from the current RGB / YUV formats. In such scenarios, direct physical subpixel rendering enables seamless processing of future pixel arrays in the GPU, regardless of how complex the physical pixel layout may be. Additionally, by performing anti-aliasing based on the physical pixel array, subpixel-level anti-aliasing can be optimized for how the image will actually be seen on the physical display panel, thereby improving image quality.

[0050]

[0056] Thus, by taking into account the physical space, geometry, and layout of the display panel in the rendering stage of the graphics processing disclosed herein, improvements in power, performance, and visual quality and sharpness can be achieved. In addition, future display technologies that use physical sub-pixels arranged in a non-uniform manner can benefit from this technique. Finally, by directly using the physical sub-pixel layout of the display panel, anti-aliasing can be more effective in improving visual quality and sharpness.

[0051]

[0057] The foregoing description has been described with reference to specific implementations. However, the above illustrative discussion is not intended to be exhaustive or to limit the scope of the claims to the precise form disclosed. Many variations are possible in light of the above teachings. The implementations have been selected and described to best explain the principles of operation and practical applications, thereby enabling others skilled in the art.

[0052]

[0058] Various figures depict elements in a particular order. However, elements that are not order dependent may be rearranged and other elements may be combined or separated. While some reorderings or other groupings are specifically mentioned, others will be apparent to those of ordinary skill in the art, and thus the ordering and groupings presented herein are not an exhaustive list of alternatives.

[0053]

[0059] As used herein, the singular forms "a," "an," and "the" include the plural forms unless the context clearly indicates otherwise, and the term "and / or" includes all possible combinations of one or more of the associated listed items. Terms such as "first," "second," etc. are used only to distinguish one element from another and do not limit the elements themselves, the term "if" may be interpreted to mean "when," "upon," "in response to," or "in according with," depending on the context, and the terms "include," "including," "comprise," and "comprising" specify particular features or operations, but do not exclude additional features or operations.

Claims

1. 1. A video processing system comprising: a graphics subsystem including a graphics processing unit (GPU) and a frame buffer; The GPU, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform subpixel arrangement; storing the subsampled image data in the frame buffer; enabling transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; 1. A video processing system comprising:

2. 2. The video processing system of claim 1, wherein the GPU is further configured to perform pixel binning on the subsampled image data having the sub-pixel rendered format.

3. 3. The video processing system of claim 2, wherein the GPU is configured to perform the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data.

4. 4. The video processing system of claim 1, wherein the GPU is further configured to perform texture processing on the subsampled image data having the sub-pixel rendered format.

5. 4. The video processing system of claim 1, wherein the GPU is further configured to perform anti-aliasing on the subsampled image data having the sub-pixel rendered format.

6. 4. A video processing system according to claim 1, wherein the GPU is configured to render only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display.

7. 4. A video processing system according to claim 1, wherein the GPU is configured to store in the frame buffer only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display.

8. 4. The video processing system of claim 1, wherein each pixel of the non-uniform sub-pixel array includes a first sub-pixel having a first size and a second sub-pixel having a second size larger than the first size.

9. 4. The video processing system of claim 1, wherein each pixel of the non-uniform sub-pixel array includes one sub-pixel having a first color and two sub-pixels having a second color different from the first color.

10. The video processing system of any one of claims 1 to 3, wherein each pixel of the non-uniform sub-pixel array comprises less than three sub-pixels or more than three sub-pixels.

11. 1. A method of operating a video processing system, comprising the steps of: In a graphics subsystem including a graphics processing unit (GPU) and a frame buffer, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement; storing the subsampled image data in the frame buffer; enabling transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; A method comprising:

12. The method of claim 11 , further comprising performing pixel binning on the sub-sampled image data having the sub-pixel rendered format.

13. 13. The method of claim 12, further comprising performing the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data.

14. The method of any one of claims 11 to 13, further comprising performing texture processing on the sub-sampled image data having the sub-pixel rendered format.

15. The method of any one of claims 11 to 13, further comprising performing anti-aliasing on the sub-sampled image data having the sub-pixel rendered format.

16. 14. The method of claim 11, further comprising rendering only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display.

17. 14. The method of claim 11, further comprising storing in the frame buffer only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display.

18. 14. The method of claim 11, wherein each pixel of the non-uniform sub-pixel array comprises a first sub-pixel having a first size and a second sub-pixel having a second size larger than the first size.

19. 14. The method of claim 11, wherein each pixel of the non-uniform sub-pixel array comprises one sub-pixel having a first color and two sub-pixels having a second color different from the first color.

20. The method according to any one of claims 11 to 13, wherein each pixel of the non-uniform sub-pixel array comprises less than three sub-pixels or more than three sub-pixels.

Images