System and method for rendering subpixel tiles in physically subsampled RGB format

Abstract

The video processing system (100) includes a graphics subsystem (110) including a graphics processing unit (GPU) (112) and a frame buffer (116). The GPU (112) is configured to: obtain a physical pixel layout corresponding to a display architecture of an electronic display, the physical pixel layout being characterized by a non-uniform sub-pixel arrangement, receive image data including a matrix of logical pixel chroma values, subsample the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement, store the subsampled image data in the frame buffer (116), and enable transfer of the subsampled image data to a display processing unit (DPU) (124) of the electronic display for composition of a frame having the non-uniform sub-pixel arrangement.

Description

[Technical Field]

[0001] The present disclosure relates to graphics rendering systems, and more particularly to graphics processing units that process image data using the physical sub-pixel arrangement of a display panel. [Background technology]

[0002]

[0002] Graphics rendering techniques process image data using a logical pixel array. Such an array typically includes red, green, and blue (RGB) pixels having the same size, shape, and layout density. A graphics processing unit (GPU) performs binning and rendering passes based on such a logical pixel array to compose a video frame and store the video frame in a frame buffer. A display processing unit (DPU), specific to a particular display, retrieves the video frame and performs further processing to convert the pixels in the video frame into a physical pixel array corresponding to the physical pixel layout of the display. The DPU displays the video frame on the display according to the physical pixel array.

[0003]

[0003] Conversion between logical-physical arrays and physical pixel arrays is often inefficient in terms of storage size and processing performance. Frame buffers must be large enough to store all of the logical pixels, and GPUs and DPUs must be efficient enough to process all of the logical pixels, regardless of their correspondence to physical pixels. As physical pixel arrays continue to become more diverse compared to traditional logical pixel arrays, such inefficiencies continue to increase, thereby reducing the performance of graphics rendering systems. Summary of the Invention

[0004]

[0004] Graphics rendering techniques according to the present disclosure process image data using pixel arrays that correspond to the physical pixel layout of a display panel on which the image data is intended to be displayed. Thus, such techniques optimize the logical-to-physical pixel layout conversion process, reduce frame buffer storage requirements, and improve performance in graphics rendering systems.

[0005] In one aspect, a video processing system includes a graphics subsystem including a graphics processing unit (GPU) and a frame buffer. The GPU is configured to: obtain a physical pixel layout corresponding to a display architecture of an electronic display, the physical pixel layout being characterized by a non-uniform sub-pixel arrangement, receive image data including a matrix of logical pixel chroma values, subsample the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a sub-pixel rendered format corresponding to the non-uniform sub-pixel arrangement, store the subsampled image data in the frame buffer, and enable transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for composition of a frame having the non-uniform sub-pixel arrangement.

[0006]

[0006] In some implementations, the GPU is further configured to perform pixel binning on the subsampled image data having a subpixel rendered format, perform pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and selected to cover overlap between physical subpixels of the physical pixel layout and logical subpixels of the received image data, perform texture processing on the subsampled image data having a subpixel rendered format, and / or perform anti-aliasing on the subsampled image data having a subpixel rendered format.

[0007]

[0007] Thus, the GPU is configured to render only those subpixels of the subsampled image data that correspond to the physical subpixels located on the electronic display, and to store in the frame buffer only those subpixels of the subsampled image data that correspond to the physical subpixels located on the electronic display. [Brief explanation of the drawings]

[0008]

[0008] For a better understanding of the various implementation forms described, reference should be made to the following "Form for Implementing the Invention" in conjunction with the following drawings in which like reference numbers refer to corresponding parts throughout the drawings. [Figure 1]

[0009] FIG. 1 is a block diagram of a video processing system, according to some implementations. [Figure 2]

[0010] FIG. 1 is a block diagram of a GPU rendering process flow according to some implementations. [Figure 3]

[0011] A diagram illustrating rendering techniques and subpixel arrangement types according to some implementations. [Figure 4]

[0012] FIG. 2 illustrates logical and physical pixel arrangements according to some implementations. [Figure 5]

[0013] A diagram illustrating a GPU rendering process flow according to some implementations. [Figure 6]

[0014] FIG. 1 is a flow diagram illustrating a bin / tile sub-pixel rendering method according to some implementations. DETAILED DESCRIPTION OF THE INVENTION

[0009]

[0015] 1 is a block diagram of a video processing system 100 according to some implementations. The video processing system 100 includes a central processing unit (CPU) 100, a graphics subsystem 110 including a memory (such as random access memory (RAM)) 104, an external memory 106, a graphics processing unit (GPU) 112, a graphics memory (such as video random access memory (VRAM)) 114, and a frame buffer 116, and a display 122 including a display processing unit (DPU) 124.

[0010]

[0016] CPU 102 (also referred to as processor or main processor) receives input data, which is image data intended to be displayed on display 122 in the form of video frames. The input data may also be referred to as an input signal. CPU 102 includes one or more processors or any other electronic circuitry configured to execute instructions comprising a computer program (e.g., a program stored in memory 104). CPU 102 performs initial processing on the input data and provides the input data to graphics subsystem 110 for rendering.

[0011]

[0017] Memories 104 and 106 include non-transitory computer-readable storage media, such as volatile memory (e.g., one or more random access memory devices) and / or non-volatile memory (e.g., one or more flash memory devices, magnetic disk storage devices, optical disk storage devices, or other non-volatile solid-state storage devices). The memories may include one or more storage devices located remotely from the processor(s). The memories store programs (described herein as modules and corresponding to instruction sets) that, when executed by the processor(s), cause video processing system 100 to perform functions as described herein. The modules and data described herein need not be implemented as separate programs, procedures, modules, or data structures. Thus, various subsets of these modules and data may be combined or otherwise rearranged in various implementations.

[0012]

[0018] GPU 112 (also called a graphics processor or graphics card) includes one or more processors or any other electronic circuitry configured to execute instructions comprising a computer program (e.g., a program stored in memory 104, 106, and / or 114). GPU 112 is a special-purpose processor configured to accelerate graphics rendering. GPU 112 is configured to process multiple image data simultaneously, compose image frames, and communicate the image frames to display 122 via frame buffer 116.

[0013]

[0019] Graphics memory 114 stores image data while it is being processed by GPU 112, and frame buffer 116 stores image data in the form of video frames upon completion of processing by GPU 112. Graphics memory 114 and frame buffer 116 each include a non-transitory computer-readable storage medium such as volatile memory (e.g., one or more random access memory devices) and / or non-volatile memory (e.g., one or more flash memory devices, magnetic disk storage devices, optical disk storage devices, or other non-volatile solid-state storage devices). Graphics memory 114 and frame buffer 116 may each include one or more storage devices located remotely from GPU 112 or integrated with GPU 112. Graphics memory 110 is dedicated to generating images in the form of frames (graphical representations) for display on display panel 122. Graphics memory 110 is sometimes referred to as GDDR or DDR / G-MEM (double data rate (DDR) memory specialized for high-speed rendering on GPUs). The GPU 112 transmits frames from the frame buffer 116 to the display 122 (also called a frame buffer).

[0014]

[0020] Display 122 is an electronic display that includes a matrix of physical pixels arranged in a particular pattern. Physical pixel arrangements are described in more detail below with reference to Figures 3 and 4. Display 122 may be a liquid crystal display (LCD) display, a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a quantum dot light emitting diode (QLED) display, an active matrix organic light emitting diode (AMOLED) display, an in-plane switching (IPS) display, a thin film transistor (TFT) display, a plane-to-plane switching (PLS) display, a low temperature polysilicon (LTPS) display, a low temperature polycrystalline oxide (LTPO) display, or any other type of electronic display with a physical pixel arrangement.

[0015]

[0021] The DPU 124 (also referred to as a display processor) includes one or more processors configured to convert display processor code corresponding to image frames communicated from the frame buffer 116 into graphical representations (images). The DPU 124 may include one or more of a display controller, a display file memory, a display generator, and a display console. The DPU 124 converts digital information corresponding to image frames into analog data corresponding to pixel voltages. This digital-to-analog conversion depends on the physical characteristics and graphics capabilities of the display 122.

[0016]

[0022] 2 is a block diagram of a GPU rendering process flow according to some implementations. In some implementations, the graphics subsystem 110 (specifically, the GPU 112) performs a tile-based rendering architecture pass on image data, including a binning pass and a rendering pass. In GPUs with memory bandwidth limitations (e.g., GPUs in mobile devices), the binning division operates on smaller subsets to reduce overall memory bandwidth consumption. Thus, the tile-based rendering achieves efficient utilization of all available resources and obtains acceptable performance.

[0017]

[0023] The binning module 202 is configured to perform a binning pass, which includes generating a stream / map between frame tiles and corresponding geometry to be rendered into a particular tile. In conventional processes, the binning pass may be performed on logical pixels having a predetermined arrangement that bears little or no relation to the physical pixel arrangement corresponding to the display 122 on which the image data is to be displayed. However, the techniques described in this disclosure perform the binning pass on pixels having an arrangement that corresponds to a particular physical pixel layout for the particular display 122 on which the image data is intended to be displayed. The binned image data is passed to the rendering module 204 for rendering.

[0018]

[0024] The rendering module 204 is configured to perform a rendering pass, which includes obtaining a map between tiles and geometry and rendering the appropriate pixels for each tile. In a conventional process, a rendering pass may be performed on logical pixels having a predetermined arrangement that bears little or no relationship to the physical pixel arrangement corresponding to the display 122 on which the image data is to be displayed. However, the techniques described in this disclosure perform a rendering pass on pixels having an arrangement that corresponds to a particular physical pixel layout for the particular display 122 on which the image data is intended to be displayed. The rendered image data is passed to the frame buffer 206.

[0019]

[0025] Frame buffer 206 temporarily stores image frames that are transmitted to display 122. The frame buffer corresponds to frame buffer 116 of Figure 1 described above. GPU 112 enables the transfer of image data to a display controller 208 of display 122.

[0020]

[0026] 1. Display controller 208 corresponds to DPU 124 I described above in Figure 1. Display controller 208 converts image data into analog signals that drive the physical pixels of display panel 210 of display 122.

[0021]

[0027] The display panel 210 includes a matrix (also called an array or arrangement) of physical pixels that, when operated at various degrees of luminance (brightness) and chromaticity (color), cause an image (a graphical representation of image data) to be displayed on the display 122.

[0022]

[0028] 3 includes diagrams illustrating rendering techniques and subpixel arrangement types, according to some implementations. Each pixel includes at least two subpixels, each corresponding to a different color.

[0023]

[0029] In pixel-based rendering, a pixel is the smallest addressable element in any point-addressable display device. It is therefore the smallest controllable element of an image represented on the display. Each pixel is accessible to the DPU, and its brightness is controlled at the pixel level. Specifically, a pixel can be lit at a particular brightness (luminance) or off. Sub-pixels within a pixel are not individually accessible to the DPU.

[0024]

[0030] In subpixel rendering, each subpixel is an addressable element, the smallest controllable element of a picture represented on a display. Each subpixel is accessible to the DPU. The DPU can therefore control the brightness of each individual subpixel. Specifically, each subpixel can be lit at a particular brightness (luminance) or off, regardless of the state of other subpixels of a given pixel. Subpixel rendering provides higher quality images with greater image texture. Image texture is a set of metrics calculated in image processing designed to quantify the perceived texture of an image. Image texture is associated with the spatial arrangement of color or intensity in an image or selected regions of an image. Because image elements are controllable at the subpixel level, the various lines and shapes that make up the image appear smoother and less variable.

[0025]

[0031] The pixels and subpixels are arranged in a particular arrangement (also called a pixel layout, pattern, or matrix). The pixels may be laid out in RGB stripes, which include repeating square or rectangular columns or rows of subpixels in an alternating pattern. Such an arrangement is also sometimes called a uniform subpixel arrangement.

[0026]

[0032] The pixels and their corresponding subpixels may be arranged in a non-uniform array. In some implementations, each pixel in the non-uniform array may include a first subpixel (e.g., green) having a first size and a second subpixel (e.g., red or blue) having a second size larger than the first size (as shown in the Pentile RGBG and Diamond Pentile Matrix arrays). In some implementations, each pixel in the non-uniform array may include one subpixel having a first color (e.g., red or blue) and two subpixels having a second color (e.g., green) different from the first color. In some implementations, each pixel in the non-uniform array may include less than three subpixels (as shown in the Pentile RGBG and Diamond Pentile Matrix arrays) or more than three subpixels (as shown in the Pentile RGBW array). A non-uniform arrangement may include any other arrangement of pixels and sub-pixels in which the color pattern does not repeat uniformly in each column (as shown in the RGB stripe arrangement of FIG. 3) and / or in each row (as shown in the RGB stripe arrangement of FIG. 4).

[0027]

[0033] 4 is a diagram illustrating logical and physical pixel arrangements according to some implementations. For purposes of illustration and clarity in comparing the logical and physical pixel arrangements, only half of the image in FIG. 4 is depicted in each format.

[0028]

[0034] 2, graphics subsystem 110 operates on logical pixels, while display 122 includes an array of physical pixels. If the logical and physical pixels have different layouts, as shown in FIG. 4, graphics subsystem 110 may waste both processing bandwidth and storage resources on logical pixels that do not have corresponding physical pixels on display 122.

[0029]

[0035] As shown in Figure 4, an image composed of logical pixels having an RGB strip arrangement may contain more pixels than an image composed of physical pixels having a pentile RGBG arrangement. Traditional graphics subsystems process the extra pixels (at the expense of processing bandwidth) and store the extra pixels in a frame buffer (at the expense of storage resources) even if the extra pixels are redundant and add nothing to the final depiction of the image on the physical display panel. Once communicated to the display, the DPU converts the image into a format that corresponds to the physical pixel layout, thereby discarding data describing redundant logical pixels that have no corresponding physical pixels.

[0030]

[0036] To minimize unnecessary processing bandwidth and storage overhead, the graphics processing techniques disclosed in this application operate on logical pixels arranged according to the particular non-uniform arrangement of the display architecture of the particular display 122 on which the image is to be displayed. Thus, referring to the example of Figure 4, if the physical pixels of a given display are arranged in a pentile RGBG arrangement, then graphics subsystem 110 processes image data according to the pentile RGBG layout (rather than according to an RGB strip layout as shown in the figure). Thus, substantially all of the image data processed in graphics subsystem 110 corresponds to physical pixels contained in display 122.

[0031]

[0037] 5 illustrates a GPU rendering process flow according to some implementations. During the binning pass, a stream of image data (sometimes called a visibility stream) is generated based on a non-uniform sub-pixel layout. The tiled rectangular binning may be expanded by an additional N logical pixels (e.g., 1-2 logical pixels) to cover overlaps in the physical sub-pixel layout.

[0032]

[0038] In some implementations, the GPU 112 performs anti-aliasing based on the physical sub-pixel layout and formats the image data using a priori knowledge of the display sub-pixel physical layout. The upsampled pixel RGB values ​​can be used to directly generate the physical sub-pixel values. The anti-aliasing parameters can be aligned or adjusted on the physical display. Thus, textures can be directly provided in the sub-pixel rendering format.

[0033]

[0039] During the rendering pass, layers may be delivered to graphics memory and ultimately resolved to system memory. Layers may be delivered in (i) a non-uniform physical sub-pixel format (e.g., pentile, etc.) or (ii) a sub-sampled RGB intermediate format (e.g., GRB4:2:2, GRB4:2:0, etc.), and the DPU can then filter and / or calibrate the delivered layers to the final physical sub-pixel values.

[0034]

[0040] In an example diamond-shaped pentile subpixel layout such as that shown in Figure 5, data associated with each subpixel may be processed and transmitted during a rendering pass according to a timing diagram associated with the physical pixel layout. For example, groups of two subpixels may be processed and transmitted on each successive clock cycle. Specifically, with reference to Figure 5, green pixel G00 and red subpixel R00 may be processed during a first clock cycle, green subpixel G01 and blue subpixel B01 may be processed during a second clock cycle, and so on.

[0035]

[0041] FIG. 6 is a flow diagram illustrating a bin / tile sub-pixel rendering method according to some implementations. The method may be governed by instructions stored in computer memory or a non-transitory computer-readable storage medium. The instructions may be included in one or more programs stored on the non-transitory computer-readable storage medium. When executed by one or more processors (e.g., 102, 112, and / or 124), the instructions cause the video processing system (e.g., 100) to perform a process. The non-transitory computer-readable storage medium may include one or more solid-state storage devices (e.g., flash memory), magnetic or optical disk storage devices, or other non-volatile memory devices. The instructions may include source code, assembly language code, object code, or any other instruction format that can be interpreted by one or more processors. Some operations in the process may be combined, and the order of some operations may be changed.

[0036]

[0042] The binning module 202 of the GPU 112 in the graphics subsystem 110 of the video processing system performs a binning pass (602) to generate a visibility stream based on physical sub-pixel information associated with the display panel 210 of the video processing system. The physical sub-pixel information includes a physical pixel layout corresponding to the display architecture of the display, where the physical pixel layout is characterized by a non-uniform sub-pixel arrangement.

[0037]

[0043] In some implementations, binning module 202 obtains the physical sub-pixel information (including a pixel layout corresponding to the display architecture of the display) as part of method 600. In other implementations, the physical sub-pixel information (including a pixel layout corresponding to the display architecture of the display) is stored locally in graphics subsystem 110 before method 600 is performed. In such implementations, binning module 202 has access to the physical sub-pixel information when method 600 begins.

[0038]

[0044] In performing the binning pass, the binning module 202 receives image data including a matrix of logical pixel chroma values ​​and subsamples the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform sub-pixel arrangement. In other words, the binning pass performs sub-pixel binning based on the non-uniform sub-pixel arrangement of the physical pixel layout of the display.

[0039]

[0045] In some implementations, binning module 202 performs pixel binning using rectangular tiles expanded by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover the overlap between (i) the physical sub-pixels of the physical pixel layout and (ii) the logical sub-pixels of the received image data. In other words, for scenarios in which the logical and physical sub-pixels do not map to each other on a one-to-one basis, binning module 202 may expand the rectangular tiles by a small number (e.g., 1 or 2) of logical pixels to ensure that all physical pixels corresponding to logical pixels in a given tile are covered within the binning pass for that tile.

[0040]

[0046] The rendering module 204 of the GPU 112 in the graphics subsystem 110 of the video processing system performs a rendering pass on the binned / tiled image data (604). In some implementations, the rendering pass includes directly texturing (providing texture) in the subpixel-rendered format (for the subsampled image data having the subpixel-rendered format). As described above, the rendering module 204 renders only those subpixels of the subsampled image data that correspond to the physical subpixels arranged on the electronic display. In other words, the logical subpixels used as the basis for the rendering pass are arranged in the same order as the physical subpixels of the display panel.

[0041]

[0047] The rendering module 204 of the GPU 112 performs anti-aliasing (for non-uniform physical sub-pixels) on the sub-sampled image data having the sub-pixel rendered format (606). Performing anti-aliasing on pixels having non-uniform physical sub-pixel arrangement provides smoother edges, thereby improving the quality of the rendered image.

[0042]

[0048] The rendering module 204 stores the rendered, subsampled image data in a frame buffer 206. In some implementations, the GPU is configured to store in the frame buffer only subpixels of the subsampled image data that correspond to physical subpixels located on the electronic display. In doing so, the GPU layers the rendered image data in graphics memory and resolves the rendered image data to system memory using the display panel's non-uniform physical subpixel format. In other words, after the rendering pass, when performing compositing, the GPU combines the layers to generate a display frame that is already composed for the display panel before being communicated to the DPU. Thus, the RGB layers are subsampled to the subpixel-rendered format, compositing is performed for the physical subpixels, and the composited frame is communicated to the frame buffer in the subsampled, subpixel-rendered format.

[0043]

[0049] In some implementations, the rendered subsampled image data corresponds directly to physical subpixels in the electronic display. In other implementations, the rendered subsampled image data may not correspond directly to physical pixels, but still provide the DPU with data that corresponds at least proportionally to physical pixels and can be converted to physical pixel data with relative computational simplicity. Specifically, while traditional rendering processes may store logical pixels that are independent of the electronic display's physical display pixel / subpixel format (e.g., logical pixels having an RGB fully sampled pixel format or a YUV422 or YUV420 related pixel format), the rendering path as described herein stores logical pixels in a frame buffer that is directly equal to or otherwise proportionally linked to the electronic display's physical display pixel / subpixel format.

[0044]

[0050] For example, logical pixels stored in a frame buffer (FB) may correspond directly to physical pixels of an electronic display. Logical pixels in a frame buffer may be referred to as having a direct physical sub-pixel format. Each byte in the frame buffer is equal to the corresponding sub-pixel value of the physical display. For example, if an electronic display contains physical pixels arranged in RGGB format (each color of a pixel is represented by, for example, 8 bits, 10 bits, or 12 bits), the following example shows a logical pixel format that may be stored in the frame buffer (as a result of a rendering pass) and passed to the DPU: R 08 bits | G 08 bits | G 08 bits | B 08 bits R 10-bit | G 10-bit | G 10-bit | B 10-bit R 12-bit | G 12-bit | G 12-bit | B 12-bit or any other physical pixel display format now known or in the future.

[0045]

[0051] As another example, logical pixels stored in a frame buffer (FB) may correspond proportionally to physical pixels of an electronic display. Logical pixels in a frame buffer may be referred to as having an intermediate physical sub-pixel format, but all bytes in the frame buffer have a simple conversion formula. For example, if an electronic display contains physical pixels arranged in RGGB format (e.g., each color of a pixel is represented by 8 bits), the following example shows a logical pixel format that may be stored in the frame buffer (as a result of a rendering pass) and passed to the DPU: R 16-bit | G 16-bit | G 16-bit | B 16-bit R 16 bits | G 08 bits | G 08 bits | B 08 bits

[0046]

[0052] In general, for intermediate physical sub-pixel formats, final physical sub-pixel value (in the DPU) = sub-pixel value (in the frame buffer) * For illustrative purposes, an example is as follows: R(at DPU)=R(in FB) * 0.6 B(at DPU)=B(in FB) * 0.6 G(at DPU)=G(in FB) * 1

[0047]

[0053] The GPU enables the transfer of subsampled image data to the DPU (208) of the display 122 for synthesis of frames having non-uniform subpixel arrangements. Specifically, the frames transferred to the DPU include only data for pixels that correspond to physical pixels included in the display panel of the display. Thus, the DPU does not need to convert logical pixels to physical pixels (or, if it does, processing is minimized because each logical pixel corresponds to a physical pixel included in the display panel). In implementations where logical pixels correspond proportionally to physical pixels, the DPU performs a relatively simple conversion from logical pixels to physical pixels (e.g., multiplying each logical pixel value of a particular color by a predetermined ratio of that color).

[0048]

[0054] In some implementations, the DPU performs additional display-panel-specific sub-pixel rendering adjustments and calibrations (610). However, this additional processing is greatly simplified because the pixel data associated with the frame communicated from the frame buffer already corresponds directly or proportionally to the physical pixel layout of the display.

[0049]

[0055] By performing the above-described method 600, the following scenarios can be achieved: Because there is no need to compose a frame using traditional RGB layers, one single layer can cover the entire frame. Furthermore, direct physical subpixel rendering saves memory usage, which benefits devices that are sensitive to memory (e.g., RAM) size. Furthermore, direct physical subpixel rendering saves graphics memory size, which benefits chipsets and devices that are sensitive to graphics memory size. In addition, direct subpixel rendering provides a relatively simple rendering context, which benefits power- and performance-sensitive devices. Furthermore, future display technologies may use physical subpixel formats that are even different from current RGB / YUV formats. In such scenarios, direct physical subpixel rendering enables seamless processing of future pixel arrays in the GPU, regardless of how complex the physical pixel layout may be. Additionally, by performing anti-aliasing based on the physical pixel array, subpixel-level anti-aliasing can be optimized for how the image will actually be viewed on the physical display panel, thereby improving image quality.

[0050]

[0056] Therefore, by taking into account the physical space, geometry, and layout of the display panel in the rendering stage of the graphics processing disclosed herein, improvements in power, performance, and visual quality and sharpness can be achieved. Additionally, future display technologies that use physical subpixels arranged in a non-uniform manner can benefit from this technique. Finally, by directly using the physical subpixel layout of the display panel, anti-aliasing can be more effective in improving visual quality and sharpness.

[0051]

[0057] The foregoing description has been described with reference to particular implementations. However, the above illustrative discussion is not intended to be exhaustive or to limit the scope of the claims to the precise form disclosed. Many variations are possible in light of the above teachings. The implementations were chosen and described to best explain the principles of operation and practical applications, thereby enabling others skilled in the art.

[0052]

[0058] The various figures depict elements in a particular order. However, elements that are not order-dependent may be rearranged, and other elements may be combined or separated. While some reorderings or other groupings are specifically mentioned, others will be apparent to those skilled in the art, and thus the ordering and groupings presented herein are not an exhaustive list of alternatives.

[0053]

[0059] As used herein, the singular forms "a," "an," and "the" include the plural forms unless the context clearly indicates otherwise, and the term "and / or" includes all possible combinations of one or more of the associated listed items. Terms such as "first," "second," etc. are used only to distinguish one element from another and do not limit the elements themselves, and the term "if" may be interpreted to mean "when," "upon," "in response to," or "in according with," depending on the context, and the terms "include," "including," "comprise," and "comprising" specify particular features or operations, but do not exclude additional features or operations. The inventions described in the claims of the present application as originally filed are set forth below. [C1] A video processing system, comprising: a graphics subsystem including a graphics processing unit (GPU) and a frame buffer; The GPU, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform subpixel arrangement; storing the subsampled image data in the frame buffer; enabling transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; 1. A video processing system configured to: [C2] The video processing system of C1, wherein the GPU is further configured to perform pixel binning on the subsampled image data having the subpixel rendered format. [C3] The video processing system of C2, wherein the GPU is configured to perform the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data. [C4] A video processing system according to any one of C1 to C3, wherein the GPU is further configured to perform texture processing on the subsampled image data having the subpixel rendered format. [C5] The video processing system of any one of C1 to C3, wherein the GPU is further configured to perform anti-aliasing on the subsampled image data having the subpixel rendered format. [C6] A video processing system as described in any one of C1 to C3, wherein the GPU is configured to render only subpixels of the subsampled image data that correspond to physical subpixels arranged on the electronic display. [C7] A video processing system as described in any one of C1 to C3, wherein the GPU is configured to store in the frame buffer only subpixels of the subsampled image data that correspond to physical subpixels arranged on the electronic display. [C8] A video processing system according to any one of C1 to C3, wherein each pixel of the non-uniform subpixel array includes a first subpixel having a first size and a second subpixel having a second size larger than the first size. [C9] A video processing system according to any one of C1 to C3, wherein each pixel of the non-uniform subpixel array includes one subpixel having a first color and two subpixels having a second color different from the first color. [C10] A video processing system according to any one of C1 to C3, wherein each pixel of the non-uniform subpixel array includes less than three subpixels or more than three subpixels. [C11] A method of operating a video processing system, comprising: In a graphics subsystem including a graphics processing unit (GPU) and a frame buffer, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform subpixel arrangement; storing the sub-sampled image data in the frame buffer; and enabling transfer of the sub-sampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; A method comprising: [C12] The method of C11, further comprising performing pixel binning on the subsampled image data having the subpixel rendered format. [C13] The method of C12, further comprising performing the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data. [C14] The method of any one of C11 to C13, further comprising performing texture processing on the subsampled image data having the subpixel rendered format. [C15] The method of any one of C11 to C13, further comprising performing anti-aliasing on the subsampled image data having the sub-pixel rendered format. [C16] The method of any one of C11 to C13, further comprising rendering only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels disposed on the electronic display. [C17] A method according to any one of C11 to C13, further comprising storing in the frame buffer only those sub-pixels of the sub-sampled image data that correspond to physical sub-pixels arranged on the electronic display. [C18] The method according to any one of C11 to C13, wherein each pixel of the non-uniform subpixel array includes a first subpixel having a first size and a second subpixel having a second size larger than the first size. [C19] The method according to any one of C11 to C13, wherein each pixel of the non-uniform subpixel array includes one subpixel having a first color and two subpixels having a second color different from the first color. [C20] The method according to any one of C11 to C13, wherein each pixel of the non-uniform subpixel array includes less than three subpixels or more than three subpixels.

Claims

1. 1. A video processing system comprising: a graphics subsystem including a graphics processing unit (GPU) and a frame buffer; The GPU, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform subpixel arrangement; performing pixel binning on the subsampled image data having the subpixel rendered format; rendering the subsampled image data after the pixel binning has been performed; storing the subsampled image data after it has been rendered in the frame buffer; enabling transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; 1. A video processing system configured to:

2. 2. The video processing system of claim 1, wherein the GPU is configured to perform the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data.

3. The video processing system of claim 1, wherein the rendering comprises performing texture processing on the subsampled image data having the subpixel rendered format.

4. The video processing system of claim 1, wherein the rendering comprises performing anti-aliasing on the subsampled image data having the subpixel rendered format.

5. 2. The video processing system of claim 1, wherein the GPU is configured to render only sub-pixels of the sub-sampled image data that correspond to physical sub-pixels disposed on the electronic display.

6. 2. The video processing system of claim 1, wherein the GPU is configured to store in the frame buffer only sub-pixels of the sub-sampled image data that correspond to physical sub-pixels located on the electronic display.

7. 2. The video processing system of claim 1, wherein each pixel in the non-uniform subpixel array includes a first subpixel having a first size and a second subpixel having a second size larger than the first size.

8. 2. The video processing system of claim 1, wherein each pixel in the non-uniform subpixel array includes one subpixel having a first color and two subpixels having a second color different from the first color.

9. The video processing system of claim 1 , wherein each pixel of the non-uniform sub-pixel arrangement includes less than three sub-pixels or more than three sub-pixels.

10. 1. A method of operating a video processing system, comprising: In a graphics subsystem including a graphics processing unit (GPU) and a frame buffer, obtaining a physical pixel layout corresponding to a display architecture of an electronic display, said physical pixel layout being characterized by a non-uniform sub-pixel arrangement; receiving image data including a matrix of logical pixel chroma values; subsampling the matrix of logical pixel chroma values ​​according to the physical pixel layout to generate subsampled image data having a subpixel rendered format corresponding to the non-uniform subpixel arrangement; performing pixel binning on the subsampled image data having the subpixel rendered format; rendering the subsampled image data after the pixel binning has been performed; storing the subsampled image data after it has been rendered in the frame buffer; enabling transfer of the subsampled image data to a display processing unit (DPU) of the electronic display for synthesis of a frame having the non-uniform sub-pixel arrangement; A method comprising:

11. 11. The method of claim 10, further comprising performing the pixel binning using rectangular tiles extended by N logical pixels, where N is an integer greater than or equal to 1 and is selected to cover overlap between physical sub-pixels of the physical pixel layout and logical sub-pixels of the received image data.

12. The method described in claim 10, wherein the rendering comprises performing texture processing on the subsampled image data having the subpixel rendered format.

13. The method of claim 10, wherein the rendering comprises performing anti-aliasing on the subsampled image data having the subpixel rendered format.

14. The method described in claim 10, wherein the rendering renders only subpixels of the subsampled image data that correspond to physical subpixels located on the electronic display.

15. The method of claim 10, wherein said storing stores in said frame buffer only those subpixels of said subsampled image data that correspond to physical subpixels located on said electronic display.

16. 11. The method of claim 10, wherein each pixel in the non-uniform subpixel array includes a first subpixel having a first size and a second subpixel having a second size larger than the first size.

17. 11. The method of claim 10, wherein each pixel in the non-uniform subpixel array includes one subpixel having a first color and two subpixels having a second color different from the first color.

18. The method of claim 10 , wherein each pixel of the non-uniform subpixel array comprises less than three subpixels or more than three subpixels.

Images