Fuzzy logic based pattern matching and corner filtering for display scalers.

Abstract

Aspects presented herein relate to a display processing method and device including an apparatus, e.g., a DPU. The apparatus may receive at least one input image for a scaling operation, the input image being associated with one or more scanning windows, each of the input image including a plurality of pixels. The apparatus may also detect one or more features within the plurality of pixels in each of the one or more scanning windows. Furthermore, the apparatus may adjust an amount of the plurality of pixels in each of the scanning windows for each of the detected features. The apparatus may also combine the adjusted amount of the plurality of pixels for each of the detected one or more features into a plurality of output pixels. The apparatus may also process each of the plurality of output pixels into at least one output image.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. patent application Ser. No. 17 / 648,414, filed Jan. 19, 2022, entitled "FUZZY LOGIC-BASED PATTERN MATCHING AND CORNER FILTERING FOR DISPLAY SCALER," the entire contents of which are expressly incorporated by reference into this specification.

[0002] The present disclosure relates generally to processing systems, and more particularly to one or more techniques for display processing. [Background technology]

[0003]

[0003] Computing devices often perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that work together to execute graphics processing commands and output frames. A central processing unit (CPU) may control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Modern CPUs are typically capable of simultaneously executing multiple applications, each of which may need to utilize the GPU during execution. The display processor is configured to convert digital information received from the CPU into analog values ​​and may issue commands to a display panel that displays the visual content. A device that provides content for visual presentation on a display may utilize a GPU and / or a display processor.

[0004]

[0004] A GPU of a device may be configured to perform processes in a graphics processing pipeline. Further, a display processor or display processing unit (DPU) may be configured to perform processes for display processing. However, with the advent of wireless communications and smaller handheld devices, there is an increasing need for improved graphics processing or display processing. Summary of the Invention

[0005]

[0005] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, nor is it intended to identify key or critical elements of all aspects, nor is it intended to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description presented later.

[0006]

[0006] In one aspect of the disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be a display processing unit (DPU), or any apparatus capable of performing display processing. The apparatus may receive at least one input image for a scaling operation, which is associated with one or more scanning windows, each of which includes a plurality of pixels. The apparatus may also detect one or more features within the plurality of pixels in each of the one or more scanning windows. In addition, the apparatus may calculate a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scanning windows. The apparatus may also adjust an amount of the plurality of pixels in each of the one or more scanning windows for each of the detected one or more features. The apparatus may also combine the adjusted amount of the plurality of pixels for each of the detected one or more features into a plurality of output pixels. Moreover, the apparatus may process each of the plurality of output pixels into at least one output image. The apparatus may also send the at least one output image to a display or panel after processing each of the plurality of output pixels.

[0007]

[0007] The details of one or more examples of this disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will become apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0008] [Figure 1]

[0008] FIG. 1 is a block diagram illustrating an exemplary content generation system. [Diagram 2]

[0009] 1 illustrates an exemplary graphics processing unit (GPU). [Diagram 3]

[0010] 1 illustrates an exemplary display framework including a display processor and a display. [Figure 4]

[0011] FIG. 2 illustrates an exemplary scaling technique for display processing. [Diagram 5]

[0012] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 6]

[0013] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 7]

[0014] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 8]

[0015] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 9]

[0016] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 10]

[0017] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 11]

[0018] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 12]

[0019] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 13]

[0020] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 14]

[0021] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 15]

[0022] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 16]

[0023] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 17]

[0024] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 18]

[0025] FIG. 2 illustrates an exemplary scaling architecture for display processing. [Figure 19]

[0026] FIG. 2 illustrates an exemplary scaling technique for display processing. [Figure 20]

[0027] FIG. 2 is a communication flow diagram illustrating exemplary communications between a memory, a DPU, and a display. [Figure 21]

[0028] FIG. 2 is a flow diagram of an exemplary method of display processing. [Figure 22]

[0029] FIG. 2 is a flow diagram of an exemplary method of display processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009]

[0030] Aspects of display processing may utilize different types of scaling components, such as scalers or DPU scalers, for different scaling operations (e.g., upscaling or downscaling). Some DPU scalers use a combination of bicubic filtering (i.e., edge-independent filtering) and gradient-based edge filtering to preserve high-frequency content during image upscaling. This technique can extract luma gradients to estimate edge direction and strength. By design, edge filtering may be strongest along diagonals (i.e., 45° and −45°) and weakest along horizontal (i.e., 0°) and vertical (i.e., 90°) directions. In some cases, edge filtering or directional output may be estimated by blending the output along two fixed directions using a weight metric. For example, output = output * Weights + Output *The formula (1-weight) may be utilized. In addition, the DPU scaler may utilize directional filtering, such as when the output is filtered in the detected direction. This filtered output in the detected direction may be a linear mix of the output along a certain angle direction (e.g., 26°-63° direction). Some scaling or filtering operations in display processing may include some drawbacks. For example, the scaling or filtering operations may not preserve the corners of the image well, since the luma gradient may be non-zero in two directions at the corners. Also, the scaling or filtering operations may not preserve the horizontal and vertical edges well, since the edge filtering may fall to 0 along these directions. Moreover, the scaling or filtering operations may not fully utilize the strong edge detection along the diagonal directions. Aspects of the present disclosure may utilize novel image scaling techniques for image reconstruction / enhancement processes in display applications. For example, aspects of the present disclosure may utilize logic-based pattern matching techniques and corner filtering techniques for the scaling or filtering operations. Also, aspects of the present disclosure may utilize scaling or filtering operations that preserve image corners, as well as preserve horizontal and vertical edges of the image. In addition, aspects of the present disclosure may utilize scaling or filtering operations that enable robust edge detection along diagonal directions of the image.

[0010]

[0031] Various aspects of the system, device, computer program product, and method are more fully described below with reference to the accompanying drawings. However, the present disclosure may be embodied in many different forms and should not be construed as limited to any specific structure or function presented throughout the present disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art should understand that the scope of the present disclosure is intended to encompass any aspect of the system, device, computer program product, and method disclosed herein, whether implemented independently of or in combination with other aspects of the present disclosure. For example, an apparatus can be implemented or a method can be practiced using any number of the aspects described herein. In addition, the scope of the present disclosure is intended to encompass such an apparatus or method that is practiced using other structures, functions, or structures and functions in addition to or other than the various aspects of the present disclosure described herein. Any aspect disclosed herein may be embodied by one or more elements of a claim.

[0011]

[0032] Although various aspects are described herein, many variations and permutations of these aspects fall within the scope of the present disclosure. Although some potential benefits and advantages of the aspects of the present disclosure are described, the scope of the present disclosure is not intended to be limited to any particular benefit, use, or purpose. Rather, the aspects of the present disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transmission protocols, some of which are illustrated by way of example in the figures and the following description. The detailed description and drawings are merely illustrative of the present disclosure, rather than limiting, the scope of the present disclosure being defined by the appended claims and their equivalents.

[0012]

[0033] Certain aspects are presented with reference to various apparatus and methods that are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, and the like (collectively referred to as "elements"). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system.

[0013]

[0034] As an example, the elements, or any portion of the elements, or any combination of the elements, may be implemented as a "processing system" (sometimes referred to as a processing unit) including one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems-on-chip (SOCs), baseband processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gate logic, discrete hardware circuits, and other suitable hardware configured to perform various functions described throughout this disclosure. The one or more processors in the processing system may execute software. Software may be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. The term application may refer to software. As described herein, one or more techniques may refer to an application, i.e., software configured to perform one or more functions.In such examples, the application may be stored on a memory, such as an on-chip memory of a processor, a system memory, or any other memory. Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more techniques described herein. As an example, the hardware may access code from a memory and execute the code accessed from the memory to perform one or more techniques described herein. In some examples, components are identified in this disclosure. In such examples, the components may be hardware, software, or a combination thereof. The components may be separate components or subcomponents of a single component.

[0014]

[0035] Thus, in one or more examples described herein, the functions described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored on or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. A storage medium may be any available medium that may be accessed by a computer. By way of example and not limitation, such computer-readable media may comprise random-access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the above types of computer-readable media, or any other medium that may be used to store computer-executable code in the form of instructions or data structures that may be accessed by a computer.

[0015]

[0036] In general, this disclosure describes techniques having a graphics processing pipeline in a device or multiple devices to improve rendering of graphical content and / or reduce the load on a processing unit, i.e., any processing unit configured to perform one or more techniques described herein, such as a GPU. For example, this disclosure describes techniques for graphics processing in any device that utilizes graphics processing. Other example benefits are described throughout this disclosure.

[0016]

[0037] Instances of the term "content" as used herein may refer to "graphical content," "images," and vice versa. This is true whether the terms are used as adjectives, nouns, or other parts of speech. In some examples, the term "graphical content" as used herein may refer to content produced by one or more processes of a graphics processing pipeline. In some examples, the term "graphical content" as used herein may refer to content produced by a processing unit configured to perform graphics processing. In some examples, the term "graphical content" as used herein may refer to content produced by a graphics processing unit.

[0017]

[0038] In some examples, the term "display content" as used herein may refer to content generated by a processing unit configured to perform display processing. In some examples, the term "display content" as used herein may refer to content generated by a display processing unit. Graphical content may be processed to become display content. For example, a graphics processing unit may output graphical content, such as a frame, to a buffer (sometimes referred to as a frame buffer). The display processing unit may read graphical content, such as one or more frames, from the buffer and perform one or more display processing techniques thereon to generate the display content. For example, a display processing unit may be configured to perform compositing on one or more rendered layers to generate a frame. As another example, a display processing unit may be configured to composite, blend, or otherwise combine two or more layers together into a single frame. A display processing unit may be configured to perform scaling, e.g., upscaling or downscaling, on a frame. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have already been blended together to form a frame, i.e., a frame includes two or more layers, and a frame including two or more layers may be blended later.

[0018]

[0039] FIG. 1 is a block diagram illustrating an example content generation system 100 configured to implement one or more techniques of the present disclosure. The content generation system 100 includes a device 104. The device 104 may include one or more components or circuits that perform various functions described herein. In some examples, one or more components of the device 104 may be components of a SOC. The device 104 may include one or more components configured to implement one or more techniques of the present disclosure. In the illustrated example, the device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, the device 104 may include several optional components, such as a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131. References to the display 131 may refer to one or more displays 131. For example, the display 131 may include a single display or multiple displays. Display 131 may include a first display and a second display. The first display may be a left-eye display and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation thereon. In other examples, the first and second displays may receive the same frames for presentation thereon. In further examples, the results of the graphics processing may not be displayed on the device, e.g., the first and second displays may not receive any frames for presentation thereon. Instead, the frames or graphics processing results may be transferred to another device. In some aspects, this may be referred to as split rendering.

[0019]

[0040] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing, such as in graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a display processor, such as display processor 127, to perform one or more display processing techniques on one or more frames generated by processing unit 120 prior to presentation by one or more displays 131. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise present frames processed by display processor 127. In some examples, the one or more displays 131 may include one or more of a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.

[0020]

[0041] Memory external to the processing unit 120 and the content encoder / decoder 122, such as a system memory 124, may be accessible to the processing unit 120 and the content encoder / decoder 122. For example, the processing unit 120 and the content encoder / decoder 122 may be configured to read from and / or write to an external memory, such as the system memory 124. The processing unit 120 and the content encoder / decoder 122 may be communicatively coupled to the system memory 124 via a bus. In some examples, the processing unit 120 and the content encoder / decoder 122 may be communicatively coupled to each other via a bus or a different connection.

[0021]

[0042] The content encoder / decoder 122 may be configured to receive graphical content from any source, such as the system memory 124 and / or the communications interface 126. The system memory 124 may be configured to store the received encoded or decoded graphical content. The content encoder / decoder 122 may be configured to receive the encoded or decoded graphical content, for example, in the form of encoded pixel data, from the system memory 124 and / or the communications interface 126. The content encoder / decoder 122 may be configured to encode or decode any graphical content.

[0022]

[0043] The internal memory 121 or the system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, the internal memory 121 or the system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic or optical data media, or any other type of memory.

[0023]

[0044] The internal memory 121 or the system memory 124 may be a non-transitory storage medium, according to some examples. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted to mean that the internal memory 121 or the system memory 124 are non-movable or that their contents are static. As one example, the system memory 124 may be removed from the device 104 and moved to another device. As another example, the system memory 124 may not be removable from the device 104.

[0024]

[0045] Processing unit 120 may be a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose GPU (GPGPU), or any other processing unit that may be configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In some examples, processing unit 120 may reside on a graphics card installed into a port in the motherboard of device 104, or may be otherwise integrated into a peripheral device configured to interoperate with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If these techniques are performed partially in software, processing unit 120 may store instructions for the software in a suitable non-transitory computer-readable storage medium, such as internal memory 121, and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, combinations of hardware and software, etc., may be considered to be one or more processors.

[0025]

[0046] The content encoder / decoder 122 may be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be embedded within the motherboard of the device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (AGAs), digital signal processors (DSPs), video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. Where these techniques are implemented in part in software, the content encoder / decoder 122 may store instructions for the software in a suitable non-transitory computer-readable storage medium, such as the internal memory 123, and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, combinations of hardware and software, etc., may be considered to be one or more processors.

[0026]

[0047] In some aspects, the content generation system 100 may include an optional communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any receiving function described herein with respect to the device 104. In addition, the receiver 128 may be configured to receive information from another device, such as eye or head position information, rendering commands, or location information. The transmitter 130 may be configured to perform any transmitting function described herein with respect to the device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined into a transceiver 132. In such an example, the transceiver 132 may be configured to perform any receiving and / or transmitting functions described herein with respect to the device 104.

[0027]

[0048] Referring again to FIG. 1 , in some aspects, the display processor 127 may include a determining component 198 configured to receive at least one input image for a scaling operation, the input image being associated with one or more scan windows, each of the input image including a plurality of pixels. The determining component 198 may also be configured to detect one or more features within the plurality of pixels in each of the one or more scan windows. The determining component 198 may also be configured to calculate a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scan windows. The determining component 198 may also be configured to adjust an amount of the plurality of pixels in each of the one or more scan windows for each of the detected one or more features. The determining component 198 may also be configured to combine the adjusted amount of the plurality of pixels for each of the detected one or more features into a plurality of output pixels. The determining component 198 may also be configured to process each of the plurality of output pixels into at least one output image. The determining component 198 may also be configured to transmit the at least one output image to a display or panel after processing each of the plurality of output pixels. The following description may focus on display processing, however the concepts described herein may be applicable to other similar processing techniques.

[0028]

[0049] As described herein, a device, such as device 104, may refer to any device, apparatus, or system configured to implement one or more techniques described herein. For example, a device may be a server, a base station, a user equipment, a client device, a station, an access point, a computer, such as a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer, an end product, an apparatus, a phone, a smartphone, a server, a video game platform or console, a handheld device, such as a portable video game device or a personal digital assistant (PDA), a wearable computing device, such as a smart watch, an augmented reality device, or a virtual reality device, a non-wearable device, a display or display device, a television, a television set-top box, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to implement one or more techniques described herein. Although the processes herein may be described as being performed by particular components (e.g., a GPU), in further embodiments, they may be performed using other components (e.g., a CPU) consistent with the disclosed embodiments.

[0029]

[0050] A GPU may process multiple types of data or data packets in a GPU pipeline. For example, in some aspects, a GPU may process two types of data or data packets, e.g., context register packets and draw call data. A context register packet may be global state information, e.g., information about global registers, shading programs, or a set of constant data, that may adjust how a graphics context is to be processed. For example, a context register packet may include information about a color format. In some aspects of a context register packet, there may be a bit that indicates which workload belongs to the context register. There may also be multiple functions or programming running simultaneously and / or in parallel. For example, a function or programming may represent a certain operation, e.g., a color mode or color format. Thus, a context register may define multiple states of a GPU.

[0030]

[0051] The context state may be utilized to determine how individual processing units function, e.g., a vertex fetcher (VFD), a vertex shader (VS), a shader processor, or a geometry processor, and / or in what mode the processing units function. To do so, the GPU may use context registers and programming data. In some aspects, the GPU may generate workloads, e.g., vertex or pixel workloads, in a pipeline based on the context register definition of a mode or state. Some processing units, e.g., the VFD, may use these states to determine some functionality, e.g., how vertices are assembled. Because these modes or states may change, the GPU may need to modify the corresponding context. In addition, the workloads corresponding to the modes or states may follow the changing mode or state.

[0031]

[0052] 2 illustrates an example GPU 200 in accordance with one or more techniques of the present disclosure. As illustrated in FIG. 2, the GPU 200 includes a command processor (CP) 210, a draw call packet 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z process engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a render backend (RB) 236, a level 2 (L2) cache (UCHE) 238, and a system memory 240. 2 shows GPU 200 including processing units 220-238, GPU 200 may include several additional processing units. In addition, processing units 220-238 are only one example, and any combination or order of processing units may be used by a GPU in accordance with this disclosure. GPU 200 also includes command buffer 250, context register packet 260, and context state 261.

[0032]

[0053] 2, the GPU may utilize a CP, e.g., CP 210, or a hardware accelerator, to parse the command buffer into context register packets, e.g., context register packet 260, and / or draw call data packets, e.g., draw call packet 212. CP 210 may then send the context register packet 260 or draw call packet 212 through separate paths to a processing unit or processing block within the GPU. Furthermore, command buffer 250 may alternate between different states of context registers and draw calls. For example, a command buffer may be constructed as follows: context register for context N, draw call(s) for context N, context register for context N+1, and draw call(s) for context N+1.

[0033]

[0054] A GPU may render an image in a variety of different ways. In some cases, a GPU may render an image using tile rendering and / or tile rendering. In a tile rendering GPU, an image may be divided or separated into different sections or tiles. After dividing the image, each section or tile may be rendered separately. A tile rendering GPU may divide a computer graphics image into a grid format, such that each portion of the grid, i.e., a tile, is rendered separately. In some aspects, during a binning pass, an image may be divided into different bins or tiles. In some aspects, during the binning pass, a visibility stream may be constructed, in which visible primitives or draw calls may be identified. In contrast to tile rendering, direct rendering does not divide a frame into smaller bins or tiles. Rather, in direct rendering, the entire frame is rendered at one time. In addition, some types of GPUs may enable both tile rendering and direct rendering.

[0034]

[0055] FIG. 3 is a block diagram 300 illustrating an exemplary display framework including a processing unit 120, a system memory 124, a display processor 127, and a display(s) 131, as may be identified with respect to an exemplary device 104.

[0035]

[0056] A GPU may be included in a device that provides content for visual presentation on a display. For example, processing unit 120 may include GPU 310 configured to render graphical data for display on a computing device (e.g., device 104), which may be a computer workstation, a mobile phone, a smartphone or other smart device, an embedded system, a personal computer, a tablet computer, a video game console, etc. The operation of GPU 310 may be controlled based on one or more graphics processing commands provided by CPU 315. CPU 315 may be configured to execute multiple applications simultaneously. In some cases, each of multiple applications executing simultaneously may utilize GPU 310 simultaneously. The processing techniques may be implemented to output frames via processing unit 120 over a physical or wireless communication channel.

[0036]

[0057] System memory 124, which may be executed by processing unit 120, may include user space 320 and kernel space 325. User space 320 (sometimes referred to as "application space") may include software application(s) and / or application framework(s). For example, the software application(s) may include an operating system, a media application, a graphical application, a workspace application, etc. The application framework(s) may include frameworks used by one or more software applications, such as libraries, services (e.g., display services, input services, etc.), application program interfaces (APIs), etc. Kernel space 325 may further include a display driver 330. Display driver 330 may be configured to control display processor 127. For example, display driver 330 may cause display processor 127 to compose frames and send data for the frames to a display.

[0037]

[0058] Display processor 127 includes a display control block 335 and a display interface 340. Display processor 127 may be configured to operate functions of display(s) 131 (e.g., based on input received from display driver 330). Display control block 335 may be further configured to output image frames to display(s) 131 via display interface 340. In some examples, display control block 335 may additionally or alternatively perform post-processing of image data provided to system memory 124 based on execution by processing unit 120.

[0038]

[0059] The display interface 340 may be configured to cause the display(s) 131 to display an image frame. The display interface 340 may output image data to the display(s) 131 according to an interface protocol, such as, for example, MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface). That is, the display(s) 131 may be configured according to the MIPI DSI standard. The MIPI DSI standard supports a video mode and a command mode. In an example where the display(s) 131 is operating in a video mode, the display processor 127 may continuously refresh the graphical content of the display(s) 131. For example, the entire graphical content may be refreshed every refresh cycle (e.g., line by line). In an example where the display(s) 131 is operating in a command mode, the display processor 127 may write the graphical content of a frame to the buffer 350.

[0039]

[0060] In some such examples, display processor 127 may not continuously refresh the graphical content of display(s) 131. Instead, display processor 127 may use a vertical synchronization (Vsync) pulse to coordinate the rendering and consumption of the graphical content in buffer 350. For example, when a Vsync pulse is generated, display processor 127 may output new graphical content to buffer 350. Thus, generation of a Vsync pulse may indicate that the current graphical content has been rendered in buffer 350.

[0040]

[0061] The frames are displayed on the display(s) 131 based on the display controller 345, the display client 355, and the buffer 350. The display controller 345 may receive image data from the display interface 340 and store the received image data in the buffer 350. In some examples, the display controller 345 may output the image data stored in the buffer 350 to the display client 355. The buffer 350 may thus represent a local memory for the display(s) 131. In some examples, the display controller 345 may output the image data received from the display interface 340 directly to the display client 355.

[0041]

[0062] The display client 355 may be associated with a touch panel that senses interaction between a user and the display(s) 131. As the user interacts with the display(s) 131, one or more sensors in the touch panel may output signals to the display controller 345 indicative of which of the one or more sensors have sensor activity, the duration of the sensor activity, the pressure applied to the one or more sensors, etc. The display controller 345 may use the sensor output to determine how the user has interacted with the display(s) 131. The display(s) 131 may further be associated with / include other devices, such as a camera, microphone, and / or speaker, that operate in conjunction with the display client 355.

[0042]

[0063] Some processing techniques of device 104 may be implemented over three stages (e.g., stage 1: rendering stage; stage 2: compositing stage; and stage 3: display / transfer stage). However, other processing techniques may combine the compositing stage and the display / transfer stage into a single stage such that the processing technique may be performed based on two total stages (e.g., stage 1: rendering stage; and stage 2: compositing / display / transfer stage). During the rendering stage, GPU 310 may process a content buffer based on the execution of an application that generates content for each pixel. During the compositing and display stage(s), pixel elements may be assembled to form a frame that is transferred to a physical display panel / subsystem (e.g., display 131) that displays the frame.

[0043]

[0064] The instructions executed by the CPU (e.g., software instructions) or display processor may cause the CPU or display processor to explore and / or generate a compositing strategy for compositing a frame based on dynamic priorities and runtime statistics associated with one or more compositing strategy groups. A frame to be displayed by a physical display device, such as a display panel, may include multiple layers. Compositing of a frame may also be based on combining multiple layers into a frame (e.g., based on a frame buffer). After the multiple layers are combined into a frame, the frame may be provided to the display panel for display on the display panel. The process of combining each of the multiple layers into a frame may be referred to as compositing, frame compositing, a compositing procedure, a compositing process, or the like.

[0044]

[0065] Aspects of display processing may utilize different types of scaling components, such as a scaler or a DPU scaler, for different scaling operations (e.g., upscaling or downscaling). Some DPU scalers use a combination of bicubic filtering (i.e., edge-independent filtering) and gradient-based edge filtering to preserve high-frequency content during image upscaling. This technique can extract luma gradients to estimate edge direction and strength. By design, edge filtering may be strongest along diagonals (i.e., 45° and −45°) and weakest along horizontal (i.e., 0°) and vertical (i.e., 90°) directions. In some cases, edge filtering or directional output may be estimated by blending the output along two fixed directions using a weight metric. For example, output = output 1 * Weights + Output 2 * The formula (1-weight) may be used. Additionally, the DPU Scaler may use directional filtering, such as when the output is filtered in the detected direction. This filtered output in the detected direction may be a linear mix of the output along a certain angle direction (e.g., a direction between 26° and 63°).

[0045]

[0066] FIG. 4 illustrates a diagram 400 including an example of a scaling technique for display processing. More specifically, the diagram 400 includes a linear combination of bicubic filtering and bidirectional filtering. As shown in FIG. 4, the diagram 400 includes an input image 410, an output image 450, and a filtering block 420 including a bicubic separable filtering component 422, a bidirectional edge filtering component 424, and a linear combination component 430. FIG. 4 illustrates how the input image 410 is filtered using the linear combination component 430 together with the bicubic separable filtering component 422 and the bidirectional edge filtering component 424 to obtain the output image 450. As shown in FIG. 4, the linear combination component 430 includes an edge strength metric. The output image 450 also illustrates the corners and edges of the image.

[0046]

[0067] Some scaling or filtering operations of display processing may include some drawbacks. For example, the scaling or filtering operations may not preserve the corners of the image well because the luma gradient may be non-zero in two directions at the corners. Also, the scaling or filtering operations may not preserve the horizontal and vertical edges because edge filtering may fall to zero along these directions while the bicubic separable filter preserves the horizontal and vertical edges. Moreover, the scaling or filtering operations may not fully utilize strong edge detection along the diagonal directions because there may be no direct filtering estimation along these directions. Based on the above, it may be beneficial for the scaling or filtering operations to preserve the corners of the image. It may also be beneficial for the scaling or filtering operations to preserve the horizontal and vertical edges of the image. Furthermore, it may be beneficial to utilize strong edge detection along the diagonal directions of the image.

[0047]

[0068] Aspects of the present disclosure may utilize novel image scaling techniques for image reconstruction / enhancement processes in display applications (e.g., high resolution display applications). For example, aspects of the present disclosure may utilize fuzzy logic based pattern matching and corner filtering techniques for the scaling or filtering operations. Also, aspects of the present disclosure may utilize scaling or filtering operations that enable preservation of image corners. Aspects presented herein may also utilize scaling or filtering operations that preserve horizontal and vertical edges of an image. Additionally, aspects of the present disclosure may utilize scaling or filtering operations that enable robust edge detection along diagonal directions of an image.

[0048]

[0069] In some cases, aspects presented herein can preserve image corners and edges while upscaling an image, which may contribute to the structure of the visual scene and thus result in higher image fidelity. Aspects of the present disclosure may provide a set of feature detection techniques that work together to preserve image corners and edges during image upscaling. For example, these feature detection techniques may include hardware-realizable fixed-point algorithms for constructing a high-resolution image using a single low-resolution input image. These feature detection techniques may outperform other edge detection-based DPU scalers, which may result in better preservation of image quality during upscaling, especially for graphics and text content.

[0049]

[0070] Aspects presented herein may include optimizations for the DPU scaler, which may be important when the input image is rich in graphics and text content. For example, aspects presented herein may utilize feature detection techniques, including fuzzy logic-based pattern matching techniques. Aspects presented herein may also include secondary luma gradient-based corner detection. Additionally, aspects presented herein may include three-way edge filtering. Aspects presented herein may also include non-maximum suppression algorithms. In some cases, aspects of the present disclosure may be based on several hypotheses, such as (1) the human visual system may be highly adapted to extract structural and geometric information from images, (2) edge-independent algorithms such as bicubic filtering may cause maximum blurring of edges along 45° and −45°, (3) well-defined and sharp corners of an image may be generally divided into two regions of opposite luma polarity, and (4) secondary luma gradients may take on high non-zero values ​​at the corners of an image.

[0050]

[0071] FIG. 5 illustrates a diagram 500 including an example of a scaling technique for display processing. More specifically, the diagram 500 includes a linear combination of bicubic filtering, corner filtering, and three-way filtering. As shown in FIG. 5, the diagram 500 includes an input image 510, an output image 550, and a filtering block 520 including a bicubic separable filtering component 522, a three-way edge filtering component 524, a pattern matching-based corner filtering component 526, and a linear combination component 530. FIG. 5 illustrates how the input image 510 is filtered using the linear combination component 530 along with the bicubic separable filtering component 522, the three-way edge filtering component 524, and the pattern matching-based corner filtering component 526 to obtain the output image 550. As shown in FIG. 5, the linear combination component 530 may include an edge strength metric and a corner strength metric. Also, the pattern matching-based corner filtering component 526 may include a prescribed turn. The output image 550 shows the corners and edges of the image, which may be sharper and more defined compared to other scaling techniques. As shown in FIG. 5, diagram 500 includes a fuzzy logic based pattern matching technique as well as a quadratic luma gradient based corner filtering technique.

[0051]

[0072] FIG. 6 illustrates a diagram 600 including an example of a scaling technique for display processing. As shown in FIG. 6, the diagram 600 includes an input image 610 and an output image 650. FIG. 6 illustrates how a scaling technique according to aspects of the present disclosure may include corner detection, edge detection, and / or edge direction estimation. For example, the aspects presented herein may include improved corner detection compared to other scaling techniques. Also, the aspects presented herein may include improved edge direction estimation compared to other scaling techniques. The aspects presented herein may also include sharper, less blurred edges of an image as well as better contrast compared to other scaling techniques.

[0052]

[0073] FIG. 7 shows a diagram 700 including an example of a scaling architecture for display processing. More specifically, the diagram 700 includes a scaling architecture for a display scaler or a DPU scaler. As shown in FIG. 7, the diagram 700 includes a pattern matching and corner detection component 710 (including non-maximum suppression), a direction detection component 712, and a finer direction estimation component 714 (including three-way estimation). The diagram 700 also includes a local storage (LS) 720, a direction pixel extractor 722, a corner pixel extractor 724, a triangle selector 730, a direction phase calculation component 732, a vertical phase (Vphase) component 734, a program pair selector 740, and a direction phase calculation component 742. In addition, diagram 700 includes a VphaseAccY component 750 (including inputs VphaseIncUV and VphaseIncY), a VphaseAccUV component 752, an HphaseAccY component 754 (including inputs HphaseIncUV and HphaseIncY), and an HphaseAccUV component 756. Diagram 700 also includes bilinear interpolators 760-763, two-dimensional (2D) separable interpolators 765-768, a direction blend component 770, a 2D separable filter 780, bilinear interpolators 781-784, a corner blend component 786, and a pixel blend component 790.

[0053]

[0074] As shown in FIG. 7, diagram 700 is a top-level scaling architecture of a DPU scaler that includes components for corner detection, corner interpolation, edge detection, and edge interpolation. The corner detection component of diagram 700 includes a pattern matching and corner detection component 710, a triangle selector 730, and a direction phase calculation component 732. The corner interpolation component of diagram 700 includes bilinear interpolators 781-784 and a corner blending component 786. The corner detection component and the corner interpolation component are novel elements in the design of diagram 700. Also, the novel non-maximum suppression algorithm may be integrated with corner filtering that allows the novel elements to work in conjunction with other features. The edge detection component of diagram 700 includes a direction detection component 712, a finer direction estimation component 714, a program pair selector 740, and a direction phase calculation component 742. In addition, the edge interpolation components of diagram 700 include bilinear interpolators 760-763, two-dimensional (2D) separable interpolators 765-768, and a direction blending component 770. The edge detection and edge interpolation components may be modified to accommodate newly introduced filtering aspects along a third direction (e.g., 45° or −45°).

[0054]

[0075] In aspects of the present disclosure, the combined block of fuzzy logic-based pattern detection and non-maximum suppression (or non-maxima suppression) may be a novel method for detecting corners in an image while mitigating false corner detection on edges, and vice versa. Thus, aspects presented herein may include novel elements of a scaling architecture (e.g., corner detection and interpolation elements) that may also be considered as standalone algorithms. Furthermore, aspects presented herein may also provide how these novel elements (e.g., corner detection and interpolation elements) interact with existing elements (e.g., edge-based filtering) through non-maximum suppression and information passing between the edge block and the corner filtering block. Based on image quality assessment, aspects presented herein may better preserve high frequency features that contribute to the structure of objects in an image. The aspects presented herein may also outperform other conventional filtering-based image scaling techniques and some artificial intelligence (AI)-based scalers for graphics or text use cases. Moreover, the aspects presented herein may be implemented with any image / video processing task that can benefit from corner detection-based filtering. In particular, the aspects presented herein may be utilized in displays that handle graphics content (e.g., gaming displays).

[0055]

[0076] In some cases, the pattern / corner detection of the present disclosure may be parameterized by a strength metric called Cornerness and a type metric called CornerType, which may indicate the orientation of the corner. The algorithm may detect two categories of corners (e.g., category 1 corners and category 2 corners). The categories of corners may differ in terms of their orientation type. In some cases, the category with lower detection strength may be discarded. Also, the four sub-detections at pixels surrounding the interpolation region may be bilinearly blended to output a final cornerness value. In some aspects, each sub-detection may be performed on a 3×3 luma grid.

[0056]

[0077] FIG. 8 shows a diagram 800 including an example of a display processing scaling technique. More specifically, FIG. 8 shows a sub-detection scheme and corner categorization. As shown in FIG. 8, the diagram 800 includes an input luma grid 810, a detection block 830 (using a neighborhood window), and cornerness and CornerType pairs 830 (e.g., four pairs of cornerness and CornerType). FIG. 8 shows that the detection block 830 utilizes detection at four points using a 3×3 neighborhood window. FIG. 8 also shows a category 1 corner and a category 2 corner (rotated 45 degrees compared to the category 1 corner).

[0057]

[0078] In category 1 corner detection (i.e., fuzzy logic based pattern matching), the detection may be based on computing a correlation metric between the input grid and a predefined pattern to parameterize the pattern strength. The correlation metric may be referred to as cornerness1. The detection process may include several different steps. For example, the detection process may include a preprocessing step that takes a 3×3 luma input and then outputs two 3×3 matrices called partition matrices (i.e., Ep and En). Local luma statistics may be exploited for preprocessing. The goal may be to partition the input grid into two regions of high luma polarity relative to the central pixel. This high polarity may be exploited in the next pattern matching step. The pattern matching step may take four defined patterns and match them against Ep and En to determine which pattern matches the luma input more closely. In addition, a non-maximum suppression block may be integrated to prevent false corner detection and mitigate interference with direction detection.

[0058]

[0079] FIG. 9 shows a diagram 900 including an example of a display processing scaling technique. More specifically, FIG. 9 shows a high-level scheme of fuzzy logic pattern detection. As shown in FIG. 9, the diagram 900 includes a 3×3 input 902, a pre-processing component 910 (i.e., a region partition component), a pattern matching component 920 (including false maximum suppression), predefined patterns 931-934 (3×3 patterns), and Cornerness and CornerType 940. The predefined patterns 931-934 show four orientations of patterns detected under category 1. Also, each of the patterns 931-934 may be of size 3×3 and may be divided into two regions (e.g., regions A and B) with opposite luma polarity.

[0059]

[0080] FIG. 10 shows a diagram 1000 including an example of a scaling technique for display processing. More specifically, FIG. 10 shows a low-level flow diagram of the pre-processing step. As shown in FIG. 10, the diagram 1000 includes an input 1010 (3×3 input) including detection points, a difference matrix 1020, a region partition matrix 1030, a region partition matrix 1040 (Ep), and an inverse partition matrix 1042 (En). FIG. 10 shows that there is a variance calculator for the input 1010 and the difference matrix 1020. There is also a sigmoid activation step after the region partition matrix 1030. As shown in FIG. 10, the diagram 1000 also includes an input 1050 that undergoes pre-processing to generate a pattern 1060. In addition, a pattern 1070 is compared with the pattern 1060 to determine whether there is a matching pattern.

[0060]

[0081] FIG. 11 shows a diagram 1100 including an example of a scaling architecture for display processing. As shown in FIG. 11, the diagram 1100 includes a pre-processing block 1102 including a 3×3 input 1110, an average calculator 1112, a variance calculator 1114, a partition matrix generator 1116, Ep 1118, and En 1119. FIG. 11 shows that the pre-processing block 1102 includes four similar instances, e.g., one instance for each 3×3 sub-window. For example, the pre-processing block 1102 includes a 3×3 input 1120, an average calculator 1122, a variance calculator 1124, a partition matrix generator 1126, Ep 1128, and En 1129. The pre-processing block 1102 also includes a 3×3 input 1130, a mean calculator 1132, a variance calculator 1134, a partition matrix generator 1136, Ep 1138, and En 1139. The pre-processing block 1102 further includes a 3×3 input 1140, a mean calculator 1142, a variance calculator 1144, a partition matrix generator 1146, Ep 1148, and En 1149.

[0061]

[0082] In some cases, the aspects presented herein may include a case-by-case breakdown of the pattern matching step. For example, the inputs in two cases may have the same pattern, but different polarities. In one case, the aspects presented herein may add all of the values ​​in a region (e.g., region A) in Ep and add all of the values ​​in a region (e.g., region B) in En. This may correspond to the following equation:

[0062]

number

[0063] In the formula, μ c represents cornerness 1, and Index type represents the CornerType. Also, Index type is Index max μ c In some cases, the first product in the formula may result in a higher value. In other cases, the second product in the formula may result in a higher value. In the case of an exact match, one of the two products may be 20.

[0064]

[0083] FIG. 12 shows a diagram 1200 including an example of a scaling architecture for display processing. More specifically, FIG. 12 shows a pattern matching block diagram. As shown in FIG. 12, the diagram 1200 includes Ep 1204, En 1206, a cornerness calculator 1210, a cornerness calculator 1211, a cornerness calculator 1212, a cornerness calculator 1213, a maximum component 1220, and a cornerness 1230. FIG. 12 shows that there may be four identical pattern matching blocks (e.g., one for each of the 3×3 inputs). Each of these blocks may have four identical sub-blocks (e.g., one for each of the four patterns). Thus, the diagram 1200 of FIG. 12 may include four times the amount of components compared to the current diagram.

[0065]

[0084] Figure 13 shows a diagram 1300 including an example of a scaling architecture for display processing. As shown in Figure 13, the diagram 1300 includes Ep1310, En1312, an Ep region A addition component 1321, an En region B addition component 1322, an Ep region B addition component 1323, an En region A addition component 1324, a sigmoid look-up table (LUT) 1330, a non-maximum suppression calculator 1340, and a cornerness 1350. Figure 13 shows a diagram of a cornerness calculator. For example, Figure 13 shows cornerness 1 modulation by a sigmoid activation step and a non-maximum suppression step.

[0066]

[0085] In some embodiments, the non-maximum suppression calculation may correspond to the following equation:

[0067]

Equation

[0068] In the equation, GST represents the gradient squared tensor. In some cases, the non-maximum suppression may be approximately equal to zero. Also, the non-maximum suppression may correspond to 0 < NonMaxSup < 1, which may be advantageous for providing sharper horizontal and vertical edges and may allow for some false detections along horizontal and vertical lines. GST is a metric used to parameterize the edge strength (which is considered to be strongest when the edge is along a diagonal). GST may also include variations associated with the edge angle. In some cases, input examples of false detections may include micro-corners present within the edge. Additionally, the non-maximum suppression can prevent false corner detections. In addition, consistent minor false detections may allow for partial bilinear / corner interpolation (e.g., when NonMaxSup = 1). In some examples, 4×4 interpolation may correspond to 2×2 interpolation. This may provide several advantages such as sharper horizontal and vertical lines, reduced halo artifacts, and / or enhanced corner sharpness. Also, this may be calculated using a bicubic filtering function or a bilinear sigmoid function.

[0069]

[0086] Aspects of the present disclosure may also utilize category 2 corner detection (i.e., corner detection using second-order luma gradients). Category 2 detection may calculate the magnitude of change in intensity between edges that intersect to form a corner. The intensity metric that quantifies the magnitude of this change is sometimes referred to as cornerness2. Edge strength may be parameterized by a set of metrics called gradient square tensors (GSTs) that are derived using first-order luma gradients. Second-order gradients (ΔGSTs) may be calculated as the difference between two first-order gradients calculated on adjacent 3×5 luma grids.

[0070]

[0087] FIG. 14 shows a diagram 1400 including an example of a display processing scaling technique. As shown in FIG. 14, the diagram 1400 includes a pixel-level representation 1410 of a sample input, as well as four GSTs (e.g., GST0=0, GST1=0, GST2=1, GST3=1). FIG. 14 shows how the primary gradient may vary horizontally and vertically with respect to the sample input, and how this may aid in the detection of category 2. In addition, the primary gradient may be invariant along the horizontal axis. Also, the primary gradient may vary along the vertical axis, suggesting the presence of a category 2 corner.

[0071]

[0088] Aspects of the present disclosure may also utilize corner interpolation. Corner interpolation may follow corner detection and may be a function of the detected CornerType index. For example, four sub-interpolants may be calculated independently for each of the four 3x3 grids and then bilinearly blended to obtain the final corner filter output. The interpolation filter may be a modified bilinear filter and may use three pixels (collectively referred to as a triangle) instead of four pixels.

[0072]

[0089] FIG. 15 illustrates a diagram 1500 including an example of a display processing scaling technique. More specifically, FIG. 15 illustrates a corner interpolation technique according to the present disclosure. As shown in FIG. 15, the diagram 1500 includes pixel 1501, pixel 1502, pixel 1503, and pixel 1504. FIG. 15 illustrates an example of category 1 corner interpolation. PhX and PhY represent the horizontal and vertical distances of the output pixel from pixel 1501. The values ​​of ph1 and ph2 (called directional phases) represent distance weight values ​​used for bilinear interpolation. As shown in FIG. 15, the horizontal nature of the edge may be preserved in one region, while the vertical nature of the edge may be preserved in another region. In some cases, category 1 triangular interpolation may be mixed with orientation-independent interpolation. By doing so, aspects presented herein may (i) ensure a smooth transition between corner and non-corner interpolation, and (ii) exploit false corner detection to better preserve horizontal and vertical edges. An orientation independent interpolation can be a simple bilinear interpolation using ph1 and ph2.

[0073]

[0090] FIG. 16 illustrates a diagram 1600 including an example of a display processing scaling technique. More specifically, FIG. 16 illustrates a category 2 corner interpolation technique according to the present disclosure. As shown in FIG. 16, the diagram 1600 includes pixel 1601, pixel 1602, pixel 1603, and pixel 1604. FIG. 16 illustrates an example of category 2 corner interpolation. PhX and PhY represent the horizontal and vertical distances of the output pixel from pixel 1601. The values ​​of ph45 and ph135 ​​(called directional phases) represent distance weight values ​​used for bilinear interpolation. As shown in FIG. 16, the diagonal nature of the edge may be preserved in one region and the diagonal nature of the edge may be preserved in another region.

[0074]

[0091] FIG. 17 illustrates a diagram 1700 including an example of a scaling architecture for display processing. More specifically, FIG. 17 illustrates a corner interpolation block diagram according to the present disclosure. As shown in FIG. 17, the diagram 1700 includes a pixel triangle selector 1710, a direction phase calculation component 1712, V0V1V2 components 1730-1733, bilinear interpolators 1740-1743, linear blending components 1750-1753, an orientation independent corner interpolator 1760, and a bilinear corner blending component 1770. FIG. 17 illustrates that a CornerType and a 2×2 pixel grid are input to the pixel triangle selector 1710. The pixel triangle selector 1710 can also utilize PhaseX and PhaseY when communicating with the bilinear corner blending component 1770, which then selects the corner out 17 also shows four parallel paths, one for each 3×3 subwindow.

[0075]

[0092] The embodiments presented herein may also utilize three-directional edge filtering. Current edge filtering may utilize bi-directional filtering, where edge filtering is estimated along 63° and 26° directions (or −63° and −26° directions) and linearly mixed based on a strength metric called finer direction estimation (FDE). The embodiments of the present disclosure may add a third estimation along 45° and 135° directions for improved preservation of edges, especially edges oriented near the diagonal direction where edge filtering strength is greatest. The scheme utilized by the embodiments presented herein may be similar for the negative direction, i.e., when the detected edge is along the negative direction. For example, 63° / −63° and 26° / −26° interpolations may be defined in the current algorithm, but the 45° / 135° directional filtering block may be new. The 45° / 135° directional filtering block may use four 3×3 2D separable filters. The choice of the 45° to 135° kernel may depend on the sign of the detected edge (indicated by the sign of the GST metric). Also, the outputs of the four separable filters may be bilinearly mixed to obtain a directional output.

[0076]

[0093] FIG. 18 illustrates a diagram 1800 including an example of a scaling architecture for display processing. More specifically, FIG. 18 illustrates the data flow of a 45° / 135° directional filtering block. As shown in FIG. 18, the diagram 1800 includes a directional phase calculation component 1812, a coefficient lookup table (LUT) 1820, a coefficient unpacking component 1822, program 3×3 45° components 1830-1833, program 3×3 135° components 1834-1837, program 3×3 components 1840-1843, separable filters 1850-1853, and a bilinear interpolator 1870. FIG. 18 illustrates that direct PhX and direct PhY are communicated from the directional phase calculation component 1812 to the coefficient lookup table (LUT) 1820. PhaseX and PhaseY are also communicated between the directional phase calculation component 1812 and the bilinear interpolator 1870. Finally, the bilinear interpolator 1870 can output a directional output.

[0077]

[0094] The aspects presented herein may also utilize pixel fusion processes, such as blending corner filtering, directional filtering, and bicubic filtering. In pixel fusion, the three directional outputs from the directional filtering block are triDir ) and the bicubic filter output can be combined using GST weights (representing edge strength). Also, the cornerness output (out cor ) and out triDir may be combined using the cornerness to provide the final output pixel.

[0078]

[0095] FIG. 19 illustrates a diagram 1900 including an example of a display processing scaling technique. More specifically, FIG. 19 illustrates a pixel fusion block architecture. As shown in FIG. 19, the diagram 1900 includes an input 1910, an output 1950, and a pixel fusion block 1902 including a bicubic separable filtering component 1920, a three-way edge filtering component 1922, a pattern matching based corner filtering component 1924, a prescribed pattern 1926, a linear combination 1930 (including GST weights), and a linear combination 1940 (including cornerness). As shown in FIG. 19, in some cases, out Dir =GST * out triDir +(1-GST) * out bicubic Also, in some cases out final =cornerness * out cor +(1-cornerness) * out dir It is.

[0079]

[0096] Aspects of the present disclosure may include several benefits or advantages. For example, aspects of the present disclosure better preserve high frequency features that contribute to the structure of objects in an image. Aspects presented herein may also outperform other conventional filtering-based image scaling techniques, as well as some AI-based scalers for graphics or text use cases.

[0080]

[0097] 20 is a communication flow diagram 2000 of display processing according to one or more techniques of the present disclosure. As shown in FIG. 20, the diagram 2000 includes example communication between a DPU 2002 (or other display processor), a memory 2004, and a display 2006 (e.g., a display panel) according to one or more techniques of the present disclosure.

[0081]

[0098] At 2010, the DPU 2002 may receive at least one input image for a scaling operation (e.g., input image 2012) associated with one or more scan windows, each of which includes a number of pixels. In some aspects, the one or more scan windows may be one or more pixel grids.

[0082]

[0099] In 2020, the DPU 2002 may detect one or more features within the pixels in each of the one or more scan windows. The one or more features within the pixels in each of the one or more scan windows may include at least one of one or more corners or one or more edges. The one or more corners may include one or more category 1 corners and one or more category 2 corners, where the one or more category 1 corners may be associated with a non-rotated horizontal axis and a non-rotated vertical axis, and the one or more category 2 corners may be associated with a 45 degree rotated horizontal axis and a 45 degree rotated vertical axis.

[0083]

[0100] At 2030, the DPU 2002 may calculate a confidence factor for each of the detected one or more features within the pixels in each of the one or more scan windows. In some cases, the calculation of the confidence factor for each of the one or more features may correspond to a non-maximum suppression process. The confidence factor for each of the one or more features may correspond to a confidence in the accuracy of each of the one or more features, and the confidence factor may be a value between 0 and 1. Also, the adjusted amount of pixel combination may be based on the confidence factor for each of the one or more features. Furthermore, the one or more features may include one or more corners and one or more edges, and the confidence factor may be an edge strength metric for each of the one or more edges and a corner strength metric for each of the one or more corners.

[0084]

[0101] At 2040, the DPU 2002 may adjust the amount of pixels in each of the one or more scan windows for each of the detected one or more features. In some aspects, adjusting the amount of pixels in each of the one or more scan windows may adjust the pixel resolution of the one or more scan windows. Also, adjusting the amount of pixels in each of the one or more scan windows may correspond to corner interpolation or edge interpolation. Edge interpolation may be associated with three-way edge filtering, and corner interpolation may be associated with pattern matching based corner filtering.

[0085]

[0102] At 2050, the DPU 2002 may combine the adjusted amount of pixels for each of the detected one or more features into a number of output pixels. The combination of the adjusted amount of pixels for each of the detected one or more features may be a linear combination.

[0086]

[0103] At 2060, the DPU 2002 may process each of the multiple output pixels into at least one output image. In some cases, the DPU may scan each of the multiple output pixels, where each of the multiple output pixels may be scanned in a scan order. For example, processing each of the multiple output pixels may include scanning each of the multiple output pixels, where each of the multiple output pixels may be scanned in a scan order. Additionally, each of the multiple output pixels may be processed in a display processing unit (DPU) or a DPU scaler.

[0087]

[0104] At 2070, the DPU 2002 may send at least one output image (e.g., output image 2072) to a display or panel (e.g., display 2006) after processing each of the multiple output pixels.

[0088]

[0105] 21 is a flow diagram 2100 of an example method of display processing in accordance with one or more techniques of this disclosure. The method may be performed by a DPU, such as a display processing device, a display processor, a wireless communication device, and / or any device that may perform display processing as used with respect to the examples of FIGS. 1-20.

[0089]

[0106] In 2102, the DPU may receive at least one input image for a scaling operation, associated with one or more scan windows, each including a plurality of pixels, as described with respect to the examples of Figures 1-20. For example, as described in 2010 of Figure 20, the DPU 2002 may receive at least one input image for a scaling operation, associated with one or more scan windows, each including a plurality of pixels. Furthermore, step 2102 may be performed by the display processor 127 of Figure 1. In some aspects, the one or more scan windows may be one or more pixel grids.

[0090]

[0107] At 2104, the DPU may detect one or more features within the pixels in each of the one or more scan windows, as described with respect to the examples of FIGS. 1-20. For example, as described at 2020 of FIG. 20, the DPU 2002 may detect one or more features within the pixels in each of the one or more scan windows. Furthermore, step 2104 may be performed by the display processor 127 of FIG. 1. The one or more features within the pixels in each of the one or more scan windows may include at least one of one or more corners or one or more edges. The one or more corners may include one or more category 1 corners and one or more category 2 corners, where the one or more category 1 corners may be associated with a non-rotated horizontal axis and a non-rotated vertical axis, and the one or more category 2 corners may be associated with a 45 degree rotated horizontal axis and a 45 degree rotated vertical axis.

[0091]

[0108] In 2108, the DPU may adjust the amount of pixels in each of the one or more scan windows for each of the one or more detected features, as described with respect to the examples of FIG. 1-20. For example, as described in 2040 of FIG. 20, the DPU 2002 may adjust the amount of pixels in each of the one or more scan windows for each of the one or more detected features. Furthermore, step 2108 may be performed by the display processor 127 of FIG. 1. In some aspects, adjusting the amount of pixels in each of the one or more scan windows may adjust the pixel resolution of the one or more scan windows. Also, adjusting the amount of pixels in each of the one or more scan windows may correspond to corner interpolation or edge interpolation. Edge interpolation may be associated with three-way edge filtering, and corner interpolation may be associated with pattern matching based corner filtering.

[0092]

[0109] In 2110, the DPU may combine the adjusted amount of pixels for each of the detected one or more features into multiple output pixels, as described with respect to the examples of Figures 1-20. For example, as described in 2050 of Figure 20, the DPU 2002 may combine the adjusted amount of pixels for each of the detected one or more features into multiple output pixels. Furthermore, step 2110 may be performed by the display processor 127 of Figure 1. The combination of the adjusted amount of pixels for each of the detected one or more features may be a linear combination.

[0093]

[0110] In 2112, the DPU may process each of the plurality of output pixels into at least one output image, as described with respect to the examples of FIGS. 1-20. For example, as described at 2060 of FIG. 20, the DPU 2002 may process each of the plurality of output pixels into at least one output image. Furthermore, step 2112 may be performed by the display processor 127 of FIG. 1. In some cases, the DPU may scan each of the plurality of output pixels, and each of the plurality of output pixels may be scanned in a scan order. For example, processing each of the plurality of output pixels may include scanning each of the plurality of output pixels, and each of the plurality of output pixels may be scanned in a scan order. Additionally, each of the plurality of output pixels may be processed in a display processing unit (DPU) or a DPU scaler.

[0094]

[0111] 22 is a flow diagram 2200 of an example method of display processing in accordance with one or more techniques of this disclosure. The method may be performed by a DPU, such as a display processing device, a display processor, a wireless communication device, and / or any device that may perform display processing as used with respect to the examples of FIGS. 1-20.

[0095]

[0112] In 2202, the DPU may receive at least one input image for a scaling operation, associated with one or more scan windows, each including a plurality of pixels, as described with respect to the examples of Figures 1-20. For example, as described in 2010 of Figure 20, the DPU 2002 may receive at least one input image for a scaling operation, associated with one or more scan windows, each including a plurality of pixels. Furthermore, step 2202 may be performed by the display processor 127 of Figure 1. In some aspects, the one or more scan windows may be one or more pixel grids.

[0096]

[0113] At 2204, the DPU may detect one or more features within the pixels in each of the one or more scan windows, as described with respect to the examples of FIGS. 1-20. For example, as described at 2020 of FIG. 20, the DPU 2002 may detect one or more features within the pixels in each of the one or more scan windows. Furthermore, step 2204 may be performed by the display processor 127 of FIG. 1. The one or more features within the pixels in each of the one or more scan windows may include at least one of one or more corners or one or more edges. The one or more corners may include one or more category 1 corners and one or more category 2 corners, where the one or more category 1 corners may be associated with a non-rotated horizontal axis and a non-rotated vertical axis, and the one or more category 2 corners may be associated with a 45 degree rotated horizontal axis and a 45 degree rotated vertical axis.

[0097]

[0114] In 2206, the DPU may calculate a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scan windows, as described with respect to the examples of FIG. 1-FIG. 20. For example, as described in 2030 of FIG. 20, the DPU 2002 may calculate a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scan windows. Furthermore, step 2206 may be performed by the display processor 127 of FIG. 1. In some cases, the calculation of the confidence factor for each of the one or more features may correspond to a non-maximum suppression process. The confidence factor for each of the one or more features may correspond to a confidence in the accuracy of each of the one or more features, and the confidence factor may be a value between 0 and 1. Also, the adjusted amount of the combination of the plurality of pixels may be based on the confidence factor for each of the one or more features. Further, the one or more features may include one or more corners and one or more edges, where the confidence factor may be an edge strength metric for each of the one or more edges and a corner strength metric for each of the one or more corners.

[0098]

[0115] In 2208, the DPU may adjust the amount of pixels in each of the one or more scan windows for each of the one or more detected features, as described with respect to the examples of FIG. 1-20. For example, as described in 2040 of FIG. 20, the DPU 2002 may adjust the amount of pixels in each of the one or more scan windows for each of the one or more detected features. Furthermore, step 2208 may be performed by the display processor 127 of FIG. 1. In some aspects, adjusting the amount of pixels in each of the one or more scan windows may adjust the pixel resolution of the one or more scan windows. Also, adjusting the amount of pixels in each of the one or more scan windows may correspond to corner interpolation or edge interpolation. Edge interpolation may be associated with three-way edge filtering, and corner interpolation may be associated with pattern matching based corner filtering.

[0099]

[0116] In 2210, the DPU may combine the adjusted amount of pixels for each of the detected one or more features into multiple output pixels, as described with respect to the examples of Figures 1-20. For example, as described in 2050 of Figure 20, the DPU 2002 may combine the adjusted amount of pixels for each of the detected one or more features into multiple output pixels. Furthermore, step 2210 may be performed by the display processor 127 of Figure 1. The combination of the adjusted amount of pixels for each of the detected one or more features may be a linear combination.

[0100]

[0117] In 2212, the DPU may process each of the plurality of output pixels into at least one output image, as described with respect to the examples of FIGS. 1-20. For example, as described at 2060 in FIG. 20, the DPU 2002 may process each of the plurality of output pixels into at least one output image. Furthermore, step 2212 may be performed by the display processor 127 of FIG. 1. In some cases, the DPU may scan each of the plurality of output pixels, and each of the plurality of output pixels may be scanned in a scan order. For example, processing each of the plurality of output pixels may include scanning each of the plurality of output pixels, and each of the plurality of output pixels may be scanned in a scan order. Additionally, each of the plurality of output pixels may be processed in a display processing unit (DPU) or a DPU scaler.

[0101]

[0118] In 2214, the DPU may send at least one output image to a display or panel after processing each of the plurality of output pixels, as described with respect to the examples of Figures 1-20. For example, as described in 2070 of Figure 20, the DPU 2002 may send at least one output image to a display or panel after processing each of the plurality of output pixels. Furthermore, step 2214 may be performed by the display processor 127 of Figure 1.

[0102]

[0119] In an arrangement, a method or apparatus for display processing is provided. The apparatus may be a DPU, a display processor, or some other processor capable of performing display processing. In an aspect, the apparatus may be a display processor 127 in the device 104, or some other hardware in the device 104 or another device. The apparatus, for example, the display processor 127, may include means for receiving at least one input image for a scaling operation, the input image being associated with one or more scanning windows, each of which includes a plurality of pixels; means for detecting one or more features in the plurality of pixels in each of the one or more scanning windows; means for adjusting the amount of the plurality of pixels in each of the one or more scanning windows for each of the detected one or more features; means for combining the adjusted amount of the plurality of pixels for each of the detected one or more features into a plurality of output pixels; means for processing each of the plurality of output pixels into at least one output image; means for calculating a confidence factor for each of the detected one or more features in the plurality of pixels in each of the one or more scanning windows; and means for transmitting at least one output image to a display or panel after processing each of the plurality of output pixels.

[0103]

[0120] The subject matter described herein may be implemented to achieve one or more benefits or advantages. For example, the display processing techniques described may be used by a DPU, a display processor, or any other processor that may perform display processing to achieve the pattern matching and corner / edge filtering techniques described herein. This may also be achieved at a low cost compared to other display processing techniques. Moreover, the display processing techniques herein may improve or accelerate the processing or execution of data. Furthermore, the display processing techniques herein may improve resource or data utilization and / or resource efficiency. In addition, aspects of the present disclosure may utilize the pattern matching and corner / edge filtering techniques to improve memory bandwidth efficiency and / or increase processing speed in the DPU.

[0104]

[0121] It should be understood that the particular order or hierarchy of the blocks in the disclosed process / flow diagrams is an example of an example approach. It should be understood that the particular order or hierarchy of the blocks in those process / flow diagrams can be rearranged based on design preferences. Further, some blocks can be combined or omitted. The accompanying method claims present elements of the various blocks in an example order, and are not meant to be limited to the particular order or hierarchy presented.

[0105]

[0122] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects set forth herein but are to be accorded the full scope consistent with the claim language, and reference to an element in the singular is not intended to mean "one and only," unless so expressly stated, but rather means "one or more." The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other aspects.

[0106]

[0123] Unless otherwise specified, the term "some" refers to one or more, and the term "or" may be interpreted as "and / or" unless the context dictates otherwise. Combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiple As, multiple Bs, or multiple Cs. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" may be A only, B only, C only, A and B, A and C, B and C, or A and B and C, and any such combination may include one or more elements of A, B, or C. All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or that later become known to those of skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be made public, regardless of whether such disclosure is expressly recited in the claims. Words such as "module," "mechanism," "element," "device," and the like may not be substitutes for the word "means." Therefore, no element of a claim should be construed as a means-plus-function unless the element is expressly recited using the phrase "means for."

[0107]

[0124] In one or more examples, the functions described herein may be realized in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique, or other module described herein is implemented in software, the function, processing unit, technique, or other module described herein may be stored on or transmitted as one or more instructions or code on a computer-readable medium.

[0108]

[0125] In accordance with the present disclosure, the term "or" may be interpreted as "and / or" unless the context dictates otherwise. In addition, phrases such as "one or more" or "at least one" may be used with respect to some features disclosed herein and not with respect to other features, but the features where such words are not used may be interpreted as having such implied meaning unless the context dictates otherwise.

[0109]

[0126] In one or more examples, the functions described herein may be realized in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique, or other module described herein is implemented in software, the function, processing unit, technique, or other module described herein may be stored on or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media may include computer data storage media or communication media, including any medium that facilitates transfer of a computer program from one place to another. In this manner, computer-readable media may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures that execute the techniques described in this disclosure. By way of example and not limitation, such computer readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer readable media. A computer program product may include a computer readable medium.

[0110]

[0127] The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), arithmetic logic units (AGAs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the above structures, or any other structure suitable for carrying out the techniques described herein. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0111]

[0128] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or sets of ICs, e.g., chipsets. In this disclosure, various components, modules or units are described to highlight functional aspects of devices configured to implement the disclosed techniques, but the components, modules or units do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in any hardware unit or may be provided by a collection of interoperable hardware units including one or more processors as described above, together with suitable software and / or firmware. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for executing the techniques described herein. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0112]

[0129] The following aspects are exemplary only and can be combined with other aspects or teachings described herein without limitation.

[0113]

[0130] Aspect 1 is a display processing apparatus including at least one processor coupled to a memory, wherein the one or more processors are configured to receive at least one input image for a scaling operation associated with one or more scanning windows, each of which includes a plurality of pixels, detect one or more features within the plurality of pixels in each of the one or more scanning windows, adjust an amount of the plurality of pixels in each of the one or more scanning windows for each of the detected one or more features, combine the adjusted amount of the plurality of pixels for each of the detected one or more features into a plurality of output pixels, and process each of the plurality of output pixels into at least one output image.

[0114]

[0131] Example 2 is the apparatus of example 1, wherein the at least one processor is further configured to calculate a confidence coefficient for each of the one or more features detected within the plurality of pixels in each of the one or more scanning windows.

[0115]

[0132] Example 3 is the apparatus of example 1 or 2, wherein the calculation of the confidence factor for each of the one or more features corresponds to a non-maximum suppression process.

[0116]

[0133] Aspect 4 is the device of any of aspects 1 to 3, wherein a confidence coefficient for each of the one or more features corresponds to a confidence in the accuracy of each of the one or more features, the confidence coefficient being a value between 0 and 1.

[0117]

[0134] Example 5 is the apparatus of any of Examples 1-4, wherein the adjusted amount of combining the plurality of pixels is based on a confidence factor for each of the one or more features.

[0118]

[0135] Example 6 is the apparatus of any of examples 1 to 5, wherein the one or more features include one or more corners and one or more edges, and the confidence factor is an edge strength metric for each of the one or more edges and a corner strength metric for each of the one or more corners.

[0119]

[0136] Example 7 is the apparatus of any of Examples 1-6, wherein adjusting the amount of pixels in each of the one or more scan windows adjusts a pixel resolution of the one or more scan windows.

[0120]

[0137] Example 8 is the apparatus of any of Examples 1 to 7, wherein adjusting the amount of pixels in each of the one or more scan windows corresponds to corner or edge interpolation.

[0121]

[0138] Example 9 is the apparatus of any of Examples 1 to 8, wherein the edge interpolation is associated with three-directional edge filtering and the corner interpolation is associated with pattern matching based corner filtering.

[0122]

[0139] Example 10 is the apparatus of any of Examples 1 to 9, wherein the one or more features within the plurality of pixels in each of the one or more scan windows include at least one of one or more corners or one or more edges.

[0123]

[0140] Example 11 is the device of any of Examples 1 to 10, wherein the one or more corners include one or more Category 1 corners and one or more Category 2 corners, wherein the one or more Category 1 corners are associated with non-rotated horizontal and non-rotated vertical axes, and the one or more Category 2 corners are associated with a 45 degree rotated horizontal axis and a 45 degree rotated vertical axis.

[0124]

[0141] Example 12 is the apparatus of any of Examples 1 to 11, wherein the combination of the adjusted amounts of the plurality of pixels for each of the detected one or more features is a linear combination.

[0125]

[0142] Example 13 is the apparatus of any of Examples 1 to 12, wherein to process each of the plurality of output pixels, at least one processor is configured to scan each of the plurality of output pixels, and each of the plurality of output pixels is scanned in a scan order.

[0126]

[0143] Example 14 is the device of any of Examples 1 to 13, wherein the at least one processor is further configured to transmit at least one output image to a display or panel after processing each of the plurality of output pixels.

[0127]

[0144] Example 15 is the apparatus of any of Examples 1 to 14, wherein the one or more scan windows are one or more pixel grids.

[0128]

[0145] Example 16 is the apparatus of any of Examples 1 to 15, wherein each of the plurality of output pixels is processed in a display processing unit (DPU) or a DPU scaler.

[0129]

[0146] Example 17 is the apparatus of any of Examples 1 to 16, further comprising at least one of an antenna or a transceiver coupled to the at least one processor.

[0130]

[0147] Aspect 18 is a method of display processing implementing any of aspects 1 to 17.

[0131]

[0148] Aspect 19 is a display processing apparatus including means for performing any of aspects 1 to 17.

[0132]

[0149] Aspect 20 is a computer-readable medium having computer-executable code stored thereon that, when executed by at least one processor, causes the at least one processor to perform any of aspects 1 through 17.

Claims

1. A display processing device, comprising: Memory and at least one processor coupled to the memory; wherein the at least one processor: receiving at least one input image for a scaling operation associated with one or more scan windows, each scan window including a plurality of pixels; Detecting one or more features within the plurality of pixels within each of the one or more scan windows; calculating a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scanning windows, wherein the calculation of the confidence factor for each of the one or more features corresponds to a non-maximum suppression process; adjusting a quantity of the plurality of pixels within each of the one or more scan windows for each of the detected one or more features; combining the adjusted amounts of the pixels for each of the detected one or more features into a plurality of output pixels; an apparatus configured to process each of the plurality of output pixels into at least one output image;

2. The apparatus of claim 1 , wherein the confidence factor for each of the one or more features corresponds to a confidence in the accuracy of each of the one or more features, the confidence factor being a value between 0 and 1.

3. The apparatus of claim 1 , wherein the combining of the adjusted amounts of the plurality of pixels is based on the confidence factor for each of the one or more features.

4. the one or more features include one or more corners and one or more edges, and the confidence factor is an edge strength metric for each of the one or more edges and a corner strength metric for each of the one or more corners; or adjusting the quantity of the plurality of pixels in each of the one or more scan windows adjusts a pixel resolution of the one or more scan windows.

10. The apparatus of claim 1.

5. the adjusting the amount of the pixels in each of the one or more scan windows corresponds to corner interpolation or edge interpolation; The apparatus of claim 1 , wherein the edge interpolation is associated with three-way edge filtering and the corner interpolation is associated with pattern-matching based corner filtering.

6. the one or more features within the pixels in each of the one or more scan windows include at least one of one or more corners or one or more edges; 2. The apparatus of claim 1, wherein the one or more corners include one or more Category 1 corners and one or more Category 2 corners, the one or more Category 1 corners being associated with non-rotated horizontal and vertical axes, and the one or more Category 2 corners being associated with 45 degree rotated horizontal and 45 degree rotated vertical axes.

7. the combination of the adjusted amounts of the pixels for each of the detected one or more features is a linear combination; or 2. The apparatus of claim 1, wherein to process each of the plurality of output pixels, the at least one processor is configured to scan each of the plurality of output pixels, wherein each of the plurality of output pixels is scanned in a scan order.

8. the at least one processor: transmitting the at least one output image to a display or panel after processing each of the plurality of output pixels; The apparatus of claim 1 , further configured to:

9. the one or more scan windows are one or more pixel grids; or 10. The apparatus of claim 1, further comprising at least one of an antenna or a transceiver coupled to the at least one processor, wherein each of the plurality of output pixels is processed in a display processing unit (DPU) or a DPU scaler.

10. 1. A method of display processing, comprising: receiving at least one input image for a scaling operation, the input image being associated with one or more scan windows, each scan window including a plurality of pixels; detecting one or more features within the plurality of pixels within each of the one or more scan windows; calculating a confidence factor for each of the detected one or more features within the plurality of pixels in each of the one or more scanning windows, wherein the calculation of the confidence factor for each of the one or more features corresponds to a non-maximum suppression process. adjusting a quantity of the plurality of pixels in each of the one or more scan windows for each of the one or more detected features; combining the adjusted amounts of the pixels for each of the detected one or more features into a plurality of output pixels; processing each of the plurality of output pixels into at least one output image; A method comprising:

11. the confidence factor for each of the one or more features corresponds to a confidence in the accuracy of each of the one or more features, the confidence factor being a value between 0 and 1; or The method of claim 10 , wherein the combining of the adjusted amounts of the plurality of pixels is based on the confidence factor for each of the one or more features.

12. the one or more features include one or more corners and one or more edges, and the confidence factor is an edge strength metric for each of the one or more edges and a corner strength metric for each of the one or more corners; or The method of claim 10 , wherein adjusting the quantity of the plurality of pixels in each of the one or more scan windows adjusts a pixel resolution of the one or more scan windows.

13. The adjusting of the amount of the plurality of pixels in each of the one or more scan windows corresponds to corner interpolation or edge interpolation, the edge interpolation being associated with three-directional edge filtering, and the corner interpolation being associated with pattern matching based corner filtering; or 11. The method of claim 10, wherein the one or more features within the plurality of pixels in each of the one or more scan windows include at least one of one or more corners or one or more edges, the one or more corners including one or more Category 1 corners and one or more Category 2 corners, the one or more Category 1 corners being associated with un-rotated horizontal and un-rotated vertical axes, and the one or more Category 2 corners being associated with 45 degree rotated horizontal and 45 degree rotated vertical axes.

14. the combination of the adjusted amounts of the plurality of pixels for each of the detected one or more features is a linear combination, and processing each of the plurality of output pixels comprises scanning each of the plurality of output pixels, wherein each of the plurality of output pixels is scanned in a scan order; or or transmitting the at least one output image to a display or panel after processing each of the plurality of output pixels; The method of claim 10 , wherein the one or more scan windows are one or more pixel grids, and each of the plurality of output pixels is processed in a display processing unit (DPU) or a DPU scaler.

15. 15. A non-transitory computer readable medium storing computer executable code for display processing, the code, when executed by a processor, causing the processor to perform the method of any one of claims 10 to 14.

Images