Reducing ar power consumption via bounding sparse content

Abstract

The present disclosure provides systems, devices, apparatuses, and methods, including computer programs encoded on storage media, for reducing AR power consumption via bounding sparse content. A processor obtains, for a frame of graphical content, a first indication of at least one of: the frame, an active pixel region tracked by a graphics processor associated with the frame, tile tracking compression metadata for the frame, or a tile associated with rendering of the frame. The processor computes, based on the first indication, a set of bounding regions of an active pixel set of the frame. The processor configures a workload for at least one of compositing or reprojection on the computed set of bounding regions. The processor outputs a second indication of the configured workload for at least one of compositing or reprojection on the computed set of bounding regions of the active pixel set of the frame.

Description

Cross-reference to related applications

[0001] This application claims the benefit of U.S. Non-Provisional Patent Application Serial No. 18 / 537,461, entitled “REDUCING AR POWER CONSUMPTION VIA BOUNDING SPARSECONTENT”, filed December 12, 2023, which is expressly and entirely incorporated herein by reference. Technical Field

[0002] This disclosure relates generally to processing systems, and more specifically, to one or more techniques for graphics processing. Background Technology

[0003] Computing devices typically perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones (such as smartphones), embedded systems, personal computers, tablet computers, and video game consoles. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output frames. A CPU controls the operation of a GPU by issuing one or more graphics processing commands to it. Modern CPUs are typically capable of executing multiple applications concurrently, each of which may require the use of a GPU during execution. A display processor can be configured to convert digital information received from the CPU into analog values ​​and can issue commands to a display panel to display visual content. Devices that provide content for visual presentation on a display can utilize a CPU, GPU, and / or display processor.

[0004] Current techniques for pixel processing rely on information from graphics processor hardware that is not configured to track and report invalid regions of the rendered content. There is a need for improved techniques to limit pixel processing on invalid regions of the rendered content. Summary of the Invention

[0005] The following is a simplified summary of one or more aspects to provide a basic understanding of these aspects. This summary is not a broad overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0006] In one aspect of this disclosure, a method, computer-readable medium, and apparatus for graphics processing are provided. The apparatus includes: a memory; and a processor coupled to the memory, and configured, based on information stored in the memory, to: obtain, for a frame of graphics content, a first indication of at least one of: the frame, a valid pixel region tracked by a graphics processor associated with the frame, tile-tracking compressed metadata of the frame, or tiles associated with the rendering of the frame; calculate a set of bounded regions of the valid pixel set of the frame based on the first indication; configure a workload for at least one of composition or reprojection on the calculated set of bounded regions of the valid pixel set of the frame; and output a second indication of the configured workload for at least one of composition or reprojection on the calculated set of bounded regions of the valid pixel set of the frame.

[0007] To achieve the foregoing and related objectives, one or more aspects include the features fully described below and specifically pointed out in the claims. The following description and drawings set forth some exemplary features of one or more aspects in detail. However, these features indicate only a few of the various ways in which the principles of the various aspects may be employed, and this description is intended to include all such aspects and their equivalents. Attached Figure Description

[0008] Figure 1 This is a block diagram illustrating an example of a system for generating content based on one or more techniques of this disclosure.

[0009] Figure 2 Example GPUs based on one or more techniques according to this disclosure are illustrated.

[0010] Figure 3 Example images or surfaces are illustrated according to one or more techniques of this disclosure.

[0011] Figure 4 This is an illustration illustrating an aspect of augmented reality (AR) according to one or more technologies of this disclosure.

[0012] Figure 5 These are illustrations illustrating example aspects of AR pixel processing according to one or more techniques of this disclosure.

[0013] Figure 6 This is an illustration of an example of an AR frame according to one or more technologies of this disclosure.

[0014] Figure 7 This is a diagram illustrating an example of an AR layer according to one or more technologies of this disclosure.

[0015] Figure 8This is an illustration of a first strategy for finding a single bounding box of an AR layer according to one or more techniques of this disclosure.

[0016] Figure 9 This is an illustration of a second strategy for finding a single bounding box of an AR layer according to one or more techniques of this disclosure.

[0017] Figure 10 This is an illustration of a third strategy for finding a single bounding box of an AR layer according to one or more techniques of this disclosure.

[0018] Figure 11 This is an illustration of a fourth strategy for finding a single bounding box of an AR layer according to one or more techniques of this disclosure.

[0019] Figure 12 This is an illustration of an example of an AR frame according to one or more technologies of this disclosure.

[0020] Figure 13 This is an illustration of a first strategy for finding multiple bounded zones of an AR frame according to one or more techniques of this disclosure.

[0021] Figure 14 This is an illustration of a second strategy for finding multiple bounded zones of an AR frame according to one or more techniques of this disclosure.

[0022] Figure 15 This is an illustration of a third strategy for finding multiple bounded zones of an AR frame according to one or more techniques of this disclosure.

[0023] Figure 16 This is a call flowchart illustrating example communication between a first effective pixel boundary component and a second effective pixel boundary component according to one or more techniques of this disclosure.

[0024] Figure 17 This is a flowchart of an example method for graphical processing according to one or more techniques of this disclosure.

[0025] Figure 18 This is a flowchart of an example method for graphical processing according to one or more techniques of this disclosure. Detailed Implementation

[0026] Various aspects of the systems, apparatuses, computer program products, and methods will be described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided to make this disclosure comprehensive and complete, and to fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of or in combination with other aspects of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover such apparatuses or methods implemented using structures, functionalities, or structures and functionalities other than or different from the various aspects of the disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.

[0027] Although various aspects are described herein, many variations and substitutions of these aspects fall within the scope of this disclosure. While some potential benefits and advantages of the aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, processing systems, networks, and transmission protocols, some of which are illustrated by way of example in the accompanying drawings and the description below. The detailed description and drawings are merely illustrative and not limiting of this disclosure, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0028] Several aspects are presented with reference to various apparatuses and methods. These apparatuses and methods are described in detail and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0029] For example, an element, any part of an element, or any combination of elements can be implemented as a “processing system” including one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic units, discrete hardware circuits, and other suitable hardware configured to perform the various functionalities described throughout this disclosure. One or more processors in a processing system can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description languages, or other names, software is broadly understood to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc.

[0030] The term "application" can refer to software. As described herein, one or more technologies can refer to an application (e.g., software) configured to perform one or more functions. In such examples, the application may be stored in memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more technologies described herein. As an example, the hardware may access and execute code accessed from memory to perform one or more technologies described herein. In some examples, components are identified in this disclosure. In such examples, a component may be hardware, software, or a combination thereof. Each component may be a separate component or a subcomponent of a single component.

[0031] In one or more examples described herein, the described functionality can be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality can be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media can be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disc storage devices, magnetic disk storage devices, other magnetic storage devices, combinations of computer-readable media of the types described above, or any other medium that can be used to store computer-executable code in the form of instructions or data structures accessible by a computer.

[0032] As used herein, instances of the term "content" may refer to "graphic content," "image," etc., regardless of whether the term is used as an adjective, noun, or other part of speech. In some examples, as used herein, the term "graphic content" may refer to content produced by one or more processes in a graphics processing pipeline. In other examples, as used herein, the term "graphic content" may refer to content produced by a processing unit configured to perform graphics processing. In yet another example, as used herein, the term "graphic content" may refer to content produced by a graphics processing unit.

[0033] Users can wear display devices to experience extended reality (XR) content. XR can refer to technologies that blend digital experiences with various aspects of the real world. XR can include augmented reality (AR), mixed reality (MR), and / or virtual reality (VR). In AR, AR objects can be overlaid on the real-world environment perceived through a display device. In one example, AR content can be experienced through AR glasses that include transparent or translucent surfaces. As the user views the environment through the glasses, AR objects can be projected onto the transparent or translucent surface of the glasses. Generally, AR objects may not exist in the real world, and the user may not interact with them. In MR, MR objects can be overlaid on the real-world environment perceived through a display device, and the user can interact with them. In some aspects, MR objects can include "video perspective" with added virtual content. In one example, the user can "touch" the MR object being displayed to the user (i.e., the user can place their hand in the real world where the MR object appears to be positioned from the user's perspective), and the MR object can "move" based on being touched (i.e., the position of the MR object on the display can change). Typically, MR content can be experienced through MR glasses (similar to AR glasses) worn by the user or through a head-mounted display (HMD) worn by the user. An HMD may include a camera and one or more display panels. The HMD captures images of the environment perceived through the camera and displays an image of the environment to the user, overlaid with MR objects. Unlike the transparent or translucent surfaces of AR / MR glasses, one or more display panels of an HMD may not be transparent or translucent. In VR, users can experience a fully immersive digital environment where the real world is obscured. VR content can be experienced through an HMD.

[0034] AR devices can be designed to be small and / or lightweight so that they are worn on / above the user's head. For example, an AR device could be a head-mounted device. The small and / or lightweight nature of AR devices may result in relatively low battery life. To help conserve battery life, the AR device (or another device) can render AR content in a "sparse" manner to limit the total number of pixels processed. That is, when displayed on a display panel, the rendered AR content may cover a first area, and the display panel of the AR device may be a second area larger than the first area, and the first area may also be smaller than a threshold area of ​​the display panel. In examples, the threshold area could be 50% of the display panel, 25% of the display panel, 10% of the display panel, 5% of the display panel, etc. In a more specific example, the AR device may include AR glasses with a transparent or semi-transparent display surface, and the AR content may be a digital character occupying 10% of the area of ​​the transparent or semi-transparent display surface. In this example, the 10% area of ​​the transparent or semi-transparent display surface may include "effective pixels" (i.e., the effective area) because the effective pixels are displaying the AR content. In contrast, the remaining 90% of a transparent or semi-transparent display surface can include "invalid pixels" (i.e., invalid areas) because invalid pixels are not used to display AR content. When a user of an AR device views valid pixels, the user perceives a digital character, while when a user views invalid pixels, the user perceives their real-world environment. To help conserve battery life for AR devices, the AR device (or another device) may skip certain processing on invalid pixels because such pixels are not used to present AR content to the user. For example, the AR device (or another device) may skip invalid pixels when performing frame compositing, color space conversion, reprojection / distortion, and / or display output. However, the rendering source (e.g., graphics processor hardware) may not be configured to track invalid pixels and report them to hardware configured to perform frame compositing, color space conversion, reprojection / distortion, and / or display output. Therefore, the aforementioned hardware may not be notified to skip which pixels during frame compositing, color space conversion, reprojection / distortion, and / or display output.

[0035] This paper describes various techniques related to reducing augmented reality (AR) power consumption by bounding sparse content. In an example, an apparatus (e.g., a display processor, a graphics processor, a hardware block between the graphics processor and the display processor, etc.) obtains a first indication for a frame of graphical content regarding at least one of the following: the frame, a valid pixel region tracked by the graphics processor associated with the frame, tile-tracked compressed metadata of the frame, or tiles associated with the rendering of the frame. The apparatus calculates a set of bounded regions for the valid pixel set of the frame based on the first indication. The apparatus configures a workload for at least one of compositing or reprojection on the calculated set of bounded regions for the valid pixel set of the frame. The apparatus outputs a second indication of the configured workload for at least one of compositing or reprojection on the calculated set of bounded regions for the valid pixel set of the frame. Instead of calculating the set of bounded regions for the valid pixel set based on the first indication, the apparatus can perform pixel processing on the valid pixel set of the frame and avoid performing pixel processing (e.g., compositing, color conversion, reprojection / distortion, display output, etc.) on invalid pixels of the frame. Therefore, the techniques described above can reduce the power consumption of the device and / or reduce the use of the device's computing resources.

[0036] AR devices may be power-constrained, and therefore, rendered content may be sparse. Power can therefore be saved by skipping rendering (i.e., suppressing rendering) invalid pixels / regions. This paper provides strategies for identifying bounded regions of valid pixels in sparse AR content. Strategies may include tracking pixel output locations (e.g., graphics processing unit (GPU) pixel output locations), calculating row and column sums, using headers / metadata of compressed regions / tiles, performing multiple bounding box calculations using GPU tile output via numbered connected component (blob) detection, identifying multiple regions / tiles using headers / metadata of compressed regions / tiles, and / or identifying multiple tiles using GPU tile output.

[0037] The examples described herein may relate to the use and functionality of a graphics processing unit (GPU). As used herein, a GPU can be any type of graphics processor, and a graphics processor can be any type of processor designed or configured to process graphical content. For example, a graphics processor or GPU can be a dedicated circuit designed to process graphical content. As an additional example, a graphics processor or GPU can be a general-purpose processor configured to process graphical content.

[0038] Figure 1This is a block diagram illustrating an example content generation system 100 configured to implement one or more technologies of this disclosure. The content generation system 100 includes a device 104. Device 104 may include one or more components or circuitry for performing the various functions described herein. In some examples, one or more components of device 104 may be components of a System-on-a-Chip (SOC). Device 104 may include one or more components configured to perform one or more technologies of this disclosure. In the illustrated example, device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, device 104 may include multiple components (e.g., a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131). Display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays, which may include a first display and a second display. The first display may be a left-eye display, and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation on the first and second displays. In other examples, the first and second displays may receive the same frames used for rendering on both displays. In yet another example, the results of graphics processing may not be displayed on the device; for example, the first and second displays may not receive any frames used for rendering on either display. Instead, the frames or graphics processing results may be transferred to another device. In some respects, this is referred to as split rendering.

[0039] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing using graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a processor configured to perform one or more display processing techniques on one or more frames generated by processing unit 120, and then display those frames through one or more displays 131. Although the processor in example content generation system 100 is configured as display processor 127, it should be understood that display processor 127 is one example of a processor and other types of processors, controllers, etc., may be used instead of display processor 127. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise present the frames processed by display processor 127. In some examples, one or more displays 131 may include one or more of the following: liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, projection display device, augmented reality display device, virtual reality display device, head-mounted display, or any other type of display device.

[0040] Memory (such as system memory 124) external to processing unit 120 and content encoder / decoder 122 may be accessible to processing unit 120 and content encoder / decoder 122. For example, processing unit 120 and content encoder / decoder 122 may be configured to read from and / or write to external memory (such as system memory 124). Processing unit 120 may be communicatively coupled to system memory 124 via a bus. In some examples, processing unit 120 and content encoder / decoder 122 may be communicatively coupled to internal memory 121 via the bus or via a different connection.

[0041] Content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store received encoded or decoded graphic content. Content encoder / decoder 122 can be configured to receive encoded or decoded graphic content from system memory 124 and / or communication interface 126, for example, in the form of encoded pixel data. Content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0042] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, static random access memory (SRAM), dynamic random access memory (DRAM), erasable programmable ROM (EPROM), EEPROM, flash memory, magnetic data media or optical storage media, or any other type of memory. According to some examples, internal memory 121 or system memory 124 may be a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be construed as meaning that internal memory 121 or system memory 124 is not removable or that its contents are static. For example, system memory 124 may be removed from device 104 and moved to another device. Alternatively, system memory 124 may not be removable from device 104.

[0043] Processing unit 120 may be a CPU, GPU, GPGPU, or any other processing unit configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In other examples, processing unit 120 may reside on a graphics card mounted in a port on the motherboard of device 104, or may otherwise be incorporated into a peripheral device configured to interoperate with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, arithmetic logic units (ALUs), DSPs, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, processing unit 120 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 121) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors.

[0044] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the content encoder / decoder 122 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 123) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0045] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any of the receiving functions described herein with respect to device 104. Additionally, the receiver 128 may be configured to receive information from another device, such as eye or head positioning information, rendering commands, and / or location information. The transmitter 130 may be configured to perform any of the transmitting functions described herein with respect to device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined to form a transceiver 132. In such an example, the transceiver 132 may be configured to perform any of the receiving and / or transmitting functions described herein with respect to device 104.

[0046] Refer again Figure 1In some aspects, processing unit 120 and / or display processor 127 may include a bounding box generator 198 configured to: for a frame of graphical content, obtain a first indication of at least one of: the frame, a valid pixel region tracked by a graphics processor associated with the frame, tile-tracked compressed metadata of the frame, or tiles associated with the rendering of the frame; calculate a set of bounding regions of the valid pixel set of the frame based on the first indication; configure a workload for at least one of compositing or reprojection on the calculated set of bounding regions of the valid pixel set of the frame; and output a second indication of the configured workload for at least one of compositing or reprojection on the calculated set of bounding regions of the valid pixel set of the frame. Although the following description may focus on graphics processing, the concepts described herein are applicable to other similar processing techniques. Furthermore, although the following description may focus on AR content and AR devices, the concepts presented herein are also applicable to other types of content, such as XR content, MR content, and / or VR content, and other types of devices, such as XR devices, MR devices, and / or VR devices. Additionally, although the following description may focus on wearable display devices (e.g., wearable AR devices), the concepts presented herein may also be applied to non-wearable display devices (e.g., non-wearable AR devices).

[0047] Devices such as device 104 can refer to any device, apparatus, or system configured to perform one or more of the technologies described herein. For example, a device can be a server, base station, user equipment, client device, station, access point, computer (such as a personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer), end product, apparatus, telephone, smartphone, server, video game platform or console, handheld device (such as a portable video game device or personal digital assistant (PDA)), wearable computing device (such as a smartwatch, augmented reality device, or virtual reality device), non-wearable device, display or display device, television, set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the technologies described herein. The processes described herein may be described as being performed by a specific component (e.g., GPU), but in other embodiments, other components (e.g., CPU) consistent with the disclosed embodiments may be used to perform them.

[0048] A GPU can process various types of data or data packets within its pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. Context register packets can be a set of global state information, such as information about global registers, shaders, or constant data, which can adjust how the graphics context will be processed. For example, a context register packet may include information about the color format. In some aspects of a context register packet, there may be one or more bits indicating which workload belongs to the context register. Additionally, multiple functions or programs can run simultaneously and / or in parallel. For example, a function or program may describe an operation, such as a color mode or color format. Therefore, context registers can define various states of the GPU.

[0049] Context states can be used to determine how individual processing units (e.g., vertex extractors (VFDs), vertex shaders (VSs), shader processors, or geometry processors) operate and / or in which mode they operate. To do this, the GPU uses context registers and programming data. In some aspects, the GPU can generate workloads in the pipeline based on the context register definitions of modes or states, such as vertex or pixel workloads. Certain processing units (e.g., VFDs) can use these states to determine certain functions, such as how to aggregate vertices. Because these modes or states can change, the GPU may need to modify the corresponding context. Additionally, the workload corresponding to a mode or state may follow the changed mode or state.

[0050] Figure 2 Example GPU 200 is illustrated according to one or more technologies according to this disclosure. For example... Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-process engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, an L2 cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 includes processing units 220 to 238, but the GPU 200 may include multiple additional processing units. Additionally, processing units 220 to 238 are merely examples, and the GPU may use any combination or order of processing units in accordance with this disclosure. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0051] like Figure 2As shown, the GPU can use a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register groups (e.g., context register group 260) and / or draw call data groups (e.g., draw call group 212). Subsequently, CP 210 can transfer the context register group 260 or the draw call group 212 to a processing unit or block in the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can simultaneously store the following information: the context register of context N, the draw call of context N, the context register of context N+1, and the draw call of context N+1.

[0052] GPUs can render images in a variety of different ways. In some cases, GPUs can render images using direct rendering and / or tiled rendering. In a tiled rendering GPU, an image can be divided or separated into different parts or tiles. After the image is divided, each part or tile can be rendered individually. A tiled rendering GPU can divide a computer graphics image into a grid format, so that each part of the grid (i.e., a tile) is rendered individually. In some aspects of tiled rendering, the image can be divided into different bins or tiles during binning passes. In some aspects, a visibility stream can be constructed during binning passes, where visible primitives or draw calls can be identified. A rendering pass can be performed after a binning pass. In contrast to tiled rendering, direct rendering does not divide a frame into smaller bins or tiles. Instead, in direct rendering, the entire frame is rendered at once (i.e., without binning passes). Additionally, some types of GPUs allow both tiled rendering and direct rendering (e.g., flex rendering).

[0053] In some respects, a GPU can apply the drawing or rendering process to different bins or tiles. For example, a GPU can render a bin and perform all drawing for the primitives or pixels within that bin. During the bin-based rendering process, the rendering target can be located in GPU Internal Memory (GMEM). In some instances, after rendering a bin, the contents of the rendering target can be moved to system memory, and GMEM can be freed to render the next bin. Additionally, a GPU can render another bin and perform drawing for the primitives or pixels within that bin. Thus, in some respects, there may be a small number of bins covering all the drawing on a surface, for example, four bins. Furthermore, a GPU can loop through all the drawing in a bin but perform drawing only for visible drawing calls, i.e., drawing calls that include visible geometry. In some respects, a visibility stream can be generated, for example, in binning passes, to determine the visibility information of each primitive in an image or scene. For example, such a visibility stream can identify whether a primitive is visible. In some respects, this information can be used to remove invisible primitives, such that, for example, invisible primitives are not rendered in a rendering pass. Additionally, at least some primitives that are marked as visible can be rendered in the rendering pass.

[0054] In some aspects of tile rendering, there can be multiple processing stages or passes. For example, rendering can be performed in two passes, such as a binning, visibility, or box visibility pass and a rendering or box rendering pass. During a visibility pass, the GPU can input a rendering workload, record the positions of primitives or triangles, and then determine which primitives or triangles fall into which bins or regions. In some aspects of a visibility pass, the GPU can also identify or mark the visibility of each primitive or triangle in the visibility stream. During a rendering pass, the GPU can input a visibility stream and process one bin or region at a time. In some aspects, the visibility stream can be analyzed to determine which primitives or primitive vertices are visible or invisible. Thus, visible primitives or primitive vertices can be processed. By doing so, the GPU can reduce the unnecessary workload of processing or rendering invisible primitives or triangles.

[0055] In some aspects, certain types of primitive geometry, such as localized geometry, can be processed during visibility passes. Additionally, primitives can be categorized into different bins or regions based on their localization or position. In some instances, categorizing primitives or triangles into different bins can be performed by determining visibility information for those primitives or triangles. For example, the GPU can determine the visibility information for each primitive in each bin or region or write it to, for example, system memory. This visibility information can be used to determine or generate a visibility stream. In a rendering pass, the primitives in each bin can be rendered individually. In these cases, the visibility stream can be retrieved from memory and used to remove primitives that are not visible to that bin.

[0056] Some aspects of a GPU or GPU architecture can provide multiple different options for rendering (e.g., software rendering and hardware rendering). In software rendering, the driver or CPU can process each view... Figure 1 The entire frame geometry is copied each time. Additionally, some different states can change depending on the viewpoint. Therefore, in software rendering, the software can copy the entire workload by changing some states that can be used for rendering for each viewpoint in the image. In some respects, this can lead to increased overhead because the GPU may submit the same workload multiple times for each viewpoint in the image. In hardware rendering, the hardware or GPU may be responsible for copying or processing the geometry for each viewpoint in the image. Therefore, the hardware can manage the copying or processing of primitives or triangles for each viewpoint in the image.

[0057] Figure 3 An image or surface 300 according to one or more techniques of this disclosure is illustrated, including multiple elements divided into multiple boxes. For example... Figure 3 As shown, the image or surface 300 includes a region 302, which includes primitives 321, 322, 323, and 324. Primitives 321, 322, 323, and 324 are divided or placed into different bins, such as bins 310, 311, 312, 313, 314, and 315. Figure 3 This example illustrates tile rendering using multiple viewpoints for primitives 321-324. For instance, primitives 321-324 are in a first viewpoint 350 and a second viewpoint 351. Therefore, GPU processing or rendering of an image or surface 300 including region 302 can utilize multi-view or multi-view rendering.

[0058] As indicated in this article, GPUs or graphics processors can use tile rendering architectures to reduce power consumption or save memory bandwidth. As further stated above, this rendering method divides the scene into multiple bins, along with visibility paths that identify the visible triangles within each bin. Therefore, in tile rendering, the entire screen can be divided into multiple bins or tiles. The scene can then be rendered multiple times, for example, once or multiple times for each bin.

[0059] In various aspects of graphics rendering, some graphics applications may render a single target (i.e., the rendering target) once or multiple times. For example, in graphics rendering, the frame buffer on system memory can be updated multiple times. The frame buffer can be part of memory or random access memory (RAM) (e.g., containing bitmaps or storage devices) to help store display data for the GPU. The frame buffer can also be a memory buffer containing a complete frame of data. Additionally, the frame buffer can be a logical buffer. In some aspects, updating the frame buffer can be performed in bin or tile rendering, where, as discussed above, the surface is divided into multiple bins or tiles, and each bin or tile can then be rendered individually. Furthermore, in tile rendering, the frame buffer can be divided into multiple bins or tiles.

[0060] As this article points out, in some aspects, such as in bin or tile rendering architectures, frame buffers allow data to be repeatedly stored or written to them, for example, when rendering from different types of memory. This can be referred to as unresolving the frame buffers or system memory. For example, when storing or writing to one frame buffer and then switching to another, the data or information on the frame buffer can be resolved from the GMEM at the GPU to system memory, i.e., memory in dual data rate (DDR) RAM or dynamic RAM (DRAM).

[0061] In some respects, system memory can also be system-on-chip (SoC) memory or another chip-based memory, such as on a device or smartphone, used for storing data or information. System memory can also be a physical data storage device shared by the CPU and / or GPU. In some respects, system memory can be, for example, a DRAM chip on a device or smartphone. Therefore, SoC memory can be a chip-based method for storing data.

[0062] In some respects, GMEM can be on-chip memory at the GPU, which can be implemented using static RAM (SRAM). Alternatively, GMEM can be stored on the device (e.g., a smartphone). As indicated herein, data or information can be transferred between system memory or DRAM and GMEM, for example, at the device. In some respects, system memory or DRAM can reside at the CPU or GPU. Furthermore, data can be stored in DDR or DRAM. In some respects, such as in bin or tile rendering, a small portion of the memory can be stored at the GPU, for example, in GMEM. In some cases, storing data at GMEM may utilize a larger processing workload and / or consume more power compared to storing data at the frame buffer or system memory.

[0063] Figure 4Figure 400 illustrates an example 401 of an augmented reality (AR) aspect according to one or more techniques of this disclosure. An AR device 402 is wearable on / above / near the head of a user 404. In the example, the AR device 402 may be or include device 104. In the example, the AR device 402 may be lightweight and / or may have a small form factor to make the AR device 402 comfortable to wear for the user 404; however, the lightweight and / or small form factor of the AR device 402 may result in the AR device 402 being power-constrained (i.e., having limited battery life).

[0064] AR device 402 may include a left display 406 and a right display 408. When AR device 402 is worn on / above / near a user's head, the left display 406 may be positioned within a few centimeters of the user's left eye, and the right display 408 may be positioned within a few centimeters of the user's right eye. Therefore, the user's left eye's left gaze may point to the left display 406, and the user's right eye's right gaze may point to the right display 408. In the example, the left display 406 and right display 408 may be made of a transparent display surface or a translucent display surface (such as glass). In the example, the left display 406 and right display 408 may be included in display 131. In one aspect ( Figure 4 (Not illustrated) AR device 402 may include a single display having a left area corresponding to the left display 406 and a right area corresponding to the right display 408.

[0065] AR device 402 may include a left camera 410 corresponding to the left eye of user 404 and a right camera 412 corresponding to the right eye of user 404. Specifically, the lens of the left camera 410 may point in a direction similar to the gaze direction of the user 404's left eye, and the lens of the right camera 412 may point in a direction similar to the gaze direction of the user 404's right eye. In this example, the left camera 410 and right camera 412 may be video cameras. The left camera 410 and right camera 412 enable AR device 402 to perceive the environment of user 404. In one aspect, the left display 406 and right display 408 may include opaque display surfaces, and thus, the left camera 410 and right camera 412 may capture images of the user 404's real-world environment, which can then be displayed on the left display 406 and right display 408, respectively.

[0066] In the example, when user 404 wears AR device 402 in an environment and when an AR application is executed on AR device 402, AR device 402 can present AR view 414 to user 404 via left display 406 and right display 408. In this example, AR view 414 may include a tree 416, where tree 416 may be a real-world object in user 404's environment as pointed to by user 404's eye. AR view 414 may also include first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422, where first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422 may be non-real-world content generated by an AR application executed on AR device 402. In other words, tree 416 may correspond to a real-world view of the user's environment, while first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422 may be AR content overlaid on the real-world view.

[0067] As used herein, the term "sparse AR content" can refer to AR content that occupies less than a threshold area (i.e., a percentage of the threshold area) of the display panel of an AR device when displayed on the display panel of the AR device. Sparse XR content can include sparse AR content. In the examples, AR content can be considered sparse AR content when it occupies less than 50% of the display panel, less than 25% of the display panel, less than 10% of the display panel, less than 5% of the display panel, etc. In one aspect, AR content can be considered sparse AR content when the sum of the areas of each instance of AR content is less than the threshold area. In another aspect, instances of AR content can be considered sparse AR content even if the total area occupied by all instances of AR content is greater than the threshold area, when the area of ​​each instance of AR content is less than the threshold area. Sparse AR content can limit the total number of pixels processed by AR device 402.

[0068] Figure 5This is a diagram 500 illustrating example aspects of AR pixel processing 502 according to one or more techniques of this disclosure. An AR device (e.g., AR device 402) may include graphics processor hardware 504 (e.g., GPU hardware) and AR frame compositing and display hardware 506. At 508, the graphics processor hardware 504 may be configured to render a frame. A frame may refer to an image as part of a series of images displayed sequentially. Rendering a frame at 508 generates pixel data 510 associated with the frame. At 512, the AR frame compositing and display hardware 506 may be configured to process the frame (i.e., process the pixel data 510). For example, the AR frame compositing and display hardware 506 may perform frame compositing, color space conversion, reprojection / distortion, segmentation, and display output on the pixel data 510. Frame compositing (which may also be referred to as "compositing") may refer to arranging different layers to generate a frame that can be displayed. Reprojection (which may also be referred to as "distortion") may refer to adjusting a frame (or a portion thereof) to account for changes in the user's head posture between rendering and display times. Color space conversion may refer to converting an image from a first color space to a second color space. Segmentation refers to the process of dividing a frame into multiple image segments. Workloads for compositing, reprojection, color space conversion, segmentation, etc., refer to a series of instructions used to perform these operations.

[0069] Subsequently, the AR device can present an AR frame 514 on a display panel, wherein the AR frame 514 is based on (processed) pixel data 510. The AR frame 514 may include valid pixels 516 corresponding to AR content (e.g., first sparse AR content 418, second sparse AR content 420, third sparse AR content 422) and invalid pixels 518 corresponding to non-AR content (e.g., the real world). As used herein, the term "valid pixel" may refer to a pixel (or area of ​​a frame corresponding to a pixel) on the display panel that outputs light corresponding to XR content (e.g., AR content, MR content). The term "valid area" may refer to an area on the display panel having valid pixels. As used herein, the term "invalid pixel" may refer to a pixel (or area of ​​a frame corresponding to a pixel) on the display panel that does not output light corresponding to XR content (e.g., AR content, MR content). The term "invalid area" may refer to an area on the display panel having invalid pixels. In other words, invalid pixels may correspond to areas on the display panel where AR content is not displayed. Therefore, users of AR devices can perceive the real world in areas of the display panel that include invalid pixels.

[0070] In one example, the concepts presented in this paper can be applied to MR content. For instance, in MR content, the "background" could be a full-screen camera perspective layer, and the content overlaid on the background could be sparse. Devices (e.g., device 104) can save power by finding pixel bounding boxes on the sparse overlay, which reduces the total number of pixels processed in the final composition.

[0071] As indicated above, AR devices may have limited battery life. AR devices can use different strategies to conserve battery power. In a first strategy, the AR device may render fewer pixels (i.e., generate sparse AR content), which reduces the AR device's power consumption. For example, graphics processor hardware 504 may render sparse AR content. In a second strategy, the AR device may skip pixel processing on invalid pixels (i.e., invalid regions) during frame compositing, color space conversion, reprojection / distortion, and / or display output. For example, AR frame compositing and display hardware 506 may skip frame compositing, color space conversion, reprojection / distortion, and / or display output on invalid pixels. However, graphics processor hardware 504 may not be configured to track valid / invalid regions (i.e., valid / invalid pixels) of the rendered content and report them to AR frame compositing and display hardware 506. Therefore, AR frame compositing and display hardware 506 may not have information about which pixels will have their pixel processing skipped.

[0072] Figure 6 Figure 600 illustrates an example of an AR frame 602 according to one or more techniques of this disclosure. This document describes various strategies for efficiently finding / determining / computing bounded regions of valid pixels in sparse AR content. A bounded region may encompass (i.e., enclose) the AR content (i.e., valid pixels). As used herein, a bounded region may include a bounding box or a bounded area. A bounding box may refer to a rectangular box that encompasses the AR content (e.g., an AR object). A bounded area may refer to a non-rectangular box that encompasses the AR content (e.g., an AR object). In the example, a bounded area may include three or more sides. The AR device may perform pixel processing (e.g., frame composition, color space conversion, reprojection / distortion, and / or display output) on pixels encompassed by the bounded region, while the AR device may skip performing pixel processing on pixels not encompassed by the bounded region. Therefore, the techniques described herein can reduce the computational and / or power burden on AR devices.

[0073] In the example, using the techniques described herein, an AR device (e.g., AR device 402) can generate an AR frame 602 and effective pixel bounding boxes, wherein each effective pixel bounding box can include an instance of AR content (or more than one instance of AR content). In the example, the AR device can generate a first effective pixel bounding box 604 covering a first sparse AR content 418 and a second sparse AR content 420. In the example, the AR device can generate the first effective pixel bounding box 604 based on the fact that the first sparse AR content 418 and the second sparse AR content 420 are within a threshold distance of each other. The AR device can also generate a second effective pixel bounding box 606 covering a third sparse AR content 422. The AR device can perform pixel processing (e.g., frame composition, color space conversion, reprojection / distortion, and / or display output) on the pixels covered by the first effective pixel bounding box 604 and the second effective pixel bounding box 606. AR devices can skip performing pixel processing (e.g., frame composition, color space conversion, reprojection / distortion, and / or display output) on invalid pixels 608 associated with AR frame 602. That is, AR devices can skip performing pixel processing on pixels not covered by the first valid pixel bounding box 604 or the second valid pixel bounding box 606.

[0074] Figure 7 Figure 700 illustrates an example of an AR layer 702 according to one or more techniques of this disclosure. A device (e.g., AR device 402) may render AR in several separate sparse layers. A layer may refer to an element of an image that, when composited with other layers, forms a complete image. After rendering, the AR device may composite each layer into a final display image presented on a display panel. In the example, using the techniques described herein, the AR device may determine / find / compute a single bounded region (e.g., bounding box, bounded area) for each layer. In another example, using the techniques described herein, the AR device may determine / find / compute more than one bounded region (e.g., bounding box, bounded area) for each layer. Some aspects presented herein relate to a set of strategies for finding a single bounded region (e.g., bounding box, bounded area) of an AR layer. Finding a single bounded region may be computationally more efficient than calculating multiple bounded regions in general.

[0075] In the example, a device (e.g., AR device 402) may generate a first AR layer 704, a second AR layer 706, and a third AR layer 708, wherein the first AR layer 704 includes first sparse AR content 418, the second AR layer 706 includes second sparse AR content 420, and the third AR layer 708 includes third sparse AR content 422. Using the techniques described herein, the device (e.g., AR device 402) may find / determine / compute a first effective pixel bounding box 710 covering the first sparse AR content 418, a second effective pixel bounding box 712 covering the second sparse AR content 420, and a third effective pixel bounding box 714 covering the third sparse AR content 422.

[0076] Figure 8 Figure 800 illustrates a first strategy 802 for finding a single bounding box of an AR layer according to one or more techniques of this disclosure. The first strategy 802 may be associated with tracking GPU pixel output locations to find the bounding region. In the first strategy 802, when an output frame or AR layer is rendered at 806, the graphics processor hardware 804 of a device (e.g., AR device 402) may track pixel output locations written for that frame or AR layer (e.g., for the first AR layer 704). Pixel output locations may include a minimum X coordinate 808 (which may also be referred to as “Min X”), a maximum X coordinate 810 (which may also be referred to as “Max X”), a minimum Y coordinate 812 (which may also be referred to as “Min Y”), and a maximum Y coordinate 814 (which may also be referred to as “Max Y”). The minimum X coordinate 808 (i.e., the minimum horizontal coordinate) may refer to the lowest horizontal pixel coordinate in which a valid pixel can be found in the first AR layer 704. The maximum X coordinate 810 (i.e., the maximum horizontal coordinate) may refer to the highest horizontal pixel coordinate in which a valid pixel can be found in the first AR layer 704. The minimum Y-coordinate 812 (i.e., the minimum vertical coordinate) can refer to the lowest vertical pixel coordinate where a valid pixel can be found in the first AR layer 704. The maximum Y-coordinate 814 (i.e., the maximum vertical coordinate) can refer to the highest vertical pixel coordinate where a valid pixel can be found in the first AR layer 704. The bounding box 816 can be defined as having four corners according to the following equation (I).

[0077] (I)

[0078] In one aspect, graphics processor hardware 804 can generate bounding box 816 according to equation (I). In another aspect, graphics processor software and / or graphics processor firmware can generate bounding box 816 according to equation (I). In yet another aspect, display processor hardware, display processor software, and / or display processor firmware can generate bounding box 816 according to equation (I). In yet another aspect, a hardware block between graphics processor hardware and display processor hardware can generate bounding box 816 according to equation (I).

[0079] Figure 9 This is a diagram 900 illustrating a second strategy 902 for finding a single bounding box of an AR layer according to one or more techniques of this disclosure. The second strategy 902 may be associated with calculating row sums and column sums to find the bounding area. In the second strategy 902, prior to hardware synthesis, display processor software 904, executed by a display processor (e.g., display processor 127 of device 104), may calculate the pixel sums for each row and each column of an image (e.g., the second AR layer 706). In this example, for each row in the image, the display processor software 904 may calculate the sum of the values ​​of each pixel in the row (i.e., calculate the row pixel sum 906). A pixel may refer to the smallest addressable element in the image. In this example, for each column in the image, the display processor software 904 may calculate the sum of the values ​​of each pixel in the column (i.e., calculate the column pixel sum 908). In one aspect, the display processor software 904 may calculate the row pixel sum 906 and the column pixel sum 908 based on a histogram of the image.

[0080] In the example, invalid pixels in the image may have zero values. Therefore, the zero sum of a row or column can indicate invalid pixels in that row or column, and the non-zero sum of a row or column can indicate (i.e., imply) the presence of valid pixels in that row or column. Thus, the display processor software 904 can read row pixels and 906 (e.g., from top to bottom of the image) to determine the first row 910 and the last row 912 of the image with non-zero sums. Similarly, the display processor software can read column pixels and 908 (e.g., from left to right of the image) to determine the first column 914 and the last column 916 of the image with non-zero sums. The display processor software 904 can calculate a bounding box 918 with corners based on the first row 910, the last row 912, the first column 914, and the last column 916. In the example, the bounding box 918 may encompass the second sparse AR content 420. More specifically, the display processor software 904 can calculate the bounding box 918 according to the following equation (II).

[0081] (II)

[0082] Although the second strategy 902 is described above as being executed / implemented by the display processor software 904, other possibilities are conceivable. For example, hardware block 920 may execute / implement the second strategy 902 as described above. In the example, hardware block 920 may be a color space conversion (CSC) hardware block.

[0083] Figure 10This is a diagram 1000 illustrating a third strategy 1002 for finding a single bounding box of an AR layer according to one or more techniques of this disclosure. The third strategy 1002 may be associated with using compressed header data to indicate the bounding area. When transmitting graphics-associated data between different hardware components of a device (e.g., device 104), the graphics-associated data (e.g., the first AR layer 704) may be compressed to reduce bandwidth requirements on the device. For example, a graphics processor may render the first AR layer 704, compress the first AR layer 704, and send the (compressed) first AR layer 704 to a display processor. The display processor may receive the (compressed) first AR layer 704, decompress the (compressed) first AR layer 704, and perform further processing on the (decompressed) first AR layer 704.

[0084] In the example, a device (e.g., a graphics processor) that compresses / decompresses data according to some compression / decompression format may subdivide a frame (e.g., a first AR layer 704) into fixed-size compressed tiles 1004. The compressed tiles can be rectangular subdivisions of frames or layers. In this example, the device may maintain compressed tile metadata 1006, including a buffer header 1008, which indicates which tiles in the compressed tiles 1004 contain data (i.e., which tiles are "non-empty"). The compressed tile metadata (which may also be referred to as tile-tracking compressed metadata) may refer to data about tiles used for compression purposes. The buffer header may refer to a portion of the compressed tile metadata that includes data about tiles used for compression purposes.

[0085] In the third strategy 1002, the display processor (e.g., display processor 127) may receive compressed tile metadata 1006 from the graphics processor (e.g., GPU 200). The display processor may search the compressed tile metadata 1006 (e.g., the buffer header 1008 of the compressed tile metadata 1006) to determine tiles in the compressed tile metadata 1006 that includes data; that is, the display processor may search the compressed tile metadata 1004 to determine valid pixels (i.e., valid tiles) in the first AR layer 704. In one example, the display processor may compute (e.g., derive) a bounding box 1010 based on the search results, wherein the bounding box 1010 covers the first sparse AR content 418. In another example, the display processor may compute (e.g., derive) a bounding region 1012 based on the search results, wherein the bounding region 1012 covers the first sparse AR content 418.

[0086] Although the third strategy 1002 is described above as being executed / implemented by the display processor, other possibilities are conceivable. For example, a hardware block between the graphics processor and the display processor may execute / implement the third strategy 1002 as described above.

[0087] Figure 11 Figure 1100 illustrates a fourth strategy 1102 for finding a single bounding box of an AR layer according to one or more techniques of this disclosure. The fourth strategy 1102 may be associated with using GPU tile output to suggest bounding regions. A graphics processor (e.g., GPU 200) may subdivide a frame (e.g., a first AR layer 704) into fixed-size graphics processor tiles 1104 during the rendering process. Graphics processor tiles 1104 may be referred to as "render-associated tiles". Render-associated tiles may be rectangular subdivisions of frames or layers.

[0088] In the fourth strategy 1102, at 1108, the graphics processor hardware 1106 can track tiles (i.e., valid tiles) with valid pixels in the graphics processor tile 1104 during frame rendering. In one example, the graphics processor hardware 1106 can compute (e.g., derive) a bounding box 1110 based on the valid tiles, where the bounding box 1110 covers the first sparse AR content 418. In another example, the graphics processor hardware 1106 can compute (e.g., derive) a bounding region 1112 based on the valid tiles, where the bounding region 1112 covers the first sparse AR content 418.

[0089] Although the fourth strategy 1102 is described above as being executed / implemented by the graphics processor hardware 1106, other possibilities are conceivable. For example, the graphics processor software 1114 or graphics processor firmware 1116 may execute / implement the fourth strategy 1102 as described above. Graphics processor hardware may refer to the physical components of a device used for graphics processing. Graphics processor software may refer to computer-readable instructions used for graphics processing. Graphics processor firmware may refer to permanent software for graphics processing programmed into read-only memory.

[0090] Figure 12Figure 1200 illustrates an example of an AR frame 1202 according to one or more techniques of the present disclosure. In some examples, the AR content may be sparse, but relatively "expanded" within the frame, meaning that the distance between different instances of the AR content in the frame may exceed a threshold distance. For example, AR frame 1202 may include expanded sparse AR content 1204. Expanded sparse AR content 1204 may include first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422, wherein the distance between each of the first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422 may exceed a threshold distance. In other words, a relatively large number of invalid pixels 1206 may exist between each of the first sparse AR content 418, second sparse AR content 420, and third sparse AR content 422. In the case of unfolded sparse AR content 1204, computing multiple bounded regions (e.g., multiple bounding boxes and / or multiple bounded areas) may be computationally more complex than computing a single bounded region; however, multiple bounded regions can provide power savings for the device (e.g., device 104). Various strategies for computing multiple bounded regions are described below.

[0091] Figure 13 This is a diagram 1300 illustrating a first strategy 1302 for finding multiple bounded regions of AR frame 1304 according to one or more techniques of this disclosure. The first strategy 1302 may be associated with a multi-boundary box calculation detected via numbered connected regions. In the first strategy 1302, a device (e.g., graphics processor hardware, graphics processor software, graphics processor firmware, display processor hardware, display processor software, display processor firmware, a hardware block between the graphics processor and the display processor) may define a connected region as a set of connected pixels, i.e., each pixel in a connected region (i.e., a connected region pixel) is within N pixels of another connected region, where N is a positive integer. The device may associate each valid pixel in pixel 1306 of AR frame 1304 with a connected region. The device may calculate the total number of connected regions (i.e., independent connected regions). The device may compute / calculate the bounded region (e.g., bounding box, bounded region) of each connected region based on the (associated) valid pixels.

[0092] More specifically, in the first step of the first strategy 1302, the device may find multiple independent connected regions and assign valid pixels in AR frame 1304 to specific connected regions via an algorithm configured to determine connected groups of pixels in a sparse image. In the example, the device may determine via the algorithm that AR frame 1304 includes a first connected region 1308 (corresponding to the first sparse AR content 418) and a second connected region 1310 (corresponding to the third sparse AR content 422). The device may assign a first valid pixel (i.e., first connected region pixel 1312) in AR frame 1304 to the first connected region 1308, and the device may assign a second valid pixel (i.e., second connected region pixel 1314) in AR frame 1304 to the second connected region 1310.

[0093] The algorithm described above (i.e., the connected component detection algorithm) will now be described. The device can read each pixel in AR frame 1304 in a right-to-left, top-to-bottom order. When the device reads a valid pixel (e.g., when the device finds a valid pixel), the device can assign the valid pixel to a connected component based on connected component assignment of pixels near the valid pixel. For example, the device can determine whether (1) a connected component assigned pixel (i.e., a pixel already assigned to a connected component) exists above the valid pixel, or (2) a connected component assigned pixel exists to the left of the valid pixel. If a connected component assigned pixel exists above or to the left of the valid pixel, the device can assign the valid pixel to the connected component of the connected component assigned pixel. If no connected component assigned pixel exists above or to the left of the valid pixel, the device can assign the valid pixel to a new connected component. The device can process all pixels of AR frame 1304 in this manner. After processing all pixels of AR frame 1304, the device can determine the total number of connected components in AR frame 1304, and the device can assign valid pixels to connected components.

[0094] In the second step of the first strategy 1302, the device may compute / calculate the bounding box or bounding region (i.e., the connected region bounding box or the connected region bounding region) for each connected region. For example, the device may compute a first bounding box 1316 for a first connected region 1308 and a second bounding box 1318 for a second connected region 1310, wherein the first bounding box 1316 may cover a first sparse AR content 418, and wherein the second bounding box 1318 may cover a third sparse AR content 422. In the example, for each connected region, the device may read the position of each pixel in the connected region to determine the minimum X coordinate (referred to as MinX), the maximum X coordinate (referred to as MaxX), the minimum Y coordinate (referred to as MinY), and the maximum Y coordinate (referred to as MaxY). The device may compute (i.e., define) the connected region bounding region of each connected region as a box having corners at the minimum X coordinate, maximum X coordinate, minimum Y coordinate, and maximum Y coordinate according to the following equation (III).

[0095] (III)

[0096] Figure 14 This is a diagram 1400 illustrating a second strategy 1402 for finding multiple boundary zones of AR frame 1404 according to one or more techniques of this disclosure. The second strategy 1402 may be associated with using compressed header data to indicate boundary zones. When transmitting graphics-associated data between different hardware components of a device (e.g., device 104), the graphics-associated data (e.g., AR frame 1404) may be compressed to reduce bandwidth requirements on the device. For example, a graphics processor may render AR frame 1404, compress AR frame 1404, and send the (compressed) AR frame 1404 to a display processor. The display processor may receive the (compressed) AR frame 1404, decompress the (compressed) AR frame 1404, and perform further processing on the (decompressed) AR frame 1404.

[0097] In this example, a device (e.g., a graphics processor) that compresses / decompresses data according to some compression / decompression format may subdivide AR frame 1404 into fixed-size compressed tiles 1406. In this example, the device may maintain compressed tile metadata 1408 including a buffer header 1410, where the buffer header 1410 indicates which tiles in the compressed tiles 1406 contain data.

[0098] In the second strategy 1402, the display processor (e.g., display processor 127) may receive compressed tile metadata 1408 from the graphics processor (e.g., GPU 200). The display processor may search the compressed tile metadata 1408 (e.g., the buffer header 1410 of the compressed tile metadata 1408) to determine tiles in the compressed tile 1406 that includes data; that is, the display processor may search the compressed tile metadata 1408 to determine valid pixels (i.e., valid tiles) in the AR frame 1404. In one example, the display processor may compute (e.g., derive) a bounding box 1412 based on the search results, where the bounding box 1412 covers the third sparse AR content 422. In another example, the display processor may compute (e.g., derive) a bounded region 1414 based on the search results, where the bounded region 1414 covers the first sparse AR content 418. In the examples, the search may be a modified version of the connected component detection algorithm described above, which uses compressed tiles 1406 instead of pixels to find connected components. The modified connected component detection algorithm can be executed in less time compared to the time taken by the connected component detection algorithm described above; however, the bounding regions and / or bounding boxes calculated by the modified connected component detection algorithm may be coarser than those calculated by the connected component detection algorithm described above.

[0099] Although the second strategy 1402 is described above as being executed / implemented by the display processor, other possibilities are conceivable. For example, a hardware block between the graphics processor and the display processor may execute / implement the second strategy 1402 as described above.

[0100] Figure 15 This is a diagram 1500 illustrating a third strategy 1502 for finding multiple bounded regions of AR frame 1504 according to one or more techniques of this disclosure. The third strategy 1502 may be associated with using GPU tile output to suggest bounded regions. A graphics processor (e.g., GPU 200) may subdivide the frame (e.g., AR frame 1504) into fixed-size graphics processor tiles 1506 during the rendering process.

[0101] In the third strategy 1502, graphics processor hardware 1508 (e.g., graphics processor hardware 1106) at 1509 can track tiles (i.e., valid tiles) with valid pixels in graphics processor tiles 1506 during frame rendering. In the example, graphics processor hardware 1508 can track tiles with valid pixels via a modified version of the connected component detection algorithm described above, which uses graphics processor tiles 1506 instead of pixels to find connected components. The modified connected component detection algorithm can be executed in less time compared to the time spent by the connected component detection algorithm described above; however, the bounding regions and / or bounding boxes calculated by the modified connected component detection algorithm may be coarser compared to those calculated by the connected component detection algorithm described above. In one example, graphics processor hardware 1508 can calculate (e.g., derive) bounding boxes 1510 based on valid tiles, where bounding boxes 1510 encompass the third sparse AR content 422. In another example, graphics processor hardware 1508 may compute (e.g., derive) a bounded region 1512 based on valid tiles, wherein the bounded region 1512 covers the first sparse AR content 418.

[0102] Although the third strategy 1502 is described above as being executed / implemented by the graphics processor hardware 1508, other possibilities are conceivable. For example, the graphics processor software 1514 (e.g., graphics processor software 1114) or the graphics processor firmware 1516 (e.g., graphics processor firmware 1116) may execute / implement the third strategy 1502 as described above.

[0103] The techniques described above can be associated with a variety of advantages. For example, for sparse AR content, the techniques described above can efficiently compute the bounded regions (e.g., bounding boxes or bounded areas) of effective pixels (i.e., effective AR content), which can reduce the power consumption of AR devices. Furthermore, since some AR devices can leverage the sparsity of AR content, and since the processing power used to process AR content can be proportional to the number of pixels processed, the techniques described above can reduce the processing resources (e.g., clock cycles, memory, etc.) used by AR devices. The techniques described above can avoid and / or mitigate pixel processing on invalid pixels during processing steps that occur after the AR content has been rendered by the graphics processor, such as compositing, color space conversion, reprojection / distortion, and / or display output. Therefore, the techniques described above can reduce the overall power consumption of AR devices.

[0104] Figure 16This is a call flowchart 1600 illustrating example communication between a first effective pixel boundary component 1602 and a second effective pixel boundary component 1603 according to one or more technologies of this disclosure. In the example, the first effective pixel boundary component 1602 and / or the second effective pixel boundary component 1603 may be included in device 104 or AR device 402. For example, the first effective pixel boundary component 1602 and / or the second effective pixel boundary component 1603 may be, or may be included in, display processor 127 and / or processing unit 120. In the example, the first effective pixel boundary component 1602 may be or include first display processor hardware, first display processor software, first display processor firmware, a first graphics processor and a first hardware block between the first display processor, first graphics processor hardware, first graphics processor firmware and / or first graphics processor software. In the example, the second effective pixel boundary component 1603 may be or include second display processor hardware, second display processor software, second display processor firmware, a second hardware block between the first graphics processor and the first display processor, second graphics processor hardware, second graphics processor firmware and / or second graphics processor software.

[0105] At 1604, the first effective pixel bounding component 1602 may obtain a first indication for at least one of the following for a frame of graphical content: the frame, an effective pixel region tracked by a graphics processor associated with the frame, tile tracking compressed metadata of the frame, or tiles associated with the rendering of the frame. At 1618, the first effective pixel bounding component 1602 may calculate a set of bounded regions of the effective pixel set of the frame based on the first indication. At 1620, the first effective pixel bounding component 1602 may configure a workload for at least one of composition or reprojection on the calculated set of bounded regions of the effective pixel set of the frame. At 1622, the first effective pixel bounding component 1602 may (e.g., to the second effective pixel bounding component 1603) output a second indication of the configured workload for at least one of composition or reprojection on the calculated set of bounded regions of the effective pixel set of the frame.

[0106] At 1606, the first effective pixel boundary component 1602 can calculate a first sum of the first pixel values ​​of each row of pixels in the frame. At 1608, the first effective pixel boundary component 1602 can identify the first row of pixels and the second row of pixels of the frame having a first non-zero sum based on the calculated first sum of the first pixel values ​​of each row of pixels in the frame, wherein each row of pixels between the first and second rows of pixels can have a third non-zero sum, and wherein each row of pixels outside the first and second rows of pixels can have a zero sum. At 1610, the first effective pixel boundary component 1602 can calculate a second sum of the second pixel values ​​of each column of pixels in the frame. At 1612, the first effective pixel boundary component 1602 may identify the first column of pixels and the second column of pixels of the frame having a second non-zero sum based on a calculated second sum of the second pixel values ​​of each column of pixels in the frame, wherein each column of pixels between the first column of pixels and the second column of pixels may have a fourth non-zero sum, and wherein each column of pixels outside the first column of pixels and the column-row pixels may have a zero sum, and wherein calculating the boundary region set of the effective pixel set of the frame based on a first indication at 1618 may include calculating the boundary region set based on the identified first row, the identified second row, the identified first column, and the identified second column. At 1624, the first effective pixel boundary component 1602 may perform at least one of compositing or reprojection based on the configured workload.

[0107] At 1614, the first effective pixel boundary component 1602 may determine a set of connected regions associated with the frame based on the frame, wherein each connected region in the set of connected regions includes a set of connected pixels. At 1616, the first effective pixel boundary component 1602 may assign each pixel in the effective pixel set to a corresponding connected region in the set of connected regions, wherein calculating the boundary region set of the effective pixel set of the frame based on the first indication at 1618 may include: calculating the boundary region set of the frame based on the assignment.

[0108] Figure 17 This is a flowchart 1700 of an example method for graphics processing according to one or more techniques of this disclosure. The method can be performed by a device (such as a device for graphics processing, a graphics processor (e.g., a GPU), software executing on the graphics processor, graphics processor firmware, a CPU, a display processing unit (DPU) or other display processor, software executing on the display processor, display processor firmware, a hardware block between the graphics processor and the display processor, a wireless communication device, device 104, AR device 402, etc.), as in combination with... Figures 1 to 16 This method is used in various aspects. In the example, it can be executed by the bounding box generator 198.

[0109] At 1702, the device obtains a first indication for at least one of the following regarding a frame of graphical content: the frame, the valid pixel area tracked by the graphics processor associated with the frame, the tile-tracked compressed metadata of the frame, or the tiles associated with the rendering of the frame. For example, Figure 16 At 1604, a first effective pixel boundary component 1602 is shown that can obtain a first indication for a frame of graphical content of at least one of the following: the frame, the effective pixel area tracked by the graphics processor associated with the frame, the tile tracking compressed metadata of the frame, or the tiles associated with the rendering of the frame. In the example, the frame may be or include AR frame 514, AR frame 602, first AR layer 704, second AR layer 706, third AR layer 708, AR frame 1202, AR frame 1304, AR frame 1404, and / or AR frame 1504. In the example, the frame may be associated with a second strategy 902 or a first strategy 1302. In the example, the effective pixel area tracked by the graphics processor associated with the frame may be associated with a first strategy 802. In the example, the tile tracking compressed metadata of the frame may be associated with a third strategy 1002 or a second strategy 1402. In the example, the tiles associated with the rendering of the frame may be associated with a fourth strategy 1102 or a third strategy 1502. In the example, tile tracking compressed metadata may be or include compressed tile metadata 1006 or compressed tile metadata 1408. In the example, the tile associated with the rendering of a frame may be or include graphics processor tile 1104 or graphics processor tile 1506. In the example, 1702 may be performed by bounding box generator 198.

[0110] At 1704, the device calculates the bounded region set of the effective pixel set of the frame based on the first indication. For example, Figure 16 At 1618, a first effective pixel bounding component 1602 is shown that can calculate a set of bounded regions for the effective pixel set of a frame based on a first indication. The set of bounded regions may include bounding boxes and / or bounding regions. For example, the set of bounded regions may be or include bounding boxes 816, 918, 1010, 1012, 1110, 1112, first bounding box 1316, second bounding box 1318, bounding region 1414, 1412, 1512, and / or 1510. In this example, the aforementioned bounding boxes may encompass first sparse AR content 418, second sparse AR content 420, and / or third sparse AR content 422. In this example, the effective pixel set of a frame may be or include effective pixels 516. In this example, 1704 may be performed by a bounding box generator 198.

[0111] At 1706, the device configures the workload for at least one of synthesis or reprojection over the calculated set of bounded regions of the effective pixel set of the frame. For example, Figure 16At 1620, a first effective pixel bounding component 1602 is shown that can configure the workload for at least one of synthesis or reprojection on a calculated set of bounded regions of the effective pixel set of a frame. In the example, 1706 can be performed by a bounding box generator 198.

[0112] At 1708, the device outputs a second indication of the configured workload for at least one of synthesis or reprojection on the calculated set of bounded regions for the effective set of pixels of a frame. For example, Figure 16 At 1622, it is shown that the first effective pixel bounding component 1602 can (e.g., to / for the second effective pixel bounding component 1603) output a second indication of the configured workload for at least one of synthesis or reprojection on the calculated set of bounded regions for the effective pixel set of a frame. In the example, 1708 can be performed by the bounding box generator 198.

[0113] Figure 18 This is a flowchart 1800 of an example method for graphics processing according to one or more techniques of this disclosure. The method can be performed by a device (such as a device for graphics processing, a graphics processor (e.g., a GPU), software executing on the graphics processor, graphics processor firmware, a CPU, a display processing unit (DPU) or other display processor, software executing on the display processor, display processor firmware, a hardware block between the graphics processor and the display processor, a wireless communication device, device 104, AR device 402, etc.), as in combination with... Figures 1 to 16 This method is used in various aspects. In the example, the method (including the various aspects detailed below) can be performed by the bounding box generator 198.

[0114] At 1802, the device obtains a first indication for a frame of graphical content of at least one of the following: the frame, the effective pixel area tracked by the graphics processor associated with the frame, the tile-tracked compressed metadata of the frame, or the tiles associated with the rendering of the frame. For example, Figure 16At 1604, a first effective pixel boundary component 1602 is shown that can obtain a first indication for a frame of graphical content of at least one of the following: the frame, the effective pixel area tracked by the graphics processor associated with the frame, the tile tracking compressed metadata of the frame, or the tiles associated with the rendering of the frame. In the example, the frame may be or include AR frame 514, AR frame 602, first AR layer 704, second AR layer 706, third AR layer 708, AR frame 1202, AR frame 1304, AR frame 1404, and / or AR frame 1504. In the example, the frame may be associated with a second strategy 902 or a first strategy 1302. In the example, the effective pixel area tracked by the graphics processor associated with the frame may be associated with a first strategy 802. In the example, the tile tracking compressed metadata of the frame may be associated with a third strategy 1002 or a second strategy 1402. In the example, the tiles associated with the rendering of the frame may be associated with a fourth strategy 1102 or a third strategy 1502. In the example, tile tracking compressed metadata may be or include compressed tile metadata 1006 or compressed tile metadata 1408. In the example, the tile associated with the rendering of a frame may be or include graphics processor tile 1104 or graphics processor tile 1506. In the example, 1802 may be performed by bounding box generator 198.

[0115] At 1816, the device calculates the bounded region set of the effective pixel set of the frame based on the first indication. For example, Figure 16 At 1618, a first effective pixel bounding component 1602 is shown that can calculate a set of bounded regions for the effective pixel set of a frame based on a first indication. The set of bounded regions may include bounding boxes and / or bounding regions. For example, the set of bounded regions may be or include bounding boxes 816, 918, 1010, 1012, 1110, 1112, a first bounding box 1316, a second bounding box 1318, a bounding region 1414, 1412, 1512, and / or 1510. In this example, the aforementioned bounding boxes may encompass a first sparse AR content 418, a second sparse AR content 420, and / or a third sparse AR content 422. In this example, the effective pixel set of a frame may be or include effective pixels 516. In this example, 1816 may be performed by a bounding box generator 198.

[0116] At 1818, the device configures the workload for at least one of synthesis or reprojection over the calculated set of bounded regions of the effective pixel set of the frame. For example, Figure 16 At 1620, a first effective pixel bounding component 1602 is shown that can configure the workload for at least one of synthesis or reprojection on a calculated set of bounded regions of the effective pixel set of a frame. In the example, 1818 can be performed by a bounding box generator 198.

[0117] At 1820, the device outputs a second indication of the configured workload for at least one of synthesis or reprojection on the calculated set of bounded regions for the effective set of pixels of a frame. For example, Figure 16 At 1622, it is shown that the first effective pixel bounding component 1602 can (e.g., to / for the second effective pixel bounding component 1603) output a second indication of the configured workload for at least one of synthesis or reprojection on the calculated set of bounded regions for the effective pixel set of a frame. In the example, 1820 can be performed by the bounding box generator 198.

[0118] In one aspect, the effective pixel region tracked by the graphics processor associated with the frame may include a minimum horizontal coordinate, a minimum vertical coordinate, a maximum horizontal coordinate, and a maximum vertical coordinate associated with the effective pixel region tracked by the graphics processor. Furthermore, calculating the bounded region set of the effective pixel set at 1618 based on a first indication may include calculating the bounded region set of the effective pixel set based on the minimum horizontal coordinate, minimum vertical coordinate, maximum horizontal coordinate, and maximum vertical coordinate. In the example, the foregoing aspect may correspond to a first strategy 802. For example, the minimum horizontal coordinate may be a minimum X coordinate 808, the minimum vertical coordinate may be a minimum Y coordinate 812, the maximum horizontal coordinate may be a maximum X coordinate 810, and the maximum vertical coordinate may be a maximum Y coordinate 814. In the example, the bounded region set may include a bounding box 816.

[0119] In one aspect, at 1804, the device can calculate the first sum of the first pixel values ​​of each row of pixels in the frame. For example, Figure 16 At 1606, it is shown that the first effective pixel bounding component 1602 can calculate a first sum of the first pixel values ​​for each row of pixels in the frame. In the example, the foregoing aspect may correspond to the second strategy 902. For example, the first sum of the first pixel values ​​may correspond to the row pixel sum 906. In the example, 1804 may be performed by the bounding box generator 198.

[0120] In one aspect, at 1806, the device can identify a first row of pixels of a frame and a second row of pixels of a frame having a first non-zero sum based on a calculated first sum of the first pixel values ​​of each row of pixels in the frame, wherein each row of pixels between the first and second rows can have a third non-zero sum, and wherein each row of pixels outside the first and second rows can have a zero sum. For example, Figure 16At 1608, it is shown that a first effective pixel bounding component 1602 can identify a first row of pixels in a frame and a second row of pixels in a frame having a first non-zero sum based on a calculated first sum of the first pixel values ​​of each row of pixels in the frame, wherein each row of pixels between the first and second rows of pixels can have a third non-zero sum, and wherein each row of pixels outside the first and second rows of pixels can have a zero sum. In the example, the foregoing aspect may correspond to a second strategy 902. For example, the first row of pixels may be a first row 910, and the second row of pixels may be the last row 912. In the example, 1806 may be performed by a bounding box generator 198.

[0121] In one aspect, at 1808, the device can calculate a second sum of the second pixel values ​​for each column of pixels in the frame. For example, Figure 16 At 1610, it is shown that the first effective pixel bounding component 1602 can calculate a second sum of the second pixel values ​​for each column of pixels in the frame. In the example, the foregoing aspect may correspond to a second strategy 902. For example, the second of the second pixel values ​​may correspond to the column pixel sum 908. In the example, 1808 may be performed by the bounding box generator 198.

[0122] In one aspect, at 1810, the device may identify a first column of pixels and a second column of pixels of a frame having a second non-zero sum based on a calculated second sum of the second pixel values ​​of each column of pixels in the frame, wherein each column of pixels between the first and second columns may have a fourth non-zero sum, and wherein each column of pixels outside the first column and the column-row pixels may have a zero sum, and wherein calculating the set of bounded regions of the effective pixel set of the frame based on a first indication may include: calculating the set of bounded regions based on the identified first row, the identified second row, the identified first column, and the identified second column. For example, Figure 16 At 1612, a first effective pixel bounding component 1602 is shown that can identify the first column of pixels and the second column of pixels of the frame having a second non-zero sum based on a calculated second sum of the second pixel values ​​of each column of pixels in the frame, wherein each column of pixels between the first and second columns of pixels can have a fourth non-zero sum, and wherein each column of pixels outside the first column and the column-row pixels can have a zero sum, and wherein calculating the bounding region set of the effective pixel set of the frame based on a first indication at 1618 can include: calculating the bounding region set based on the identified first row, the identified second row, the identified first column, and the identified second column. In the example, the foregoing aspects may correspond to the second strategy 902. For example, the first column may be the first column 914, and the second column may be the last column 916. In the example, 1810 may be performed by the bounding box generator 198.

[0123] In one aspect, the tile tracking compression metadata of a frame may include a buffer header indicating tiles associated with the frame that include a valid set of pixels, and calculating the bounded region set of the frame's valid set of pixels based on the first indication may include calculating the bounded region set based on the buffer header. In an example, the foregoing aspect may correspond to a third strategy 1002. For example, the tile compression metadata may be or include compressed tile metadata 1006, the buffer header may be or include a buffer header 1008, and the tile may be or include compressed tile 1004. In an example, the foregoing aspect may correspond to a second strategy 1402. For example, the tile compression metadata may be or include compressed tile metadata 1408, the buffer header may be or include a buffer header 1410, and the tile may be or include compressed tile 1406. In another example, calculating the bounded region set of the frame's valid set of pixels based on the first indication at 1618 may include calculating the bounded region set based on the buffer header.

[0124] In one aspect, the rendering-associated tiles of a frame may be tracked by at least one of graphics processor software, graphics processor firmware, or graphics processor hardware, wherein the rendering-associated tiles of a frame may include a valid set of pixels, and wherein calculating the set of bounded regions of the valid set of pixels of a frame based on a first indication may include: calculating the set of bounded regions of a frame based on the rendering-associated tiles of the frame. In an example, the foregoing aspect may correspond to a fourth strategy 1102. For example, the graphics processor software may be or include graphics processor software 1114, the graphics processor firmware may be or include graphics processor firmware 1116, and / or the graphics processor hardware may be or include graphics processor hardware 1106. In an example, the rendering-associated tiles may be or include graphics processor tiles 1104. In an example, the foregoing aspect may correspond to a third strategy 1502. For example, the graphics processor software may be or include graphics processor software 1514, the graphics processor firmware may be or include graphics processor firmware 1516, and / or the graphics processor hardware may be or include graphics processor hardware 1508. In an example, the rendering-associated tiles may be or include graphics processor tiles 1506. In another example, calculating the bounded region set of the effective pixel set of the frame at 1618 based on the first indication may include calculating the bounded region set of the frame based on the tiles associated with the rendering of the frame.

[0125] In one aspect, the set of bounding regions may include at least one of a set of bounding boxes or a set of bounding regions, wherein each bounding box in the set of bounding boxes may include a rectangular shape, and wherein each bounding region in the set of bounding regions may include a non-rectangular shape. For example, the set of bounding regions may be or include bounding boxes 816, 918, 1010, bounding region 1012, 1110, 1112, first bounding box 1316, second bounding box 1318, bounding region 1414, bounding box 1412, bounding region 1512, and / or bounding box 1510. In the example, Figure 15 The boundary frame 1510 is shown to have a rectangular shape, and the boundary region 1512 may have a non-rectangular shape.

[0126] In one aspect, at 1812, the device can determine a set of connected regions associated with a frame based on the frame, wherein each connected region in the set of connected regions includes a set of connected pixels. For example, Figure 16 At 1614, a first effective pixel boundary component 1602 is shown that can determine a set of connected regions associated with a frame based on the frame, wherein each connected region in the set of connected regions includes a set of connected pixels. In the example, the foregoing aspect may correspond to a first strategy 1302. For example, the set of connected regions may be or include a first connected region 1308 and / or a second connected region 1310. In the example, Figure 13 The connected pixels of the first connected region 1308 are represented by "1", and the connected pixels of the second connected region 1310 are represented by "2". In the example, 1812 can be performed by the bounding box generator 198.

[0127] In one aspect, at 1814, the device may assign each pixel in the effective pixel set to a corresponding connected region in the connected region set, wherein calculating the bounded region set of the effective pixel set of the frame based on the first indication may include: calculating the bounded region set of the frame based on the assignment. For example, Figure 15 At 1614, a first effective pixel bounding component 1602 is shown that can assign each pixel in the effective pixel set to a corresponding connected region in the connected region set, wherein calculating the bounded region set of the effective pixel set of the frame based on a first indication at 1618 may include: calculating the bounded region set of the frame based on the assignment. In the example, 1814 may be performed by a bounding box generator 198.

[0128] In one aspect, the effective set of pixels may correspond to sparse extended reality (XR) content, and the sparse XR content may include a region smaller than a threshold region of at least one of the frames or display panels. For example, the sparse XR content may be or include first sparse AR content 418, second sparse AR content 420, and / or third sparse AR content 422. In the example, the first sparse AR content 418, the second sparse AR content 420, and / or the third sparse AR content 422 may have a region smaller than a threshold region of AR frame 602.

[0129] In one aspect, configuring workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame may include at least one of the following operations: calculating the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame; allocating the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame; or adjusting the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame. For example, configuring workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame at 1620 may include at least one of the following operations: calculating the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame; allocating the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame; or adjusting the workload for at least one of the composite or reprojection on the calculated bounded region set of the effective pixel set of a frame.

[0130] In one aspect, at 1822, the device can perform at least one of synthesis or reprojection based on the configured workload. For example, Figure 16 At 1624, it is shown that the first effective pixel bounding component 1602 can perform at least one of compositing or reprojection based on the configured workload. In the example, 1822 can be performed by the bounding box generator 198.

[0131] In one aspect, performing at least one of synthesis or reprojection based on the configured workload may include performing segmentation based on the configured workload. For example, performing at least one of synthesis or reprojection based on the configured workload at 1624 may include performing segmentation based on the configured workload.

[0132] In one aspect, performing at least one of compositing or reprojection based on the configured workload may include performing at least one of compositing or reprojection on the effective set of pixels. For example, performing at least one of compositing or reprojection based on the configured workload at 1624 may include performing at least one of compositing or reprojection on the effective set of pixels.

[0133] In one aspect, the effective pixel set may include a first effective pixel set and a second effective pixel set, wherein the first effective pixel set may correspond to a first layer and the second effective pixel set may correspond to a second layer, and wherein performing compositing may include compositing the first layer and the second layer. For example, the first effective pixel set may correspond to a first AR layer 704 and the second pixel set may correspond to a second AR layer 706, and performing compositing at 1624 may include compositing the first AR layer 704 and the second AR layer 706.

[0134] In one aspect, a frame may include a set of valid pixels and a set of invalid pixels, wherein the set of valid pixels may correspond to a first area set of a display panel displaying graphic content, and wherein the set of invalid pixels may correspond to a second area set of a display panel not displaying graphic content. For example, valid pixels may be or include valid pixel 516, and the set of invalid pixels may be or include invalid pixel 518.

[0135] In one aspect, outputting a second indication of the configured workload may include at least one of the following operations: sending a second indication of the configured workload; or storing a second indication of the configured workload in at least one of a memory, buffer, or cache. For example, outputting a second indication of the configured workload at 1622 may include sending a second indication of the configured workload (e.g., at 1622A). In another example, outputting a second indication of the configured workload at 1622 may include storing a second indication of the configured workload in at least one of a memory, buffer, or cache.

[0136] The configuration provides a method or apparatus for graphics processing. The apparatus may be a GPU, a CPU, or some other processor capable of performing graphics processing. In various aspects, the apparatus may be a processing unit 120 within device 104, or some other hardware within device 104 or another device. The apparatus may be a DPU, a display processor, or some other processor capable of performing display processing. In various aspects, the apparatus may be a display processor 127 within device 104, or some other hardware within device 104 or another device. The apparatus may include components for obtaining a first indication for at least one of the following for a frame of graphical content: a frame, a valid pixel region tracked by a graphics processor associated with the frame, tile-tracked compressed metadata of the frame, or tiles associated with the rendering of the frame. The apparatus may also include components for calculating a set of bounded regions of the valid pixel set of the frame based on the first indication. The apparatus may also include components for configuring workloads for at least one of composition or reprojection on the calculated set of bounded regions of the valid pixel set of the frame. The apparatus may further include a second indication of a configured workload for outputting at least one of synthesis or reprojection on a calculated set of bounded regions for a valid set of pixels in a frame. The apparatus may further include a first sum of first pixel values ​​for each row of pixels in the frame. The apparatus may further include a component for identifying a first row of pixels and a second row of pixels in a frame having a first non-zero sum based on the calculated first sum of the first pixel values ​​for each row of pixels in the frame, wherein each row of pixels between the first and second rows has a third non-zero sum, and wherein each row of pixels outside the first and second rows has a zero sum. The apparatus may further include a component for calculating a second sum of second pixel values ​​for each column of pixels in the frame. The apparatus may further include means for identifying a first column of pixels and a second column of pixels of a frame having a second non-zero sum based on a calculated second sum of a second pixel value for each column of pixels in the frame, wherein each column of pixels between the first and second columns has a fourth non-zero sum, and wherein each column of pixels outside the first column and the column / row pixels has a zero sum, and wherein calculating a set of bounded regions of the effective pixel set of the frame based on a first indication includes calculating the set of bounded regions based on the identified first row, the identified second row, the identified first column, and the identified second column. The apparatus may further include means for determining a set of connected regions associated with the frame based on the frame, wherein each connected region in the set of connected regions includes a set of connected pixels. The apparatus may further include means for assigning each pixel in the effective pixel set to a corresponding connected region in the set of connected regions, wherein calculating the set of bounded regions of the effective pixel set of the frame based on the first indication includes calculating the set of bounded regions of the frame based on the assignment. The apparatus may further include means for performing at least one of compositing or reprojection based on a configured workload.

[0137] It should be understood that the specific order or hierarchy of boxes / steps in the processes, flowcharts, and / or call flowcharts disclosed herein are merely illustrative of example methods. It should be understood that the specific order or hierarchy of boxes / steps in these processes, flowcharts, and / or call flowcharts may be rearranged based on design preferences. Furthermore, some boxes / steps may be combined or omitted. Other boxes / steps may also be added. The appended method claims provide the elements of various boxes / steps in an exemplary order, but are not intended to limit one to the given specific order or hierarchy.

[0138] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims, wherein, unless specifically stated otherwise, references to elements in the singular are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0139] Unless otherwise specified, the term "some" refers to one or more, and the term "or" may be interpreted as "and / or" unless otherwise specified in the context. Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, which may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, wherein any such combination may contain one or more members of A, B, or C. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. The words “module,” “mechanism,” “element,” “device,” etc., cannot replace the word “component.” Therefore, no claim element will be construed as a functional component unless the element is explicitly recited using the phrase “component for…”. Unless otherwise stated, the phrase “processor” may refer to “any processor in one or more processors” (e.g., one processor in one or more processors, a plurality (more than one) of one or more processors, or all processors in one or more processors), and the phrase “memory” may refer to “any memory in one or more memories” (e.g., one memory in one or more memories, a plurality (more than one) of one or more memories, or all memories in one or more memories).

[0140] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then such functionality, processing unit, technique, or other module may be stored on or transmitted on a computer-readable medium as one or more instructions or code.

[0141] Computer-readable media may include computer data storage media and communication media, including any media that facilitates the transfer of computer programs from one place to another. In this way, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium; or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures for implementing the techniques described in this disclosure. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, compressed optical disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices, or other magnetic storage devices. As used herein, magnetic disks and optical discs include compressed optical discs (CD), laser optical discs, optical discs, digital versatile optical discs (DVD), floppy disks, and Blu-ray discs, wherein magnetic disks typically magnetically copy data, while optical discs optically copy data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0142] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in any hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0143] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.

[0144] Aspect 1 is a graphics processing method, the method comprising: for a frame of graphics content, obtaining a first indication of at least one of: the frame, an effective pixel region tracked by a graphics processor associated with the frame, tile tracking compressed metadata of the frame, or tiles associated with rendering of the frame; calculating a set of bounded regions of the effective pixel set of the frame based on the first indication; configuring a workload for at least one of composition or reprojection on the calculated set of bounded regions of the effective pixel set of the frame; and outputting a second indication of the configured workload for at least one of composition or reprojection on the calculated set of bounded regions of the effective pixel set of the frame.

[0145] Aspect 2 can be combined with aspect 1, wherein the effective pixel region tracked by the graphics processor associated with the frame includes a minimum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, a minimum vertical coordinate associated with the effective pixel region tracked by the graphics processor, a maximum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, and a maximum vertical coordinate associated with the effective pixel region tracked by the graphics processor, and wherein calculating the set of bounded regions of the effective pixel set based on the first indication includes: calculating the set of bounded regions of the effective pixel set based on the minimum horizontal coordinate, the minimum vertical coordinate, the maximum horizontal coordinate, and the maximum vertical coordinate.

[0146] Aspect 3 may be combined with any of Aspects 1 to 2, and further includes: calculating a first sum of first pixel values ​​for each row of pixels in the frame; identifying a first row of pixels and a second row of pixels in the frame having a first non-zero sum based on the calculated first sum of the first pixel values ​​for each row of pixels in the frame, wherein each row of pixels between the first row of pixels and the second row of pixels has a third non-zero sum, and wherein each row of pixels outside the first row of pixels and the second row of pixels has a zero sum; calculating a second sum of second pixel values ​​for each column of pixels in the frame; and identifying a first column of pixels and a second column of pixels in the frame having a second non-zero sum based on the calculated second sum of the second pixel values ​​for each column of pixels in the frame, wherein each column of pixels between the first column of pixels and the second column of pixels has a fourth non-zero sum, and wherein each column of pixels outside the first column of pixels and the column of pixels has the zero sum, and wherein calculating the set of bounded regions of the effective pixel set of the frame based on the first indication includes: calculating the set of bounded regions based on the identified first row, the identified second row, the identified first column, and the identified second column.

[0147] Aspect 4 may be combined with any of aspects 1 to 3, wherein the tile tracking compressed metadata of the frame includes a buffer header indicating tiles associated with the frame that include the set of valid pixels, and wherein calculating the set of bounded regions of the set of valid pixels of the frame based on the first indication includes calculating the set of bounded regions based on the buffer header.

[0148] Aspect 5 may be combined with any of Aspects 1 to 4, wherein the tiles associated with the rendering of the frame are tracked by at least one of graphics processor software, graphics processor firmware, or graphics processor hardware, wherein the tiles associated with the rendering of the frame include the set of valid pixels, and wherein calculating the set of bounded regions of the set of valid pixels of the frame based on the first indication includes: calculating the set of bounded regions of the frame based on the tiles associated with the rendering of the frame.

[0149] Aspect 6 may be combined with any of aspects 1 to 5, wherein the set of bounding regions includes at least one of a set of bounding frames or a set of bounding areas, wherein each bounding frame in the set of bounding frames includes a rectangular shape, and wherein each bounding area in the set of bounding areas includes a non-rectangular shape.

[0150] Aspect 7 may be combined with any of aspects 1 to 6, and further includes: determining a set of connected regions associated with the frame based on the frame, wherein each connected region in the set of connected regions includes a set of connected pixels; and assigning each pixel in the set of valid pixels to a corresponding connected region in the set of connected regions, wherein calculating the set of bounded regions of the set of valid pixels of the frame based on the first indication includes: calculating the set of bounded regions of the frame based on the assignment.

[0151] Aspect 8 may be combined with any of aspects 1 to 7, wherein the effective set of pixels corresponds to sparse extended reality (XR) content, and wherein the sparse XR content includes a region smaller than a threshold region of at least one of the frames or display panels.

[0152] Aspect 9 may be combined with any of Aspects 1 to 8, wherein configuring the workload for at least one of the synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame includes at least one of the following operations: calculating the workload for at least one of the synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame; allocating the workload for at least one of the synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame; or adjusting the workload for at least one of the synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame.

[0153] Aspect 10 may be combined with any of aspects 1 to 9, and further includes performing at least one of the synthesis or the reprojection based on the configured workload.

[0154] Aspect 11 may be combined with aspect 10, wherein performing at least one of the synthesis or the reprojection based on the configured workload includes performing segmentation based on the configured workload.

[0155] Aspect 12 may be combined with any of aspects 10 to 11, wherein performing at least one of the synthesis or the reprojection based on the configured workload includes performing at least one of the synthesis or the reprojection on the effective set of pixels.

[0156] Aspect 13 may be combined with any of aspects 10 to 12, wherein the effective pixel set includes a first effective pixel set and a second effective pixel set, wherein the first effective pixel set corresponds to a first layer and the second effective pixel set corresponds to a second layer, and wherein performing the synthesis includes: synthesizing the first layer and the second layer.

[0157] Aspect 14 may be combined with any of aspects 1 to 13, wherein the frame includes the set of valid pixels and the set of invalid pixels, wherein the set of valid pixels corresponds to a first area set of the display panel displaying the graphic content, and wherein the set of invalid pixels corresponds to a second area set of the display panel not displaying the graphic content.

[0158] Aspect 15 may be combined with any of aspects 1 to 14, wherein the output of the second indication for the configured workload includes at least one of the following operations: sending the second indication for the configured workload; or storing the second indication for the configured workload in at least one of a memory, a buffer, or a cache.

[0159] Aspect 16 is an apparatus for graphics processing, the apparatus including a processor coupled to a memory, and configured to implement the method according to any one of aspects 1 to 15 based on information stored in the memory.

[0160] Aspect 17 may be combined with aspect 16 and includes: the device is a wireless communication device, the wireless communication device including at least one of an antenna or a transceiver coupled to the processor.

[0161] Aspect 18 is an apparatus for graphics processing, the apparatus comprising components for implementing the method according to any one of aspects 1 to 15.

[0162] Aspect 19 is a computer-readable medium (e.g., a non-transitory computer-readable medium) that stores computer-executable code, which, when executed by a processor, causes the processor to implement the method according to any one of aspects 1 to 15.

[0163] Various aspects have been described herein. These and other aspects are within the scope of the following claims.

Claims

1. An apparatus for graphics processing, the apparatus comprising: Memory; and A processor, coupled to the memory, and configured based on information stored in the memory, to: For a frame of graphical content, a first indication is obtained for at least one of the following: the frame, a valid pixel area tracked by a graphics processor associated with the frame, tile tracking compressed metadata of the frame, or a tile associated with the rendering of the frame. The set of bounded regions of the effective pixel set of the frame is calculated based on the first indication; The workload is configured for at least one of synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame; as well as Output a second indication of the workload configured for at least one of the synthesis or the reprojection on the calculated set of bounded regions for the effective set of pixels for the frame.

2. The apparatus of claim 1, wherein the effective pixel region tracked by the graphics processor associated with the frame includes a minimum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, a minimum vertical coordinate associated with the effective pixel region tracked by the graphics processor, a maximum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, and a maximum vertical coordinate associated with the effective pixel region tracked by the graphics processor, and wherein, in order to calculate the set of bounded regions of the effective pixel set based on the first indication, the processor is configured to calculate the set of bounded regions of the effective pixel set based on the minimum horizontal coordinate, the minimum vertical coordinate, the maximum horizontal coordinate, and the maximum vertical coordinate.

3. The apparatus of claim 1, wherein the processor is further configured to: Calculate the first sum of the first pixel values ​​of the first pixel in each row of pixels in the frame; The first row of pixels in the frame and the second row of pixels in the frame are identified by a first sum calculated based on the first pixel value of each row of pixels in the frame, wherein each row of pixels between the first row of pixels and the second row of pixels has a third non-zero sum, and wherein each row of pixels outside the first row of pixels and the second row of pixels has a zero sum; Calculate the second sum of the second pixel values ​​for each column of pixels in the frame; and The first column of pixels and the second column of pixels in the frame are identified by a second sum calculated based on the second pixel value of each column of pixels in the frame, wherein each column of pixels between the first column of pixels and the second column of pixels has a fourth non-zero sum, and wherein each column of pixels outside the first column of pixels and the column of pixels has the zero sum, and wherein in order to calculate the set of bounded regions of the set of valid pixels of the frame based on the first indication, the processor is configured to calculate the set of bounded regions based on the identified first row, the identified second row, the identified first column, and the identified second column.

4. The apparatus of claim 1, wherein the tile tracking compressed metadata of the frame includes a buffer header indicating tiles associated with the frame that include the set of valid pixels, and wherein the processor is configured to calculate the set of bounded regions based on the buffer header in order to calculate the set of bounded regions of the set of valid pixels of the frame based on the first indication.

5. The apparatus of claim 1, wherein the rendering-associated tiles of the frame are tracked by at least one of graphics processor software, graphics processor firmware, or graphics processor hardware, wherein the rendering-associated tiles of the frame include the effective set of pixels, and wherein, in order to calculate the set of bounded regions of the effective set of pixels of the frame based on the first indication, the processor is configured to calculate the set of bounded regions of the frame based on the rendering-associated tiles of the frame.

6. The apparatus of claim 1, wherein the set of bounding regions includes at least one of a set of bounding frames or a set of bounding areas, wherein each bounding frame in the set of bounding frames includes a rectangular shape, and wherein each bounding area in the set of bounding areas includes a non-rectangular shape.

7. The apparatus of claim 1, wherein the processor is further configured to: Based on the frame, a set of connected regions associated with the frame is determined, wherein each connected region in the set of connected regions includes a set of connected pixels; and Each pixel in the set of valid pixels is assigned to a corresponding connected region in the set of connected regions, wherein, in order to calculate the set of bounded regions of the set of valid pixels of the frame based on the first indication, the processor is configured to calculate the set of bounded regions of the frame based on the assignment.

8. The apparatus of claim 1, wherein the effective set of pixels corresponds to sparse extended reality (XR) content, and wherein the sparse XR content includes a region smaller than a threshold region of at least one of the frames or display panels.

9. The apparatus of claim 1, wherein, in order to configure the workload for at least one of the synthesis or the reprojection on a calculated set of bounded regions of the effective pixel set of the frame, the processor is configured to perform at least one of the following operations: Calculate the workload of at least one of the synthesis or the reprojection on the calculated bounded region set of the effective pixel set for the frame; The workload allocated to at least one of the synthesis or the reprojection on the calculated bounded region set of the effective pixel set of the frame; or Adjust the workload of at least one of the synthesis or the reprojection on the calculated bounded region set of the effective pixel set for the frame.

10. The apparatus of claim 1, wherein the processor is configured to: The synthesis or reprojection is performed based on the configured workload.

11. The apparatus of claim 10, wherein the processor is configured to perform segmentation based on the configured workload in order to perform at least one of the synthesis or the reprojection based on the configured workload.

12. The apparatus of claim 10, wherein, in order to perform at least one of the synthesis or the reprojection based on a configured workload, the processor is configured to perform at least one of the synthesis or the reprojection on the effective set of pixels.

13. The apparatus of claim 10, wherein the effective pixel set includes a first effective pixel set and a second effective pixel set, wherein the first effective pixel set corresponds to a first layer and the second effective pixel set corresponds to a second layer, and wherein, in order to perform the synthesis, the processor is configured to synthesize the first layer and the second layer.

14. The apparatus of claim 1, wherein the frame includes the set of valid pixels and the set of invalid pixels, wherein the set of valid pixels corresponds to a first area set of the display panel displaying the graphic content, and wherein the set of invalid pixels corresponds to a second area set of the display panel not displaying the graphic content.

15. The apparatus of claim 1, wherein in order to output the second indication to the configured workload, the processor is configured to perform at least one of the following operations: Send the second instruction for the configured workload; or The second instruction for the configured workload is stored in at least one of the memory, buffer, or cache.

16. The apparatus of claim 1, wherein the apparatus is a wireless communication device, the wireless communication device comprising at least one of a transceiver or an antenna coupled to the processor.

17. A method for image processing, the method comprising: For a frame of graphical content, a first indication is obtained for at least one of the following: the frame, a valid pixel area tracked by a graphics processor associated with the frame, tile tracking compressed metadata of the frame, or a tile associated with the rendering of the frame. The set of bounded regions of the effective pixel set of the frame is calculated based on the first indication; The workload is configured for at least one of synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame; as well as Output a second indication of the workload configured for at least one of the synthesis or the reprojection on the calculated set of bounded regions for the effective set of pixels for the frame.

18. The method of claim 17, wherein the effective pixel region tracked by the graphics processor associated with the frame includes a minimum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, a minimum vertical coordinate associated with the effective pixel region tracked by the graphics processor, a maximum horizontal coordinate associated with the effective pixel region tracked by the graphics processor, and a maximum vertical coordinate associated with the effective pixel region tracked by the graphics processor, and wherein calculating the set of bounded regions of the effective pixel set based on the first indication includes: The set of bounded regions of the effective pixel set is calculated based on the minimum horizontal coordinate, the minimum vertical coordinate, the maximum horizontal coordinate, and the maximum vertical coordinate.

19. The method of claim 17, further comprising: Calculate the first sum of the first pixel values ​​of the first pixel in each row of pixels in the frame; The first row of pixels in the frame and the second row of pixels in the frame are identified by a first sum calculated based on the first pixel value of each row of pixels in the frame, wherein each row of pixels between the first row of pixels and the second row of pixels has a third non-zero sum, and wherein each row of pixels outside the first row of pixels and the second row of pixels has a zero sum; Calculate the second sum of the second pixel values ​​for each column of pixels in the frame; as well as The first column of pixels and the second column of pixels in the frame are identified by a second sum calculated based on the second pixel value of each column of pixels in the frame, wherein each column of pixels between the first column of pixels and the second column of pixels has a fourth non-zero sum, and wherein each column of pixels outside the first column of pixels and the column of pixels has the zero sum, and wherein the set of bounded regions for calculating the set of valid pixels of the frame based on the first indication includes: calculating the set of bounded regions based on the identified first row, the identified second row, the identified first column, and the identified second column.

20. A computer-readable medium storing computer-executable code, which, when executed by a processor, causes the processor to: For a frame of graphical content, a first indication is obtained for at least one of the following: the frame, a valid pixel area tracked by a graphics processor associated with the frame, tile tracking compressed metadata of the frame, or a tile associated with the rendering of the frame. The set of bounded regions of the effective pixel set of the frame is calculated based on the first indication; The workload is configured for at least one of synthesis or reprojection on the calculated bounded region set of the effective pixel set of the frame; as well as Output a second indication of the workload configured for at least one of the synthesis or the reprojection on the calculated set of bounded regions for the effective set of pixels for the frame.

Images