Content shift in foveated rendering

Abstract

The present disclosure relates to methods and apparatuses, including devices, such as GPUs, for graphics processing. The device processes (820) at least one frame of frame content associated with a grid comprising a plurality of grid portions, each of the plurality of portions of frame content being aligned with at least a portion of at least one of the plurality of grid portions. The device shifts (830) the frame content relative to the grid such that at least one of the plurality of portions of frame content is aligned with at least a portion of at least one different grid portion of the plurality of grid portions. Finally, the device stores (840) the shifted frame content, the shifted frame content comprising at least one portion of frame content aligned with at least a portion of the at least one different grid portion.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Patent Application No. 17 / 229,242, filed April 13, 2021, entitled “METHODS AND APPARATUS FORCONTENT SHIFTING IN FOVEATED RENDERING”, which is expressly incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to processing systems, and more specifically, to one or more techniques for graphics processing. Background Technology

[0004] Computing devices typically perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. A GPU is configured to execute a graphics processing pipeline comprising one or more processing stages that operate together to execute graphics processing commands and output frames. A CPU can control the operation of a GPU by issuing one or more graphics processing commands to the GPU. Modern CPUs are typically capable of executing multiple applications simultaneously, each of which may require the GPU to be utilized during execution. A display processor is configured to convert digital information received from the CPU into analog values ​​and can issue commands to a display panel for displaying visual content. Devices providing content for visual presentation on a display can utilize a GPU and / or a display processor.

[0005] The device's GPU can be configured to execute processes in the graphics processing pipeline. Additionally, a display processor or display processing unit (DPU) can be configured to execute display processing processes. However, with the advent of wireless communication and smaller handheld devices, the demand for improved graphics or display processing has increased. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not a comprehensive summary of all anticipated aspects, nor is it intended to identify key or important elements of all aspects, nor to depict the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed descriptions that follow.

[0007] In one aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be a graphics processing unit (GPU) or any apparatus capable of performing graphics processing. The apparatus may receive at least one frame including frame content, wherein the at least one frame is processed after being received. The apparatus may also process at least one frame including frame content associated with a grid comprising a plurality of grid portions, each of the plurality of portions of the frame content being aligned with at least a portion of at least one of the plurality of grid portions. Additionally, the apparatus may shift the frame content relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of the plurality of grid portions. The apparatus may also store the shifted frame content while shifting it relative to the grid, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid portion. The apparatus may also retrieve the shifted frame content while storing it. Furthermore, the apparatus may re-shift the retrieved frame content when it is retrieved, such that the re-shifted frame content corresponds to the processed frame content. The apparatus may also render the re-shifted frame content while re-shifting it. The device can also send one or more frames that include the re-shifted frame content when the frame content is re-shifted.

[0008] Details of one or more examples of the invention are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of this disclosure will be apparent from the specification, drawings, and claims. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating an example content generation system based on one or more techniques according to this disclosure.

[0010] Figure 2 An example GPU is described according to one or more techniques of the present invention.

[0011] Figure 3 Example images or surfaces illustrating one or more techniques according to the present invention.

[0012] Figure 4A This is a diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0013] Figure 4B This is a diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0014] Figure 5A A diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0015] Figure 5BThis is a diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0016] Figure 6A This is a diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0017] Figure 6B This is a diagram illustrating example content in a frame according to one or more techniques of the present invention.

[0018] Figure 7A A graph illustrating the movement of content over time according to one or more techniques of the present invention.

[0019] Figure 7B A graph illustrating the movement of content over time according to one or more techniques of the present invention.

[0020] Figure 8 This is a communication flowchart illustrating an example of communication between a GPU component and a memory / buffer according to one or more technologies of the present invention.

[0021] Figure 9 This is a flowchart of an example method for graphics processing according to one or more techniques of the present invention. Detailed Implementation

[0022] In some aspects of graphics processing, virtual reality (VR), augmented reality (AR), or extended reality (XR) applications can leverage different rendering techniques to reduce rendering workload. For example, VR / AR / XR applications can utilize foveated rendering, a rendering technique that uses eye trackers integrated with the headset. Foveated rendering can help reduce rendering workload by scaling content outside a certain area (e.g., the area of ​​focus for the user's vision). In some aspects, when the fovea is associated with a mesh, it can become difficult to consistently move the user's focus area. Furthermore, this can lead to significant quality variations in the transformation, i.e., noticeable or "bursting" quality changes. Additionally, eye tracking utilized by foveated rendering (i.e., moving the focus along with the user's eye focus) can be difficult to perform smoothly. In some aspects, certain content may need to be rendered multiple times, which contrasts with the expectation of fast rendering in VR / AR / XR applications. Therefore, reducing the rendering time associated with foveated rendering in VR / AR / XR applications can be beneficial. In some cases, adjusting the mesh (e.g., tile mesh) during foveated rendering processing can have a high probability of negatively impacting performance. For example, adjusting the shape or area of ​​regions with different qualities during foveated rendering can lead to unpredictable rendering performance. When utilizing foveated rendering, aspects of this disclosure can allow for consistent visual quality. Furthermore, aspects of this disclosure can smoothly and consistently perform certain functions used with foveated rendering, such as eye tracking. Aspects of this disclosure can also adjust regions or meshes associated with foveated rendering without negatively impacting the performance observed by the user. For example, aspects of this disclosure can adjust the shape or area of ​​regions with different qualities during foveated rendering while maintaining consistent rendering performance. Additionally, aspects of this disclosure can reduce the rendering time of foveated rendering in VR / AR / XR applications.

[0023] Various aspects of the systems, apparatus, computer program products, and methods are described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function given throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of the systems, apparatus, computer program products, and methods disclosed herein, whether implemented independently of or in combination with other aspects of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover such apparatus or methods practiced using structures, functions, or structures and functions other than or different from the aspects of this disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.

[0024] While various aspects have been described herein, many variations and arrangements of these aspects also fall within the scope of this disclosure. Although some potential benefits and advantages of the aspects of this disclosure have been mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transport protocols, some of which are illustrated by way of example in the accompanying drawings and the following description. The detailed description and accompanying drawings are merely illustrative and not limiting of this disclosure, the scope of which is defined by the appended claims and their equivalents.

[0025] Several aspects are given with reference to various apparatuses and methods. These apparatuses and methods are described in detail below and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively, “elements”). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system.

[0026] As an example, an element, any part of an element, or any combination of elements may be implemented as a “processing system” including one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. One or more processors in a processing system can execute software. Software can be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or other terms. The terminology may be used to refer to software. As described herein, one or more technologies can refer to an application, i.e., software, configured to perform one or more functions. In such examples, the application may be stored on memory, such as the processor's on-chip memory, system memory, or any other memory. Hardware such as a processor described herein may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more technologies described herein. As an example, the hardware may access code from memory and execute the code accessed from memory to perform one or more technologies described herein. In some examples, components are identified in this disclosure. In such examples, a component may be hardware, software, or a combination thereof. A component may be a single component or a subcomponent of a single component.

[0027] Accordingly, in one or more examples described herein, the described functionality can be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality can be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media can be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of computer-readable media of the foregoing types, or any other medium that can be used to store computer-executable code in the form of instructions or data structures accessible to a computer.

[0028] Generally, this invention describes techniques for having a graphics processing pipeline in a single device or multiple devices, improving the rendering of graphics content, and / or reducing the load on processing units (i.e., any processing unit, such as a GPU, configured to perform one or more of the techniques described herein). For example, this invention describes techniques for utilizing graphics processing in any device. Other exemplary benefits are described throughout this disclosure.

[0029] As used herein, instances of the term "content" can refer to "graphic content," "image," or vice versa. This is true regardless of whether the term is used as an adjective, noun, or other part of speech. In some examples, as used herein, the term "graphic content" can refer to content produced by one or more processes in the graphics processing pipeline. In some examples, as used herein, the term "graphic content" can refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used herein, the term "graphic content" can refer to content produced by a graphics processing unit.

[0030] In some examples, as used herein, the term "display content" can refer to content generated by a processing unit configured to perform display processing. Graphical content can be processed to become display content. For example, a graphics processing unit can output graphical content, such as frames, to a buffer (which may be referred to as a frame buffer). A display processing unit can read graphical content (such as one or more frames) from the buffer and perform one or more display processing techniques on it to generate display content. For example, a display processing unit can be configured to perform compositing on one or more rendering layers to generate frames. As another example, a display processing unit can be configured to combine, blend, or otherwise combine two or more layers together to form a single frame. A display processing unit can be configured to perform scaling on frames, such as zooming in or out. In some examples, a frame can refer to a layer. In other examples, a frame can refer to two or more layers that have been blended together to form a frame; that is, a frame comprises two or more layers, and frames comprising two or more layers can be subsequently blended.

[0031] Figure 1This is a block diagram illustrating an example content generation system 100 configured to implement one or more techniques of the present disclosure. The content generation system 100 includes a device 104. Device 104 may include one or more components or circuitry for performing the various functions described herein. In some examples, one or more components of device 104 may be components of a System-on-a-Chip (SOC). Device 104 may include one or more components configured to perform one or more techniques of the present invention. In the illustrated example, device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, device 104 may include several optional components, such as a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131. Reference to display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays. Display 131 may include a first display and a second display. The first display may be a left-eye display, and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for rendering thereon. In other examples, the first and second displays may receive the same frames for rendering on them. In yet another example, the results of graphics processing may not be displayed on the device; for example, the first and second displays may not receive any frames for rendering on them. Alternatively, frames or graphics processing results may be transmitted to another device. In some respects, this can be referred to as segmented rendering.

[0032] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing, such as in graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a display processor, such as display processor 127, to perform one or more display processing techniques on one or more frames generated by processing unit 120 before being rendered by one or more displays 131. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise render the frames processed by display processor 127. In some examples, one or more displays 131 may include one or more of the following: liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, projection display device, augmented reality display device, virtual reality display device, head-mounted display, or any other type of display device.

[0033] External memory (such as system memory 124) of processing unit 120 and content encoder / decoder 122 may be accessible to processing unit 120 and content encoder / decoder 122. For example, processing unit 120 and content encoder / decoder 122 may be configured to read from and / or write to external memory (such as system memory 124). Processing unit 120 and content encoder / decoder 122 may be communicatively coupled to system memory 124 via a bus. In some examples, processing unit 120 and content encoder / decoder 122 may be communicatively coupled to each other via a bus or a different connection.

[0034] Content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store the received encoded or decoded graphic content. Content encoder / decoder 122 can be configured to receive encoded or decoded graphic content, for example, from system memory 124 and / or communication interface 126 in the form of encoded pixel data. Content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0035] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media or optical storage media, or any other type of memory.

[0036] According to some examples, internal memory 121 or system memory 124 may be a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be construed as meaning that internal memory 121 or system memory 124 is not movable or that its contents are static. As an example, system memory 124 may be removed from device 104 and moved to another device. As another example, system memory 124 may be not removable from device 104.

[0037] Processing unit 120 may be a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose GPU (GPGPU), or any other processing unit configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In some examples, processing unit 120 may reside on a graphics card mounted in a port on the motherboard of device 104, or may otherwise be incorporated into a peripheral device configured to interact with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is implemented in part in software, then processing unit 120 may store instructions for the software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 121), and may use one or more processors to execute the instructions in hardware to perform the technology of the present invention. Any of the foregoing, including hardware, software, and combinations of hardware and software, can be considered as one or more processors.

[0038] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is implemented in part in software, then the content encoder / decoder 122 may store instructions for the software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 123), and the instructions may be executed in hardware by one or more processors to perform the technology of the present invention. Any of the foregoing, including hardware, software, combinations of hardware and software, etc., can be considered as one or more processors.

[0039] In some aspects, the content generation system 100 may include an optional communication interface 126. Communication interface 126 may include a receiver 128 and a transmitter 130. Receiver 128 may be configured to perform any of the receiving functions described herein with respect to device 104. Additionally, receiver 128 may be configured to receive information from another device, such as eye or head position information, rendering commands, or location information. Transmitter 130 may be configured to perform any of the transmitting functions described herein with respect to device 104. For example, transmitter 130 may be configured to send information to another device, which may include a request for content. Receiver 128 and transmitter 130 may be combined to form transceiver 132. In such examples, transceiver 132 may be configured to perform any of the receiving and / or transmitting functions described herein with respect to device 104.

[0040] Refer again Figure 1 In some aspects, processing unit 120 may include determining component 198 configured to receive at least one frame including frame content, wherein the at least one frame is processed after being received. Determining component 198 may also be configured to process at least one frame including frame content associated with a grid comprising a plurality of grid portions, each of the plurality of portions of the frame content being aligned with at least a portion of at least one of the plurality of grid portions. Determining component 198 may also be configured to shift the frame content relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of the plurality of grid portions. Determining component 198 may also be configured to store the shifted frame content when shifting the frame content relative to the grid, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid portion. Determining component 198 may also be configured to retrieve the shifted frame content when storing the shifted frame content. Determining component 198 may also be configured to re-shift the retrieved frame content when it is retrieved, such that the re-shifted frame content corresponds to the processed frame content. The determining component 198 can also be configured to render the re-shifted frame content when it is re-shifted. The determining component 198 can also be configured to send one or more frames including the re-shifted frame content when it is re-shifted. Although the following description focuses on graphics processing, the concepts described herein are applicable to other similar processing techniques.

[0041] As described herein, a device (such as device 104) can refer to any device, apparatus, or system configured to perform one or more of the technologies described herein. For example, a device can be a server, base station, user equipment, client device, station, access point, computer (e.g., personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer), terminal product, apparatus, telephone, smartphone, server, video game platform or console, handheld device such as a portable video game device or personal digital assistant (PDA), wearable computing device such as a smartwatch, augmented reality device or virtual reality device, non-wearable device, display or display device, television, set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the technologies described herein. The processes described herein may be described as being performed by a specific component (e.g., GPU), but in other embodiments, other components (e.g., CPU) consistent with the disclosed embodiments may be used to perform them.

[0042] GPUs can process various types of data or data packets within the GPU pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. Context register packets can be a set of global state information, such as information about global registers, shaders, or constant data, which can regulate how the graphics context will be processed. For example, a context register packet may include information about the color format. In some aspects of context register packets, there may be bits indicating which workload belongs to the context register. Furthermore, there may be multiple functions or programs running simultaneously and / or in parallel. For example, a function or program may describe an operation, such as a color mode or color format. Therefore, context registers can define multiple states of the GPU.

[0043] Context states can be used to determine how individual processing unit functions (e.g., vertex extractor (VFD), vertex shader (VS), shader processor, or geometry processor) operate, and / or in what mode the processing unit operates. To do this, the GPU can use context registers and programming data. In some aspects, the GPU can generate workloads in the pipeline based on the mode or state defined by the context registers, such as vertex or pixel workloads. Specific processing units (e.g., VFDs) can use these states to determine specific functions, such as how to assemble vertices. Because these modes or states can change, the GPU may need to modify the corresponding context. Additionally, workloads corresponding to modes or states can follow changing modes or states.

[0044] Figure 2 An example GPU 200 according to one or more technologies of the present invention is shown. For example... Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-processing engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, a level 2 (L2) cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 is shown to include processing units 220-238, but the GPU 200 may include several additional processing units. Furthermore, processing units 220-238 are merely examples, and the GPU according to this disclosure may use any combination or order of processing units. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0045] like Figure 2 As shown, the GPU can utilize a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register packets (e.g., context register packet 260) and / or draw call data packets (e.g., draw call packet 212). CP 210 can then send the context register packet 260 or the draw call data packet 212 to a processing unit or block within the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can be constructed as follows: context register of context N, draw call of context N, context register of context N+1, and draw call of context N+1.

[0046] GPUs can render images in a variety of different ways. In some cases, GPUs can render images using both tiling and / or direct rendering. In a tiling GPU, an image can be divided or separated into different parts or tiles. After the image is divided, each part or tile can be rendered individually. A tiling GPU can divide a computer graphics image into a grid format, so that each part of the grid (i.e., a tile) is rendered individually. In some aspects, the image can be divided into different grids or tiles during a binning pass. In some aspects, a visibility stream can be constructed during a binning pass, where visible primitives or draw calls can be identified. In contrast to tiling, direct rendering does not divide the frame into smaller grids or tiles. Instead, in direct rendering, the entire frame is rendered in a single time. Additionally, some types of GPUs allow both tiling and direct rendering.

[0047] In some respects, a GPU can apply drawing or rendering processes to different lattices or tiles. For example, a GPU can render to a lattice and perform all drawing on the primitives or pixels within that lattice. During the rendering to a lattice process, the render target may reside in GMEM. In some cases, after rendering to a lattice, the contents of the render target can be moved to system memory, and GMEM can be freed for rendering the next lattice. Alternatively, a GPU can render to another lattice and perform drawing on the primitives or pixels within that lattice. Thus, in some respects, there may be a small number of lattices, such as four, that cover all drawing on a surface. Furthermore, a GPU can cycle through all drawing in a lattice but perform drawing on visible drawing calls (i.e., drawing calls that include visible geometry). In some respects, a visibility stream can be generated, for example, during a lattice pass to determine the visibility information of each primitive in an image or scene. For example, the visibility stream can identify whether a particular primitive is visible. In some respects, this information can be used to remove (e.g., during a rendering pass) invisible primitives. Furthermore, at least some of the primitives marked as visible can be rendered in a rendering pass.

[0048] In some aspects of tiling rendering, multiple processing stages or passes can exist. For example, rendering can be performed in two passes (e.g., a visibility or grid visibility pass and a render or grid render pass). During the visibility pass, the GPU can input a rendering workload, record the positions of primitives or triangles, and then determine which primitives or triangles fall into which grid or region. In some aspects of the visibility pass, the GPU can also identify or mark the visibility of each primitive or triangle in the visibility stream. During the render pass, the GPU can input the visibility stream and process one grid or region at a time. In some aspects, the visibility stream can be analyzed to determine which primitives or primitive vertices are visible or invisible. Thus, visible primitives or primitive vertices can be processed. By doing so, the GPU can reduce the unnecessary workload of processing or rendering invisible primitives or triangles.

[0049] In some aspects, certain types of primitive geometry can be processed during visibility passes, such as position-only geometry. Additionally, depending on the position or location of a primitive or triangle, primitives can be classified into different grids or regions. In some cases, classifying primitives or triangles into different grids can be performed by determining the visibility information of these primitives or triangles. For example, the GPU can determine or write visibility information for each primitive in each grid or region, for example, into system memory. This visibility information can be used to determine or generate a visibility stream. In a rendering pass, the primitives in each grid can be rendered individually. In these cases, the visibility stream can be retrieved from memory used to discard primitives that are not visible to that grid.

[0050] Some aspects of the GPU or GPU architecture can provide several different options for rendering (e.g., software rendering and hardware rendering). In software rendering, the driver or CPU can process each view... Figure 1 The entire frame geometry is copied multiple times. Additionally, some different states may change depending on the viewpoint. Therefore, in software rendering, the software can copy the entire workload by changing some states that can be used to render each viewpoint in the image. In some respects, this can introduce added overhead because the GPU may submit the same workload multiple times for each viewpoint in the image. In hardware rendering, the hardware or GPU can be responsible for copying or processing the geometry of each viewpoint in the image. Therefore, the hardware can manage the copying or processing of primitives or triangles for each viewpoint in the image.

[0051] Figure 3 An image or surface 300 is shown, comprising multiple primitives divided into multiple grids. For example... Figure 3 As shown, the image or surface 300 includes region 302, which includes primitives 321, 322, 323, and 324. Primitives 321, 322, 323, and 324 are divided or placed into different grids, such as grids 310, 311, 312, 313, 314, and 315. Figure 3 An example of tiled rendering using primitives 321-324 from multiple viewpoints is shown. For example, primitives 321-324 are in a first viewpoint 350 and a second viewpoint 351. In this way, a GPU that processes or renders an image or surface 300 including region 302 can utilize multiple viewpoints or multi-view rendering.

[0052] As indicated in this article, GPUs or graphics processing units can use a tiling rendering architecture to reduce power consumption or save memory bandwidth. As further stated above, this rendering method divides the scene into multiple grids, including visibility iterations that identify triangles visible in each grid. Therefore, in tiling rendering, the entire screen can be divided into multiple grids or tiles. The scene can then be rendered multiple times, for example, once or multiple times for each grid.

[0053] In graphics rendering, some graphics applications may render to a single target (i.e., the render target) once or multiple times. For example, in graphics rendering, the frame buffer on system memory may be updated multiple times. The frame buffer may be part of memory or random access memory (RAM), such as containing bitmaps or storage to help store the GPU's display data. The frame buffer may also be a memory buffer containing complete data frames. Alternatively, the frame buffer may be a logical buffer. In some aspects, updating the frame buffer may be performed in grid or tile rendering, where, as discussed above, the surface is divided into multiple grids or tiles, and then each grid or tile may be rendered individually. Furthermore, in tiling rendering, the frame buffer may be divided into multiple grids or tiles.

[0054] The graphics processing aspect is associated with several different types of content, such as virtual reality (VR) content, augmented reality (AR) content, and / or extended reality (XR) content. In VR content, the content displayed at the user's or client device can correspond to augmented content, such as content rendered at the host device, server, or client device. In AR or XR content, a portion of the content displayed at the client device can correspond to real-world content, such as real-world objects, and a portion of the content can be augmented content. Furthermore, augmented content and real-world content can be displayed in optical or video perspective devices, allowing the user to view both real-world objects and augmented content simultaneously.

[0055] In some aspects, in VR / AR / XR mode, a client or user can view VR / AR / XR content using a head-mounted display (HMD) or headset. For example, the HMD or headset can utilize two outward-pointing external cameras to capture real-world objects. Images from the left and right cameras can also be displayed on an internal LCD or OLED display. Additionally, a pair of display lenses can be mounted in front of the display. The user can then view the external world through the display lenses, the display, and / or the external cameras. As described herein, VR / AR / XR content can be generated or viewed using a host device or server and a client or user device. In some aspects, the host device can be a mobile device, smartphone, personal computer, laptop computer, and / or any other suitable device. Additionally, the client device can be a head-mounted display (HMD), headset, and / or any other suitable device. In one configuration, the VR / AR / XR content can be a split configuration, such as an HMD tethered to a mobile device. In these configurations, VR / AR / XR computation can be handled by the mobile device, and the HMD can provide sensor data to the mobile device and provide a display to the user.

[0056] In some aspects of graphics processing, VR / AR / XR applications can utilize different rendering techniques to reduce rendering workload. For example, VR / AR / XR applications can utilize foveated rendering, which is a rendering technique that can be used in HMDs or headsets. Foveated rendering techniques can help reduce rendering workload by scaling content outside a certain area (e.g., the area where the user's vision is focused). That is, foveated rendering can reduce rendering workload by reducing the image quality in the user's peripheral vision (i.e., outside the area viewed by the user's fovea (the small dimple in the retina of the eye with the highest visual acuity)). Foveated rendering methods aim to mimic the sensitivity of the human eye while utilizing the lens quality used in VR / AR / XR headsets. In some aspects, eye tracking can be used to supplement or fixate on foveated rendering.

[0057] Figure 4A and 4B Figures 400 and 410 show the contents of frames 402 and 412, respectively. More specifically, Figure 4A and 4B Certain types of rendering (e.g., tile-based rendering) can include foveated rendering associated with a mesh (e.g., a tile mesh). In foveated tile-based rendering, tile granularity can be maintained when interpreted through foveated rendering. Figure 4A and 4B As shown, the content rendered from the foveated point is 404 ( Figure 4A The circle in the middle can be converted to content 414 ( Figure 4B (The tile type in the tile grid). Thus... Figure 4A The focus of the content can be shifted to Figure 4B Some tiles in the tile grid. Figure 4A and 4B In the text, the dashed portion of the frame content (e.g., content 404 and 414) corresponds to higher quality or resolution; that is, it corresponds to the user's focus area.

[0058] In some respects, when the foveation point is associated with a grid, it can become difficult to consistently move the user's focus area. For example, moving the user's focus can align the foveation point with the grid. Furthermore, this can lead to significant variations in quality during transitions—i.e., noticeable or "explosive" quality changes. These types of transitions used in foveated rendering can also be difficult to pinpoint accurately on specific features. For example, certain types of user interface (UI) elements (e.g., text-based UI elements) may require higher rendering quality in peripheral areas than is typically provided in foveated rendering. Additionally, eye tracking (i.e., moving the focus along with the user's eye focus) utilized by foveated rendering can be difficult to execute smoothly. In some respects, certain content may need to be rendered multiple times, which contrasts with the expectation of fast rendering in VR / AR / XR applications. Therefore, reducing rendering time in VR / AR / XR applications associated with foveated rendering can be beneficial.

[0059] In some cases, adjusting the mesh (e.g., tile mesh) during foveated rendering processing can have a high probability of negatively impacting performance. For example, adjusting the shape or area of ​​regions with different qualities during foveated rendering can lead to unpredictable rendering performance. Based on the above, providing consistent visual quality can be beneficial when utilizing foveated rendering. Smoothly performing eye tracking used with foveated rendering can also be beneficial. Furthermore, adjusting regions associated with foveated rendering without negatively impacting performance can be beneficial. For example, adjusting the shape or area of ​​regions with different qualities during foveated rendering while maintaining consistent rendering performance can be beneficial.

[0060] When utilizing foveated rendering, aspects of this disclosure can provide consistent visual quality. Furthermore, aspects of this disclosure can smoothly and consistently perform certain functions used with foveated rendering, such as eye tracking. Aspects of this disclosure can also adjust regions or meshes associated with foveated rendering without negatively impacting performance observed by the user. For example, aspects of this disclosure can adjust the shape or area of ​​regions with different qualities during foveated rendering while maintaining consistent rendering performance. Additionally, aspects of this disclosure can reduce rendering time for foveated rendering in VR / AR / XR applications.

[0061] To maintain consistent rendering performance when adjusting the shape / area of ​​regions with different qualities, aspects of this disclosure can offset or adjust the area of ​​a mesh or tile, i.e., utilize mesh offset or tile offset. In mesh offset or tile offset, aspects of this disclosure can shift the content on the mesh or tile grid by a desired offset. For example, the mesh or tile grid can be of any suitable size, including conceptually infinite sizes. In some cases, when offsetting or adjusting a mesh or tile region, aspects of this disclosure may not shift the input focus itself. By maintaining the input focus, this keeps the interpretation of the foveation parameters unchanged. Furthermore, this allows aspects of this disclosure to utilize viewport shifts at the pixel granularity of the content.

[0062] In some respects, from the perspective of frame content, offsetting or adjusting a grid or tile region can be functionally equivalent to grid offset, such as tile offset of a tile grid. For example, aspects of this disclosure can shift frame content relative to a grid. Aspects of this disclosure can also store the frame content after it has been shifted. After shifting relative to the grid and storing the frame content, aspects of this disclosure can undo the content shift by moving the frame content back to the surface origin. In some cases, for certain types of images, such as sparse images, texture sampling offset can be used to recover the original frame content.

[0063] Various aspects of this disclosure can smoothly shift frame content relative to a grid or tile grid. Therefore, this disclosure can utilize tile-based rendering techniques, but shift the frame content compared to a tile grid. Thus, when shifted or adjusted relative to the tile grid, the high-quality focus area of ​​the foveated rendering can remain the same. For example, the content or high-quality focus area can maintain the same shape, but with an offset shifted on the tile grid. Furthermore, when the shifted content is stored in system memory, the content position can be restored and the conversion can be undone. Therefore, from the perspective of an image or frame, the tile grid can be shifted. Thus, by shifting image content, the tile grid can appear to move (even though the grid itself has not actually moved), so the effect can be that the tile grid is shifted. Additionally, the aforementioned shifting or adjustment of frame content or image content can be applied to operations other than tile rendering. For example, the aforementioned shifting of frame content and the granularity of the shift can be applied to different types of rendering operations or shifting operations, such as operations with limitations on granularity.

[0064] Figure 5A and 5B Figures 500 and 510 show the contents of frames 502 and 512, respectively. Figure 5A and 5B It shows certain types of rendering, such as tile-based rendering, which utilizes foveated rendering with a grid (e.g., a tile grid). Figure 5AThe original position of frame content 504 relative to the tile grid is depicted. Figure 5B This shows the shift position of frame content 514 relative to the tile grid, that is, after the frame content has already been shifted relative to the grid. Therefore, Figure 5A and 5B The above example illustrates the offset of frame content across the block grid.

[0065] Various aspects of this disclosure can also reduce memory usage when shifting or adjusting frame content relative to a tile grid. Certain types of images (e.g., sparse images) and storage methods may have alignment constraints that prevent content from returning to its original position. Other methods may require excessive allocation of system memory. In contrast, various aspects of this disclosure can shift and re-shift content while maintaining or reducing memory usage. In some cases, since the behavior can be based on tile granularity, various aspects of this disclosure can define or wrap specific offsets based on tile size (i.e., at the edges of tiles). Therefore, various aspects of this disclosure can define shift offsets or set limits on areas in which content can be shifted. Various aspects of this disclosure can also shift or adjust a scaling factor by an amount, such as a percentage of the tile size or the full tile size. Furthermore, various aspects of this disclosure can shift the user's viewpoint by an amount, such as a percentage of the tile size or the full tile size. This method can utilize a small amount of additional memory for storage, such as the amount of additional memory used for storage equal to the tile size.

[0066] Figure 6A and 6B Figures 600 and 610 show the shifted content within frames 602 and 612, respectively. Figure 6A It shows the origin from content 604 (e.g., Figure 6A The circle in the middle) is the expected offset to the shifted content 606. Figure 6B The desired offset of content 614, including shifted content 616 and shifted content 618, is described using unaligned shifting. More specifically, Figure 6B This demonstrates the use of small unaligned shifts ( Figure 6B The solid arrow in the middle) is used to achieve the desired offset. Figure 6A (arrows in the image), but tiles that change the desired behavior are aligned with the origin (e.g., arrow ...). Figure 6B (circles in the text) to achieve effective offset, that is, Figure 6B The solid arrow in the middle plus Figure 6B The offset of the dashed arrow in the text. Figure 6A and 6B The present disclosure illustrates a shift offset region in which the contents may be shifted, or which may be enclosed or defined by various aspects of the disclosure.

[0067] To achieve the aforementioned shift of frame content relative to the tile grid, aspects of this disclosure can perform several different steps or processes. For example, aspects of this disclosure can increase the allocation of system memory by a certain amount. Aspects of this disclosure can also increase the size of the grid or tile grid, i.e., increase the grid width and / or height. Furthermore, aspects of the invention can offset the gaze point scaling factor read to adapt the content shift to a certain amount. Aspects of this disclosure can also perform content shifting based on a tile offset or a percentage of the tile size. Additionally, aspects of this disclosure can store the content after the content shift offset.

[0068] In one example, when using a tile grid with tiles of a specific size (e.g., tiles with a width of 96 pixels), aspects of this disclosure can shift the content by a certain number of pixels, for example, 200 pixels. In doing so, aspects of this disclosure can perform the following steps (1)-(5): (1) Expanding the system memory allocation by 96 pixels in each direction. (2) Expanding the tile grid width and height by 1 tile in each direction. (3) Offsetting the foveation scaling factor read to accommodate the shift amount. To do this, the foveation scaling factor read can be offset by 2 tiles, for example, expanded to tile size (200 / 96)*96 = 192 pixels. Furthermore, if the offset is outside the tile grid, i.e., outside the boundary, the offset can be defined or clamped to the edge value of the tile grid. (4) Shifting the viewport by a certain number of pixels, for example, taking (tile size - (offset % tile size) % tile size). Since there is a 192-pixel offset in shifting the scaling factor over two tiles, this can result in an effective 200-pixel offset. Therefore, a 200-pixel shift corresponds to an 8-pixel viewport shift. After this, the content can be rendered. (5) The content is stored after being shifted by the offset, i.e., the shifted offset is stored. In doing so, aspects of this disclosure can shift the offset by a specific amount to move the content on storage. This avoids allocating the entire grid-size system memory surface.

[0069] Various aspects of this disclosure may include numerous benefits or advantages. For example, aspects of this disclosure may allow frame content to be smoothly shifted. Figure 7A and 7B Figures 700 and 710 are relative to the content of the grid-shifted frame, respectively. For example, Figure 7A and 7B The content movement over time 702 and 712 is depicted respectively. Figure 7A The image shows the shifted frame content without using the shifting techniques described above. For example, Figure 7A The content movement 702 is stepped and clumsy. Figure 7B The image shows the shifted frame content when the aforementioned shifting techniques of various aspects of this disclosure are utilized. For example, Figure 7BThe movement of content within 712 is smooth and gradual. For example... Figure 7A and 7B As shown, the aforementioned shifting techniques of the aspects of this disclosure can shift frame content more smoothly. Therefore, in the aspects of this disclosure, users can observe higher quality content shifting utilizing shifting techniques.

[0070] Figure 8 This is a communication flowchart 800 based on one or more techniques of this disclosure for graphical processing. For example... Figure 8 As shown, Figure 800 includes example communication between GPU component 802, GPU component 804 and memory or buffer 806 according to one or more technologies of this disclosure.

[0071] At 810, GPU component 802 may receive at least one frame, such as frame 812, which includes frame content, and the at least one frame is processed after being received.

[0072] At 820, GPU component 802 can process at least one frame (e.g., frame 812) including frame content associated with a grid comprising multiple grid portions, each of the multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid portions. The grid may be a tiled grid, and the multiple grid portions may be multiple tiles, such that each of the multiple portions of the frame content is aligned with at least a portion of at least one of the multiple tiles.

[0073] At 830, GPU component 802 can shift frame content relative to the grid such that at least a portion of a plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of a plurality of grid portions.

[0074] In some aspects, the shifted frame content can be associated with at least one of a shift distance or a shift region. The shift distance or shift region of the shifted frame content can be defined by an offset region in the grid. Furthermore, at least some of the shifted frame content can correspond to regions smaller than the grid portion. The granularity of the shifted frame content can be smaller than the granularity of the grid portion. Additionally, each of the plurality of portions of the frame content can be shifted separately, or all of the plurality of portions of the frame content can be shifted together. A first portion of the frame content can correspond to a first portion of the display, and a second portion of the frame content can correspond to a second portion of the display, such that the first portion and the second portion of the frame content are shifted separately. Furthermore, the frame content can be shifted relative to the grid by a graphics processing unit (GPU).

[0075] At 840, GPU component 802 can store the shifted frame content relative to a grid when shifting the frame content, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid segment. The shifted frame content can be stored in at least one of memory, graphics processing unit (GPU) memory, system memory, or a buffer. Furthermore, if stored in system memory, the shifted frame content can be shifted by a different amount compared to being stored in GPU memory. For example, if stored in system memory, the shifted frame content can be shifted by a smaller amount compared to being stored in GPU memory. Additionally, the frame content can be shifted while being stored, such that the frame content is shifted and stored simultaneously.

[0076] At 850, GPU component 802 can retrieve the shifted frame content while storing the shifted frame content.

[0077] At 860, GPU component 802 can re-shift the retrieved frame content when it is retrieved, so that the re-shifted frame content corresponds to the processed frame content. In some cases, the re-shifted frame content can be equivalent to the processed frame content.

[0078] At 870, GPU component 802 can render the re-shifted frame content when the frame content is re-shifted.

[0079] At 880, GPU component 802 may send one or more frames, such as frame 882, that include the re-shifted frame content when the frame content is re-shifted.

[0080] Figure 9 This is a flowchart 900 illustrating an example method of graphics processing according to one or more techniques of the present invention. The method may be performed by means of a device for graphics processing, a GPU, another graphics processor, a GPU pipeline, a wireless communication device, and / or a combination thereof. Figures 1 to 8 The example uses any device capable of performing graphics processing.

[0081] At position 902, the device can receive at least one frame including frame content, wherein the at least one frame is processed after being received. For example, as... Figure 8 As described in 810, GPU component 802 can receive at least one frame including frame content, wherein the at least one frame is processed after being received. Furthermore, Figure 1 The processing unit 120 can execute step 902.

[0082] At 904, the apparatus can process at least one frame comprising frame content associated with a grid comprising multiple grid portions, each of the multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid portions. For example, as in Figure 8 As described in 820, GPU component 802 can process at least one frame including frame content associated with a grid comprising multiple grid portions, each of the multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid portions. Furthermore, Figure 1 The processing unit 120 can perform step 904. The grid can be a tile grid, and the multiple grid portions can be multiple tiles, such that each of the multiple portions of the frame content is aligned with at least a portion of at least one of the multiple tiles.

[0083] At 906, the device can shift the frame content relative to the grid such that at least a portion of a plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of a plurality of grid portions. For example, as in Figure 8 As described in 830, GPU component 802 can shift frame content relative to a grid such that at least a portion of a plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of a plurality of grid portions. Furthermore, Figure 1 The processing unit 120 can execute step 906.

[0084] In some aspects, the shifted frame content can be associated with at least one of a shift distance or a shift region. The shift distance or shift region of the shifted frame content can be defined by an offset region in the grid. Furthermore, at least some of the shifted frame content can correspond to regions smaller than the grid portion. The granularity of the shifted frame content can be smaller than the granularity of the grid portion. Additionally, each of the plurality of portions of the frame content can be shifted separately, or all of the plurality of portions of the frame content can be shifted together. A first portion of the frame content can correspond to a first portion of the display, and a second portion of the frame content can correspond to a second portion of the display, such that the first portion and the second portion of the frame content are shifted separately. Furthermore, the frame content can be shifted relative to the grid by a graphics processing unit (GPU).

[0085] At 908, the apparatus can store the shifted frame content relative to the grid when shifting the frame content, the shifted frame content including at least a portion of the frame content that is aligned with at least a portion of at least one different grid section. For example, as Figure 8 As described in 840, GPU component 802 can store shifted frame content relative to a grid, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid section. Furthermore, Figure 1 The processing unit 120 can execute step 908. The shifted frame content can be stored in at least one of memory, graphics processing unit (GPU) memory, system memory, or a buffer. Furthermore, if stored in system memory, the shifted frame content can be shifted by a different amount compared to being stored in GPU memory. For example, if stored in system memory, the shifted frame content can be shifted by a smaller amount compared to being stored in GPU memory. Additionally, the frame content can be shifted while being stored, such that the frame content is shifted and stored simultaneously.

[0086] At point 910, the device can retrieve the shifted frame content while storing it. For example, as... Figure 8 As described in 850, GPU component 802 can retrieve the shifted frame content while storing the shifted frame content. Furthermore, Figure 1 The processing unit 120 can execute step 910.

[0087] At position 912, the device can re-shift the retrieved frame content when it is found, so that the re-shifted frame content corresponds to the processed frame content. For example, as... Figure 8 As described in 860, GPU component 802 can re-shift the retrieved frame content when it is retrieved, so that the re-shifted frame content corresponds to the processed frame content. Furthermore, Figure 1 The processing unit 120 can execute step 912. In some cases, the re-shifted frame content can be equivalent to the processed frame content.

[0088] At position 914, the device can render the re-shifted frame content when it is re-shifted. For example, as... Figure 8 As described in 870, GPU component 802 can render the re-shifted frame content when the frame content is re-shifted. Furthermore, Figure 1 The processing unit 120 can execute step 914.

[0089] At 916, the device can transmit one or more frames including the re-shifted frame content when re-shifting the frame content. For example, as... Figure 8 As described in 880, GPU component 802 can send one or more frames including the re-shifted frame content when re-shifting the frame content. Furthermore, Figure 1 The processing unit 120 can execute step 916.

[0090] In this configuration, a method or apparatus for graphics processing is provided. The apparatus may be a GPU, a graphics processing unit, or some other processor capable of performing graphics processing. In various aspects, the apparatus may be a processing unit 120 within device 104, or may be some other hardware within device 104 or another device. The apparatus (e.g., processing unit 120) may include: means for processing at least one frame comprising frame content associated with a grid comprising a plurality of grid portions, each of the plurality of portions of the frame content being aligned with at least a portion of at least one of the plurality of grid portions; means for shifting the frame content relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of the plurality of grid portions; means for storing the shifted frame content when shifting the frame content relative to the grid, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid portion; means for retrieving the shifted frame content when storing the shifted frame content; means for re-shifting the retrieved frame content when the frame content is retrieved such that the re-shifted frame content corresponds to the processed frame content; means for rendering the re-shifted frame content when the frame content is re-shifted; means for transmitting the re-shifted frame content when the frame content is re-shifted; and means for receiving at least one frame comprising frame content, wherein the at least one frame is processed after being received.

[0091] The subjects described herein can be implemented to achieve one or more benefits or advantages. For example, the described graphics processing techniques can be used by a GPU, a graphics processing unit, or some other processor capable of performing graphics processing to implement the foveated rendering content shifting techniques described herein. This can also be implemented at a lower cost compared to other graphics processing techniques. Furthermore, the graphics processing techniques of this invention can improve or accelerate data processing or execution. In addition, the graphics processing techniques of this invention can improve resource or data utilization and / or resource efficiency. Furthermore, aspects of this disclosure can utilize foveated rendering content shifting to improve memory bandwidth efficiency and / or increase processing speed at the GPU.

[0092] It should be understood that the specific order or hierarchy of boxes in the disclosed process / flowchart is illustrative of the exemplary method. It should be understood that the specific order or hierarchy of boxes in the process / flowchart may be rearranged based on design preferences. Furthermore, some boxes may be combined or omitted. The appended method claims present the elements of the individual boxes in a sample order and are not intended to limit one to the specific order or hierarchy presented.

[0093] The preceding description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but rather to be consistent with the full scope of the language of the claims, wherein, unless specifically stated otherwise, references to elements in the singular are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0094] Unless otherwise specified, the term "some" means one or more, and the term "or" can be interpreted as "and / or," where the context does not specify otherwise. Combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, where any such combination may include one or more members or members of A, B, or C. All structural and functional equivalents of elements throughout the various aspects described in this disclosure that are known to or will be known hereafter by those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is expressly stated in the claims. Terms such as “module,” “mechanism,” “element,” and “device” are not necessarily substitutes for the term “unit.” Therefore, no claim element should be interpreted as a component plus a function unless that element is explicitly stated using the phrase “component for…”.

[0095] In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" has been used throughout the invention, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique, or other module described herein is implemented in software, then the function, processing unit, technique, or other module described herein may be stored on or transmitted via a computer-readable medium as one or more instructions or code.

[0096] According to this disclosure, unless otherwise indicated by the context, the term "or" may be interpreted as "and / or". Additionally, while phrases such as "one or more" or "at least one" may be used for some features disclosed herein but not others, features not using such language may be interpreted as having this meaning implied by the context unless otherwise indicated.

[0097] In one or more instances, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" has been used throughout the invention, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique, or other module described herein is implemented in software, then the function, processing unit, technique, or other module described herein may be stored on or transmitted via a computer-readable medium as one or more instructions or code. A computer-readable medium may include a computer data storage medium or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another. In this way, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0098] The code can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), arithmetic logic units (ALUs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, the techniques can be fully implemented in one or more circuit or logic elements.

[0099] The technology of this invention can be implemented in various devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs, such as chipsets. Various components, modules, or units are described in this invention to emphasize functional aspects of a device configured to perform the disclosed technology, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in any hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the technology described herein. Moreover, the technology can be fully implemented in one or more circuit or logic elements.

[0100] The following aspects are illustrative only and may be combined with, but are not limited to, other aspects or teachings described herein.

[0101] Aspect 1 is a method for graphics processing. The method includes: processing at least one frame comprising frame content associated with a grid comprising a plurality of grid portions, each of the plurality of portions of the frame content being aligned with at least a portion of at least one of the plurality of grid portions; shifting the frame content relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid portion of the plurality of grid portions; and storing the shifted frame content when shifting the frame content relative to the grid, the shifted frame content including at least a portion of the frame content aligned with at least a portion of at least one different grid portion.

[0102] Aspect 2 is the method according to aspect 1, wherein the shifted frame content is associated with at least one of the shift distance or shift region.

[0103] Aspect 3 is the method according to any one of Aspects 1 and 2, wherein the shift distance or shift region of the shifted frame content is defined by an offset region in the grid.

[0104] Aspect 4 is the method according to any one of aspects 1 to 3, wherein at least some of the shifted frame content corresponds to regions smaller than the grid portion.

[0105] Aspect 5 is the method according to any one of aspects 1 to 4, wherein the granularity of the shifted frame content is smaller than the granularity of the region of the grid portion.

[0106] Aspect 6 is the method according to any one of aspects 1 to 5, further comprising: retrieving the shifted frame content while storing the shifted frame content.

[0107] Aspect 7 is the method according to any one of aspects 1 to 6, further comprising: when frame content is retrieved, re-shifting the retrieved frame content such that the re-shifted frame content corresponds to the processed frame content.

[0108] Aspect 8 is the method according to any one of aspects 1 to 7, wherein the re-shifted frame content is equivalent to the processed frame content.

[0109] Aspect 9 is the method according to any one of aspects 1 to 8, further comprising: rendering the re-shifted frame content when re-shifting the frame content.

[0110] Aspect 10 is the method according to any one of aspects 1 to 9, further comprising: transmitting one or more frames including the re-shifted frame content when re-shifting the frame content.

[0111] Aspect 11 is the method according to any one of aspects 1 to 10, wherein the shifted frame content is stored in at least one of a memory, a graphics processing unit (GPU) memory, a system memory, or a buffer.

[0112] Aspect 12 is the method according to any one of aspects 1 to 11, wherein if the shifted frame content is stored in system memory, the shifted frame content is shifted by a different amount compared to being stored in GPU memory.

[0113] Aspect 13 is the method according to any one of aspects 1 to 12, wherein the shifted frame content is shifted by a smaller amount if stored in system memory compared to being stored in GPU memory.

[0114] Aspect 14 is the method according to any one of aspects 1 to 13, wherein the frame content is shifted while storing the frame content, such that the frame content is shifted and stored simultaneously.

[0115] Aspect 15 is the method according to any one of aspects 1 to 14, wherein the grid is a tile grid and the plurality of grid portions are a plurality of tiles, such that each of the plurality of portions of the frame content is aligned with at least a portion of at least one of the plurality of tiles.

[0116] Aspect 16 is the method according to any one of aspects 1 to 15, further comprising: receiving at least one frame including frame content, wherein the at least one frame is processed after being received.

[0117] Aspect 17 is the method according to any one of aspects 1 to 16, wherein each of the plurality of portions of the frame content is shifted separately, or all of the plurality of portions of the frame content are shifted together.

[0118] Aspect 18 is the method according to any one of aspects 1 to 17, wherein a first portion of the frame content corresponds to a first portion of the display, and a second portion of the frame content corresponds to a second portion of the display, such that the first portion of the frame content and the second portion of the frame content are shifted separately.

[0119] Aspect 19 is the method according to any one of aspects 1 to 18, wherein the frame content is shifted relative to the grid by a graphics processing unit (GPU).

[0120] Aspect 20 is an apparatus for graphics processing, comprising at least one processor coupled to a memory and configured to implement the method as described in any one of aspects 1 to 19.

[0121] Aspect 21 is an apparatus for graphics processing, comprising components for implementing the method as described in any one of aspects 1 to 19.

[0122] Aspect 22 is a computer-readable medium storing computer-executable code that, when executed by at least one processor, causes the at least one processor to implement the method as described in any one of aspects 1 to 19.

Claims

1. An apparatus for graphics processing, comprising: Memory; as well as At least one processor, said at least one processor being coupled to the memory and configured to: The processing includes at least one frame of frame content associated with a grid comprising multiple grid partitions, each of multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid partitions; The frame content is shifted relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid partition among the plurality of grid partitions, wherein the shifted frame content is associated with a shift region, and wherein the shift region of the shifted frame content is defined by an offset region in the grid. Based on the at least one processor being configured to shift the frame content relative to the grid, the shifted frame content is stored, the shifted frame content including at least a portion of the frame content that is aligned with at least said portion of the at least one different grid partition; Based on the at least one processor being configured to store the shifted frame content, the shifted frame content is retrieved; and Based on the at least one processor being configured to retrieve the frame content, the retrieved frame content is re-shifted such that the re-shifted frame content corresponds to the processed frame content. Among them, at least some of the shifted frame content corresponds to regions smaller than the regions of the grid partitions in the plurality of grid partitions.

2. The apparatus according to claim 1, wherein, The granularity of the shifted frame content is smaller than the granularity of the grid partition region.

3. The apparatus according to claim 1, wherein, The re-shifted frame content is equivalent to the processed frame content.

4. The apparatus according to claim 1, wherein, The at least one processor is further configured to: The at least one processor is configured to re-shift the frame content, and the re-shifted frame content is rendered.

5. The apparatus according to claim 1, wherein, The at least one processor is further configured to: Based on the at least one processor being configured to re-shift the frame content, one or more frames including the re-shifted frame content are transmitted.

6. The apparatus according to claim 1, wherein, The at least one processor is configured to store the shifted frame content in at least one of a first memory, a graphics processing unit (GPU) memory, a system memory, or a buffer.

7. The apparatus according to claim 6, wherein, The at least one processor is configured to shift the shifted frame content by a different amount compared to when the at least one processor is configured to store the shifted frame content in the GPU memory.

8. The apparatus according to claim 7, wherein, The at least one processor is configured to shift the shifted frame content by a smaller amount compared to if the at least one processor is configured to store the shifted frame content in the system memory.

9. The apparatus according to claim 1, wherein, The at least one processor is configured to: shift the frame content while storing the frame content, such that the at least one processor is configured to simultaneously shift and store the frame content.

10. The apparatus according to claim 1, wherein, The grid is a tile grid, and the plurality of grid partitions are plurality of tiles, such that each of the plurality of portions of the frame content is aligned with at least a portion of at least one of the plurality of tiles.

11. The apparatus according to claim 1, wherein, The at least one processor is further configured to: Receive the at least one frame including the frame content, wherein the at least one processor is configured to process the at least one frame after the at least one processor is configured to receive the at least one frame.

12. The apparatus according to claim 1, wherein, The at least one processor is configured to separately shift each of the plurality of portions of the frame content.

13. The apparatus according to claim 1, wherein, The first portion of the frame content corresponds to a first portion of the display, and the second portion of the frame content corresponds to a second portion of the display, such that the at least one processor is configured to separately shift the first portion of the frame content and the second portion of the frame content.

14. The apparatus according to claim 1, wherein, The at least one processor is configured to shift all of the plurality of portions of the frame content together, and the at least one processor is configured to shift the frame content relative to the grid at the graphics processing unit (GPU).

15. A graphics processing method, comprising: The processing includes at least one frame of frame content associated with a grid comprising multiple grid partitions, each of multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid partitions; The frame content is shifted relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid partition among the plurality of grid partitions, wherein the shifted frame content is associated with a shift region, and wherein the shift region of the shifted frame content is defined by an offset region in the grid. Based on shifting the frame content relative to the grid, the shifted frame content is stored, the shifted frame content including at least a portion of the frame content that is aligned with at least said portion of the at least one different grid partition; Retrieve the shifted frame content based on the stored shifted frame content; and Based on the retrieved frame content, the retrieved frame content is re-shifted so that the re-shifted frame content corresponds to the processed frame content, wherein the re-shifted frame content is equivalent to the processed frame content. Among them, at least some of the shifted frame content corresponds to regions smaller than the regions of the grid partitions in the plurality of grid partitions.

16. The method according to claim 15, wherein, The granularity of the shifted frame content is smaller than the granularity of the region of the grid partition among the plurality of grid partitions.

17. The method according to claim 15, wherein, The re-shifted frame content is equivalent to the processed frame content.

18. The method of claim 15, further comprising: Based on the re-shifted frame content, render the re-shifted frame content; as well as Based on the re-shifted frame content, one or more frames including the re-shifted frame content are transmitted.

19. The method according to claim 15, wherein, The shifted frame content is stored in at least one of memory, graphics processing unit (GPU) memory, system memory, or a buffer, wherein the shifted frame content is shifted by a different amount compared to if it were stored in the system memory, compared to if it were stored in the GPU memory.

20. The method of claim 15, wherein, The frame content is shifted while being stored, so that the frame content is both shifted and stored simultaneously.

21. The method according to claim 15, wherein, The grid is a tile grid, and the plurality of grid partitions are plurality of tiles, such that each of the plurality of portions of the frame content is aligned with at least a portion of at least one of the plurality of tiles.

22. The method of claim 15, further comprising: Receive the at least one frame including the frame content, wherein the at least one frame is processed after being received.

23. An apparatus for graphics processing, comprising: A component for processing at least one frame comprising frame content associated with a grid comprising multiple grid partitions, each of multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid partitions; A component for shifting the frame content relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid partition among the plurality of grid partitions, wherein the shifted frame content is associated with a shift region, and wherein the shift region of the shifted frame content is defined by an offset region in the grid. A component for storing shifted frame content based on the shift of the frame content relative to the grid, the shifted frame content including at least a portion of the frame content that is aligned with at least said portion of the at least one different grid partition; A component for retrieving the shifted frame content based on the storage shifted frame content; and A component for re-shifting the retrieved frame content based on the retrieved frame content, such that the re-shifted frame content corresponds to the processed frame content, wherein the re-shifted frame content is equivalent to the processed frame content. Among them, at least some of the shifted frame content corresponds to regions smaller than the regions of the grid partitions in the plurality of grid partitions.

24. A computer-readable medium storing computer-executable code for graphics processing, said code causing the processor, when executed by a processor, to: The processing includes at least one frame of frame content associated with a grid comprising multiple grid partitions, each of multiple portions of the frame content being aligned with at least a portion of at least one of the multiple grid partitions; The frame content is shifted relative to the grid such that at least a portion of the plurality of portions of the frame content is aligned with at least a portion of at least one different grid partition among the plurality of grid partitions, wherein, The shifted frame content is associated with a shift region, wherein the shift region of the shifted frame content is defined by an offset region in the grid; Based on shifting the frame content relative to the grid, the shifted frame content is stored, the shifted frame content including at least a portion of the frame content that is aligned with at least said portion of the at least one different grid partition; Retrieve the shifted frame content based on the stored shifted frame content; and Based on the retrieved frame content, the retrieved frame content is re-shifted so that the re-shifted frame content corresponds to the processed frame content, wherein the re-shifted frame content is equivalent to the processed frame content. Among them, at least some of the shifted frame content corresponds to regions smaller than the regions of the grid partitions in the plurality of grid partitions.

25. A computer program product comprising computer-readable instructions that, when executed by at least one processor of a means for graphics processing, cause the means to perform the graphics processing method of any one of claims 15-22.

Images