Piecewise linear irregular rasterization

Abstract

Method for rendering a geometry object in a computer-generated scene, wherein the method comprises: Dividing a screen space assigned to a display screen into multiple regions; for each region: Calculating a first sampling factor in a horizontal dimension, which represents a horizontal sampling factor for pixels localized in the region, Calculating a second sampling factor in a vertical dimension, representing a vertical sampling factor for pixels localized in the region, Calculating a first offset in the horizontal dimension, which represents a horizontal position assigned to the region, and Calculating a second offset in the vertical dimension, representing a vertical position assigned to the region; Determine that the geometry object intersects more than one region; Creating a first instance of the geometry object for a first region that intersects the geometry object; and Creating a second instance of the geometry object for a second region that intersects the geometry object, wherein the first region and the second region are arranged in the screen space adjacent to each other in at least one of the horizontal and vertical directions, wherein the first sampling factor of the first region differs from the first sampling factor of the second region, and the second sampling factor of the first region differs from the second sampling factor of the second region.

Description

BACKGROUND OF THE INVENTION Area of ​​the invention

[0001] Embodiments of the present invention generally relate to computer graphics processing and, in particular, piecewise linear irregular rasterization. Description of related techniques

[0002] Virtual reality (VR) headsets are an increasingly popular device for achieving a more immersive experience when playing computer games, watching three-dimensional (3D) movies, or viewing other 3D media content. VR headsets are worn on the user's head and block out light from the real world. They typically include two flat display screens, one for viewing by the left eye and the other for viewing by the right eye. Game content or other 3D media is transmitted to these two screens, allowing the user to see the 3D content while wearing the VR headset. Because the two screens are positioned some distance in front of the user's eyes, they cannot, on their own, provide peripheral content.In other words, the viewer sees content in front of their eyes, but does not see the content above, below, or to the sides of the two display screens.

[0003] Consequently, the VR headset also incorporates a non-linear distortion lens in front of each display screen. This lens causes the content near a specific location, such as the center of the screen, to optically converge, while causing it to optically diverge near the edges. The diverging content is then projected onto the peripheral vision of the viewer. As a result, the viewer is able to perceive the 3D content both in their peripheral vision and in front of them, leading to a more immersive virtual reality experience.

[0004] In addition to VR headsets, these techniques are also used for other widescreen rendering applications, such as curved viewing surfaces that encompass one or more display screens. Such widescreen rendering applications include curved high-definition and ultra-high-definition displays for home use and tiled multi-screen displays for industrial applications, such as control rooms. In these applications, the perceived distance between adjacent pixels changes from one section of the display screen to another, resulting in distortion artifacts similar to those observed when using VR headsets.

[0005] Although nonlinear distortion lenses offer the advantage of projecting content to the periphery of the view, this type of distortion lens has the undesirable side effect of making objects rendered on the display screens and projected into the viewer's eyes appear distorted. A common form of optical distortion is pincushion distortion, where the perceived location of a point on an object is closer to the center of the display screen relative to where the point is actually located. As a result, objects with straight lines, such as squares and rectangles, appear to have curved edges when viewed through VR glasses.To counteract this phenomenon, graphics processing units (GPUs) or other processors typically render 3D content intended for viewing with VR headsets by intentionally inducing nonlinear barrel distortion. This means that the perceived location of a point on an object is farther from the center of the display screen relative to where the point is actually located. The induced barrel distortion counteracts optical pincushion distortion, allowing the viewer to see undistorted content when wearing a VR headset.

[0006] One drawback of the above approach is that content near the center of the screen is enlarged, while content near the edges is compressed. Consequently, content directly in front of the viewer's eyes has a relatively lower resolution, resulting in a loss of detail. In other words, too few pixels are rendered in the center of the screen relative to the number of pixels needed for a high-quality viewing experience. This loss of detail can be perceived as soft, blurry, or blocky video. Conversely, content at the periphery of what the viewer can see has a higher resolution than required for high-quality viewing.In other words, more pixels are rendered near the edges of the display screen relative to the number of rendered pixels needed for a high-quality viewing experience, resulting in a waste of computing resources.

[0007] Another disadvantage of the above approach is that GPUs are usually optimized to render content using a linear raster. Because pincushion lens distortion is nonlinear, ideally the barrel distortion induced by the GPU during rendering should also be nonlinear. However, rendering using a nonlinear raster can lead to significantly lower GPU performance relative to rendering using a linear raster. Brian Guenter et al. describe a method using different layers during rendering depending on the viewing direction in "Foveated 3D graphics." (ACM Transactions on Graphics, 8TOG, 2012, Vol. 31, No. 6, pp. 1-10).

[0008] As the foregoing illustrates, what is needed in the technology is a more effective and efficient approach to rendering content for VR glasses and other widescreen rendering applications. SUMMARY OF THE INVENTION

[0009] One embodiment of the present invention provides a method for rendering a geometric object in a computer-generated scene according to claim 1. The method includes, among other things, dividing a screen space associated with a display screen into a set of regions. The method further includes, for each region: calculating a first sampling factor in a horizontal dimension, representing a horizontal sampling factor for pixels localized in the region; calculating a second sampling factor in a vertical dimension, representing a vertical sampling factor for pixels localized in the region; calculating a first offset in the horizontal dimension, representing a horizontal position associated with the region; and calculating a second offset in the vertical dimension, representing a vertical position associated with the region. The method further includes determining that the geometric object intersects more than one region.The procedure further includes creating a first instance of the geometry object for a first region that intersects the geometry object. The procedure further includes creating a second instance of the geometry object for a second region that intersects the geometry object.

[0010] Other embodiments of the present invention include, without limitation, a computer-readable medium comprising instructions for carrying out one or more aspects of the disclosed techniques, as well as a system for carrying out one or more aspects of the disclosed techniques.

[0011] One advantage of the revealed technique is that pixels located near the edge of the screen space are rendered at a lower resolution, resulting in improved performance relative to previously known widescreen rendering techniques.

[0012] It should be understood that the features mentioned above and those to be explained below can be used not only in the respective combinations specified, but also in other combinations or individually, without deviating from the scope of the present invention. Features of the aspects and embodiments mentioned above can be combined with one another in other embodiments, unless explicitly stated otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To specify how the aforementioned features of the present invention are to be understood in detail, a more detailed description of the invention, which is briefly summarized above, can be obtained with reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings merely represent typical embodiments of this invention and are therefore not to be considered as limiting its scope of protection, since the invention may permit other equally effective embodiments. Fig. Figure 1 is a block diagram representing a computer system designed to implement one or more aspects of the present invention; Fig. Figure 2 is a block diagram of a parallel processing unit located in the parallel processing subsystem of Fig. 1 is included according to one embodiment of the present invention; Fig. 3A is a block diagram of a general processing cluster used in the parallel processing unit of Fig. 2 comprising, according to one embodiment of the present invention; Fig. 3B is a concept diagram of a graphics processing pipeline used in the parallel processing unit of Fig. 2 2 can be implemented according to an embodiment of the present invention; Fig. Figure 4 is a concept diagram of a cache tile for which the graphics processing pipeline of Fig. 3B is designed to generate and process these, according to an embodiment of the present invention; Fig. Figure 5 is a detailed view of a section of the graphics processing pipeline of Fig. 3B according to an embodiment of the present invention; Fig. Figures 6A-6C illustrate exemplary scanning grids that can be implemented during piecewise irregular rasterization according to various embodiments of the present invention; wherein Fig. 6C shows a comparative example, and Fig. Figure 7 illustrates a flowchart of process steps for performing piecewise irregular rendering according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following description provides numerous specific details to facilitate a more thorough understanding of the present invention. However, a person skilled in the art will recognize that the present invention can be practiced without one or more of these specific details. System overview

[0015] Fig. Figure 1 is a block diagram illustrating a computer system 100 designed to implement one or more aspects of the present invention. As shown, the computer system 100 comprises, without limitation, a central processing unit (CPU) 102 and a system memory 104, which are connected to a parallel processing subsystem 112 via a memory bridge 105 and a communication path 113. The memory bridge 105 is further connected to an I / O (input / output) bridge 107 via a communication path 106, and the I / O bridge 107 is in turn connected to a switch 116.

[0016] In operation, the I / O bridge 107 is designed to receive user input information from input devices 108, such as a keyboard or mouse, and to forward the input information to the CPU 102 for processing via the communication path 106 and the memory bridge 105. The switch 116 is designed to provide connections between the I / O bridge 107 and other components of the computer system 100, such as a network adapter 118 and various expansion cards 120 and 121.

[0017] As also shown, the I / O bridge 107 is connected to a system disk 114, which may be configured to store content, applications, and data for use by the CPU 102 and the parallel processing subsystem 112. In general, the system disk 114 provides non-volatile storage space for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD (compact floppy disk read-only storage), DVD (digital versatility disk-ROM), Blu-ray, HD-DVD (high-definition DVD), or other magnetic, optical, or electronic storage devices. Finally, although not explicitly shown, other components, such as a universal serial bus or other port connections, compact disk drives, DVD (digital versatility disk) drives, movie recording devices, and the like, may also be connected to the I / O bridge 107.

[0018] In various embodiments, the memory bridge 105 can be a north-bridge chip and the I / O bridge 107 can be a south-bridge chip. Furthermore, the communication paths 106 and 113, as well as other communication paths within the computer system 100, can be implemented using any technically suitable protocols, including, but not limited to, AGP (Accelerated Graphics Port), HyperTransport, or other bus or point-to-point communication protocols known in the prior art.

[0019] In some embodiments, the parallel processing subsystem 112 is part of a graphics subsystem that supplies pixels to a display device 110, which may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, or the like. In such embodiments, the parallel processing subsystem 112 comprises circuits optimized for graphics or video processing, including, for example, video output circuits. As described below in Fig. As described in more detail in Section 2, such circuits can be included in one or more parallel processing units (PPUs) that are part of the parallel processing subsystem 112. In other embodiments, the parallel processing subsystem 112 includes a circuit optimized for general-purpose processing and / or computational processing. Such a circuit can itself be included in one or more PPUs that are part of the parallel processing subsystem 112 and are designed to perform such general-purpose operations and / or computations. In still other embodiments, the one or more PPUs included in the parallel processing subsystem 112 can be designed to perform graphics processing, general-purpose processing, and computational processing operations.The system memory 104 includes at least one device driver 103 designed to manage the processing operations of one or more PPUs in the parallel processing subsystem 112.

[0020] In various embodiments, the parallel processing subsystem 112 can be integrated into one or more of the other elements of Fig. 1. It can be integrated to form a single system. For example, the parallel processing subsystem 112 can be integrated with the CPU 102 and other interconnect circuitry on a single chip to form a system-on-a-chip (SoC).

[0021] It should be noted that the system shown here is for illustrative purposes only and that variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112, can be modified as needed. For example, in some embodiments, the system memory 104 could be connected directly to the CPU 102 instead of via the memory bridge 105, and other devices could communicate with the system memory 104 via the memory bridge 105 and the CPU 102. In other alternative topologies, the parallel processing subsystem 112 could be connected to the I / O bridge 107 or directly to the CPU 102 instead of via the memory bridge 105. In still other embodiments, the I / O bridge 107 and the memory bridge 105 could be integrated on a single chip instead of existing as one or more discrete components.Finally, in certain embodiments, one or more of the features described in . Fig. The components shown in Figure 1 may not be present. For example, switch 116 could be omitted, and network adapter 118 and add-on cards 120 and 121 could be directly connected to I / O bridge 107.

[0022] Fig. Figure 2 is a block diagram of a parallel processing unit (PPU) 202, which is part of the parallel processing subsystem 112 of Fig. 1 is included, according to one embodiment of the present invention. Although Fig. Since Figure 2 represents a PPU 202, the parallel processing subsystem 112 can, as indicated above, comprise any number of PPUs 202. As shown, the PPU 202 is connected to a local parallel processing (PP) memory 204. The PPU 202 and the PP memory 204 can be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or memory devices, or in any other technically feasible manner.

[0023] In some embodiments, the PPU 202 includes a graphics processing unit (GPU) that can be configured to implement a graphics rendering pipeline for performing various operations related to generating pixel data from graphics data supplied by the CPU 102 and / or the system memory 104. When processing graphics data, the PP memory 204 can be used as a graphics memory that stores one or more conventional frame buffers and, if required, one or more other render targets. Among other things, the PP memory 204 can be used to store and update pixel data and to route final pixel data or display blocks to the display device 110 for rendering. In some embodiments, the PPU 202 can also be configured for general-purpose processing and computational operations.

[0024] During operation, the CPU 102 is the master processor of the computer system 100, controlling and coordinating the operation of other system components. Specifically, the CPU 102 issues instructions that control the operation of the PPU 202. In some embodiments, the CPU 102 writes a stream of instructions for the PPU 202 to a data structure (in Fig. 1 or Fig. (2 not explicitly shown), which may reside in system memory 104, PP memory 204, or another memory location accessible to both the CPU 102 and the PPU 202. A pointer to the data structure is written to a shift buffer to initiate the processing of the stream of instructions in the data structure. The PPU 202 reads instruction streams from the shift buffer and then executes instructions asynchronously relative to the operating mode of the CPU 102. In embodiments where multiple shift buffers are created, execution priorities for each shift buffer can be specified by an application program via the device driver 103 to control the allocation of the different shift buffers.

[0025] As also shown, the PPU 202 includes an I / O (input / output) unit 205, which communicates with the rest of the computer system 100 via the communication path 113 and the memory bridge 105. The I / O unit 205 generates packets (or other signals) for transmission on the communication path 113 and also receives all incoming packets (or other signals) from the communication path 113, routing the incoming packets to appropriate components of the PPU 202. For example, commands related to processing tasks can be routed to a higher-level interface or master interface 206, while commands related to memory operations (for example, reading from and writing to the PP memory 204) are sent to a junction unit 210. The master interface 206 reads each shift buffer and sends the command stream stored in the shift buffer to a front area 212.

[0026] As previously mentioned in connection with Fig. As mentioned in Section 1, the connection of the PPU 202 to the rest of the computer system 100 can vary. In some embodiments, the parallel processing subsystem 112, which includes at least one PPU 202, is implemented as an add-on card that can be inserted into an expansion slot of the computer system 100. In other embodiments, the PPU 202 can be integrated on a single chip with a bus bridge, such as the memory bridge 105 or the I / O bridge 107. In still other embodiments, some or all elements of the PPU 202, together with the CPU 102, can be included in a single integrated circuit or system-on-a-chip (SoC).

[0027] During operation, the front area 212 sends processing tasks received from the master interface 206 to a workload distribution unit (not shown) in the task / work unit 207. The workload distribution unit receives pointers to processing tasks, which are encoded as task metadata (TMD) and stored in memory. The pointers to the TMD are contained in an instruction stream, which is stored as a shift buffer and received by the front area unit 212 from the master interface 206. Processing tasks, which can be encoded as TMD, include indices associated with the data to be processed, as well as state parameters and instructions that specify how the data is to be processed. For example, the state parameters and instructions can specify the program to be executed on the data.Task / Work Unit 207 receives tasks from the front area 212 and ensures that GPCs 208 are configured in a valid state before initiating the processing task specified by each set of TMDs. A priority can be specified for each set of TMDs, which is used to schedule the execution of the processing tasks.

[0028] Processing tasks can also be received from the processing cluster array 230. Optionally, the TMD can include a parameter that controls whether the TMD is added to the beginning or the end of a list of processing tasks (or a list of pointers to the processing tasks), thus providing an additional layer of control beyond execution priority.

[0029] The PPU 202 advantageously implements a highly parallel processing architecture based on a processing cluster array 230, which comprises a group of C general-purpose processing clusters (GPCs) 208, where C ≥ 1. Each GPC 208 is capable of executing a large number (for example, hundreds or thousands) of threads concurrently, with each thread being an instance of a program. In different applications, different GPCs 208 can be allocated to process different types of programs or to perform different types of computations. The allocation of GPCs 208 can vary depending on the workload that arises for each type of program or computation.

[0030] A memory interface 214 comprises a group of D partition units 215, where D ≥ 1. Each partition unit 215 is connected to one or more dynamic random access memory (DRAM) units 220 located in the PP memory 204. In one embodiment, the number of partition units 215 is equal to the number of DRAM units 220, and each partition unit 215 is connected to a different DRAM unit 220. In other embodiments, the number of partition units 215 differs from the number of DRAM units 220. Those skilled in the art will recognize that a DRAM unit 220 can be replaced by any other technically suitable memory device.During operation, various render targets, such as texture mappings and frame buffers, can be stored across the DRAMs 220, enabling the partition units 215 to describe sections of each render target in parallel to efficiently utilize the available bandwidth of the PP memory 204.

[0031] A given GPC 208 can process data to be written to one or more of the DRAMs 220 in the PP memory 204. The junction unit 210 is designed to route the output of each GPC 208 to the input of any partition unit 215 or to another GPC 208 for further processing. The GPCs 208 communicate with the memory interface 214 via the junction unit 210 to read from or write to various DRAMs 220. In one embodiment, the junction unit 210 has a connection to the I / O unit 205 in addition to a connection to the PP memory 204 via the memory interface 214, enabling the processing cores in the different GPCs 208 to communicate with the system memory 104 or with other memory that is not local to the PPU 202. In the embodiment of Fig. In 2, the crossing unit 210 is directly connected to the I / O unit 205. In various embodiments, the crossing unit 210 can use virtual channels to separate traffic flows between the GPCs 208 and the partition units 215.

[0032] The GPCs 208, in turn, can be programmed to perform processing tasks related to a wide variety of applications, including, but not limited to, linear and non-linear data transformations, video and / or audio filtering, modeling operations (for example, applying physical laws to determine the position, velocity, and other attributes of objects), image rendering operations (for example, programs for tessellation shading, vertex shading, geometry shading, and / or pixel / fragment shading), general computational methods, etc. In operation, the PPU 202 is designed to transfer data from the system memory 104 and / or the PP memory 204 to one or more on-chip memory units, process the data, and write the resulting data back to the system memory 104 and / or the PP memory 204.The result data can then be accessed by other system components, including the CPU 102, another PPU 202 within the parallel processing subsystem 112, or another parallel processing subsystem 112 in the computer system 100.

[0033] As noted previously, any number of PPUs 202 can be included in a parallel processing subsystem 112. For example, multiple PPUs 202 can be provided in a single add-on card, or multiple add-on cards can be connected to the communication path 113, or one or more of the PPUs 202 can be integrated into a bridge chip. The PPUs 202 in a multi-PPU system can be identical or different from one another. For example, different PPUs 202 can have a different number of processing cores and / or different sizes of PP memory 204. In embodiments where multiple PPUs 202 are present, these PPUs can be operated in parallel to process data with a higher throughput than would be possible with a single PPU 202.Systems incorporating one or more PPUs 202 can be implemented in a variety of configurations and form factors, including, but not limited to, desktop computers, mobile computers, handheld personal computers or other handheld devices, service provider computers, workstation computers, game consoles, embedded systems and the like.

[0034] Fig. 3A is a block diagram of a GPC 208, which is located in the PPU 202 of Fig. 2 is included according to one embodiment of the present invention. In operation, the GPC 208 can be configured to execute a large number of threads in parallel to perform operations for graphics, general processing, and / or computation. As used here, a "thread" denotes an instance of a specific program that is executed on a specific set of input data. In some embodiments, single-instruction, multiple-data (SIMD) instruction output techniques are used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In other embodiments, single-instruction, multiple-thread (SIMT) techniques are used to support the parallel execution of a large number of generally synchronized threads, with a common instruction unit designed to issue instructions to a group of processing units within the GPC 208.

[0035] Unlike a SIMD execution regime, in which all processing units typically execute identical instructions, a SIMT execution regime allows different threads to more efficiently follow divergent execution paths through a given program. Experts will recognize that a SIMD processing regime is a functional subset of a SIMT processing regime.

[0036] The operation of the GPC 208 is controlled by a pipeline manager 305, which distributes processing tasks received from a work distribution unit (not shown) in the task / work unit 207 to one or more data stream multiprocessors (SMs) 310. The pipeline manager 305 can also be configured to control a work distribution junction unit 330 by specifying destinations for processed data output by the SMs 310.

[0037] In one embodiment, the GPC 208 comprises a group M of SMs 310, where M ≥ 1. Furthermore, each SM 310 comprises a group of function execution units (not shown), such as execution units and load-store units. The processing of operations specific to any of the function execution units can be pipelined or executed in parallel, making it possible for a new instruction to be issued for execution before the execution of a previous instruction has finished. Any combination of function execution units can be provided in a given SM 310.The function execution units can be designed in various configurations to support a wide variety of operations, including integer and floating-point arithmetic (e.g., addition and multiplication), comparison operations, Boolean operations (AND, OR, EXCLUSIVE OR), bit shifting, and the calculation of various algebraic functions (e.g., plane interpolation, trigonometric, exponential, and logarithmic functions, etc.). Advantageously, the same function execution unit can be designed to perform different operations.

[0038] In operation, each SM 310 is designed to process one or more thread groups. As used here, a "thread group" or "bulge" refers to a group of threads that simultaneously execute the same program on different input data, with each thread in the group being assigned to a different execution unit within an SM 310. A thread group may contain fewer threads than the number of execution units within the SM 310, in which case some of the execution units may be idle during cycles when that thread group is being processed. A thread group may also contain more threads than the number of execution units within the SM 310, in which case processing may extend over successive clock cycles.Since each SM 310 can support up to G thread groups simultaneously, it follows that up to G*M thread groups can be executed simultaneously in the GPC 208.

[0039] Furthermore, multiple related thread groups can be active simultaneously in an SM 310 (at different stages of execution). This collection of thread groups is referred to here as a "cooperative thread array" ("CTA") or simply a "thread array." The size of a specific CTA is equal to m*k, where k is the number of concurrently executing threads in a thread group, which is typically an integer multiple of the number of execution units within the SM 310, and m is the number of thread groups simultaneously active in the SM 310.

[0040] Although in Fig. Not shown in Figure 3A, each SM 310 includes a level one (L1) cache memory or uses space in a corresponding L1 cache memory outside the SM 310 to support, among other things, load and store operations performed by the execution units. Each SM 310 also has access to level two (L2) cache memory (not shown) shared by all GPCs 208 in the PPU 202. The L2 cache memory can be used to exchange data between threads. Finally, the SMs 310 can also have access to an off-chip “global” memory, which may include PP memory 204 and / or system memory 104. It should also be noted that any memory outside the PPU 202 can be used as global memory. As further shown in Figure 3A, the following applies: Fig. As shown in Figure 3A, a level one-point-five (L1.5) cache memory can be used in the GPC 208 and configured to receive and hold data requested from the memory by the SM 310 via the memory interface 214. Such data can include, without limitation, instructions, uniform data, and constant data. In embodiments with multiple SMs 310 within the GPC 208, the SMs 310 can advantageously share common instructions and data cached in the L1.5 cache memory 325.

[0041] Each GPC 208 can have an associated memory management unit (MMU) 320 designed to map virtual addresses to physical addresses. In various embodiments, the MMU 320 can reside either within the GPC 208 or in the memory interface 214. The MMU 320 comprises a group of page table entries (PTEs) used to map a virtual address to a physical address of a tile or memory page and optionally to a cache row index. The MMU 320 can include address translation sideshow buffers (TLBs) or cache memory, which may reside in the SMs 310 in one or more L1 cache memories or in the GPC 208.

[0042] The GPC 208 can be configured for graphics and computing applications, so that each SM 310 is connected to a texture unit 315 to perform texture mapping requirements, such as determining texture sampling positions, reading texture data, and filtering texture data.

[0043] In operation, each SM 310 sends a processed task to the work distribution junction unit 330 to provide the processed task to another GPC 208 for further processing, or to store the processed task in an L2 cache memory (not shown), the parallel processing memory 204, or the system memory 104 via the junction unit 210. Additionally, a pre-raster operations (pre-ROP) unit 325 is designed to receive data from the SM 310, feed data to one or more raster operations (ROP) units in the partition units 215, perform color mixing optimizations, organize pixel color data, and perform address translations.

[0044] It should be noted that the core architecture described here is for illustrative purposes only and that variations and modifications are possible. Among other things, the GPC 208 can include any number of processing units, such as 310 SMs, 315 texture units, or 325 pre-ROP units. As further explained above in connection with Fig. As described in Figure 2, the PPU 202 can include any number of GPCs 208 designed to be functionally similar to one another, so that the execution behavior does not depend on which GPC 208 receives a particular processing task. Furthermore, each GPC 208 operates independently of the other GPCs 208 in the PPU 202 to perform tasks for one or more application programs. In light of the foregoing, those skilled in the art recognize that the Fig. The architecture described in 1-3A is in no way limiting to the scope of protection of the present invention. Graphics pipeline architecture

[0045] Fig. 3B is a concept diagram of a graphics processing pipeline 350, which is located in the PPU 202 of Fig. 2 according to an embodiment of the present invention. As shown, the graphics processing pipeline 350 comprises, without limitation, a primitive distributor (PD) 355; a vertex attribute picker (VAF) 360; a vertex tessellation geometry processing (VTG) 365; a display field scaler, select and slicer (VPC) 370; a tiling unit 375; a setup unit 380; a raster unit 385; a fragment processing unit, also referred to as a pixel shading unit (PS) 390; and a raster operations unit (ROP) 395.

[0046] The PD 355 collects vertex data, which is associated with higher-order surfaces, graphic primitives and the like, from the front area 212 and sends the vertex data to the VAF 360.

[0047] The VAF 360 retrieves vertex attributes associated with each of the incoming vertices from the shared memory and stores the vertex data along with the associated vertex attributes in the shared memory.

[0048] The VTG 365 is a programmable execution unit designed to run vertex shading, tessellation, and geometry programs. These programs process vertex data and attributes received from the VAF 360 and generate graphic primitives, such as color values, surface normal vectors, and transparency values ​​at each vertex, for further processing within the Graphics Processing Pipeline 350. Although not explicitly shown, in some embodiments the VTG 365 may include one or more of the following units: a vertex processing unit, a tessellation initialization processing unit, a task generation unit, a task distribution unit, a topology generation unit, a tessellation processing unit, and a geometry processing unit.

[0049] The Vertex Processing Unit is a programmable execution unit designed to process vertex shading programs, lighting, and transformation of vertex data as specified by the vertex shading programs. For example, the Vertex Processing Unit can be programmed to transform vertex data from an object-based coordinate representation (object space) to a coordinate system with an alternative basis, such as world space or normalized device coordinate (NDC) space. The Vertex Processing Unit can read vertex data and vertex attributes stored in shared memory by the VAF and can process the vertex data and attributes. The Vertex Processing Unit 415 stores processed vertices in shared memory.

[0050] The tessellation initialization processing unit is a programmable execution unit designed to run tessellation initialization shading programs. The tessellation initialization processing unit processes vertices generated by the vertex processing unit and creates graphic primitives known as patches. The tessellation initialization processing unit also generates various attributes for patches. The tessellation initialization processing unit then stores the patch data and patch attributes in shared memory. In some embodiments, the tessellation initialization shading program may be called envelope shading or tessellation control shading.

[0051] The task generation unit retrieves data and attributes for vertices and spots from shared memory. The task generation unit then creates tasks to process the vertices and spots for later processing by stages in the 350 graphics processing pipeline.

[0052] The task allocation unit redistributes the tasks generated by the task generation unit. The tasks generated by the various instances of the vertex shading program and the tessellation initialization program can differ significantly from one graphics processing pipeline 350 to another. The task allocation unit redistributes these tasks so that each graphics processing pipeline 350 has approximately the same workload during subsequent pipeline stages.

[0053] The topology generation unit retrieves tasks distributed from the task management unit. The topology generation unit indexes the vertices, including those associated with spots, and computes (U, V) coordinates for tiling vertices and the indices that connect the tiled vertices to form graphical primitives. The topology generation unit then stores the indexed vertices in shared memory.

[0054] The tessellation processing unit is a programmable execution unit designed to run tessellation shading programs. The tessellation processing unit reads input data from shared memory and writes output data to it. This output data in shared memory is then passed as input data to the next shading stage, the geometry processing unit 445. In some embodiments, the tessellation shading program may be referred to as an area shading unit or a tessellation evaluation shading unit.

[0055] The geometry processing unit is a programmable execution unit designed to run geometry shading programs, transforming graphic primitives. It groups vertices to form graphic primitives for processing, where graphic primitives include triangles, line segments, points, and the like. For example, the geometry processing unit can be programmed to subdivide the graphic primitives into one or more new graphic primitives and to calculate parameters, such as coefficients for a plane equation, which are used to divide the new graphic primitives into grids.

[0056] The geometry processing unit sends the parameters and vertices specifying the new graphic primitives to the VPC 370. The geometry processing unit can read data stored in the shared memory to use in processing the geometry data. The VPC 370 performs cutting, selection, perspective correction, and a transformation of the display field to determine which graphic primitives can potentially be viewed in the final image and which cannot. The VPC 370 then sends the processed graphic primitives to the tile unit 375.

[0057] The tile unit 375 is a graphic primitive sorting unit located between a world-space pipeline 252 and a screen-space pipeline 354, as further described here. Graphic primitives are processed in the world-space pipeline 352 and then passed to the tile unit 375. The screen-space is subdivided into cache tiles, with each cache tile corresponding to a region of the screen-space. For each graphic primitive, the tile unit 375 identifies the group of cache tiles that intersect with the graphic primitives, a process referred to as "dividing into tiles." After a certain number of graphic primitives have been tiled, the tile unit 375 processes the graphic primitives at the cache-tile level, sending graphic primitives belonging to a specific cache tile to the setup unit 380.The junction unit 375 sends graphical primitives to the setup unit 380, transferring a single cache tile per time unit. Graphical primitives that truncate multiple cache tiles are typically processed once in the world-space pipeline 352, but are then transferred multiple times to the screen-space pipeline 354.

[0058] This technique improves cache locality during processing in the screen-space pipeline 354, whereby multiple memory operations associated with a first cache tile access a region of the L2 cache memory, or another technically feasible cache memory that is retained during the screen-space processing of the first cache tile. Once the graphical primitives belonging to the first cache tile have been processed by the screen-space pipeline 354, the portion of the L2 cache memory belonging to the first cache tile can be cleared, and the tile unit can transfer graphical primitives belonging to a second cache tile. Multiple memory operations associated with a second cache tile can then access the region of the L2 cache memory that is retained during the screen-space processing of the second cache tile.Consequently, the total memory traffic to the L2 cache stores and render targets can be reduced. In some embodiments, the world-space computation for a given graphical primitive is performed only once, regardless of the number of cache tiles in the screen space that truncate the graphical primitive.

[0059] The setup unit 380 receives vertex data from the VPC 370 via the intersection unit 375 and calculates parameters associated with the graphic primitive, including, but not limited to, edge equations, subplane equations, and low-plane equations. The setup unit 380 then sends the processed graphic primitives to the raster unit 385.

[0060] The raster unit 385 converts the new graphic elements by scanning and sends fragments and coverage data to the pixel shading unit 390. Furthermore, the raster unit 385 can be configured to perform z-selection and other z-based optimizations.

[0061] The Pixel Shading Unit 390 is a programmable execution unit designed to run fragment shading programs that transform fragments received by the Raster Unit 385, as specified by the fragment shading programs. Fragment shading programs can shade fragments at pixel-level resolution, and such shading programs can be called pixel shading programs. Alternatively, fragment shading programs can shade fragments at sample-level resolution, where each pixel comprises multiple samples, and each sample represents a portion of a pixel. Alternatively, fragment shading programs can shade fragments at any other technically feasible resolution, depending on the programmed sample rate.

[0062] In various embodiments, the fragment processing unit 460 can be programmed to perform operations such as perspective correction, texture mapping, shading, blending, and the like to generate shaded fragments, which are then sent to the ROP 395. The pixel shading unit 300 can read data stored in the shared memory.

[0063] The ROP 395 is a processing unit that performs raster operations, such as stenciling, z-testing, blending, and the like, and transfers the pixel data as processed graphics data for storage in graphics memory via the memory interface 214, where the graphics memory is typically structured as one or more render targets. The processed graphics data can be stored in the graphics memory, in the parallel processing memory 204, or in the system memory 104 for display on the display device 110 or for further processing by the CPU 102 or the parallel processing subsystem 112. In some embodiments, the ROP 395 is configured to compress z-data or color data written to memory and to decompress z-data or color data read from memory.In various embodiments, the ROP 395 can be arranged in the memory interface 214, in the GPCs 208, in the processing cluster array 230 outside the GPCs, or in a separate unit (not shown) in the PPUs 202.

[0064] The graphics processing pipeline can be implemented by one or more processing elements in the PPU 202. For example, one of the SM 310s from Fig. 3A may be designed to perform the functions of the VTG 365 and / or the pixel shading unit 390. The functions of the PD 355, the VAF 360, the VPC 450, the tiling unit 375, the setup unit 380, the raster unit 385, and the ROP 395 may also be performed by processing elements within a special GPC 208 in conjunction with a corresponding partition unit 215. Alternatively, the graphics processing pipeline 350 may be implemented using special connection elements with a defined function for one or more of the aforementioned functions. In various embodiments, the PPU 202 may be designed to implement one or more graphics processing pipelines 350.

[0065] In some embodiments, the graphics processing pipeline 350 can be subdivided into an image-space pipeline 352 and a screen-space pipeline 354. The world-space pipeline 352 processes graphic objects in 3D space, in which the position of each graphic object relative to other graphic objects and relative to a 3D coordinate system is known. The screen-space pipeline 354 processes graphic objects that have been projected from the 3D coordinate system onto a 2D planar surface representing the surface of the display device 110. For example, the world-space pipeline 352 can include pipeline stages in the graphics processing pipeline 350 from PD 355 up to VPC 370. The screen space pipeline 354 can have pipeline stages in the graphics processing pipeline 350 from the setup unit 380 up to the ROP 395. The tile unit 375 would occur after the last stage of the world space pipeline 352, i.e., after the VPC 370.The tile unit 375 would occur before the first stage of the screen-room pipeline 354, i.e., before the setup unit 380.

[0066] In some embodiments, the image-space pipeline 352 can be further subdivided into an alpha-phase pipeline and a beta-phase pipeline. For example, the alpha-phase pipeline can include pipeline stages in the graphics processing pipeline 350 from the PD 355 up to the task generation unit. The beta-phase pipeline can include pipeline stages in the graphics processing pipeline 350 from the topology generation unit up to the VPC 370. The graphics processing pipeline 350 performs a first group of operations during processing in the alpha-phase pipeline and performs a second group of operations during processing in the beta-phase pipeline. In the sense used here, a group of operations is defined as one or more instructions executed by a single thread, a group of threads, or multiple thread groups acting as a unit.

[0067] In a system with multiple graphics processing pipelines 350, the vertex data and vertex attributes associated with a group of graphic objects can be partitioned so that each graphics processing pipeline 350 has approximately the same amount of workload during the alpha phase. Alpha-phase processing can significantly expand the amount of vertex data and vertex attributes, so that the amount of vertex data and vertex attributes generated by the task generation unit is considerably larger than the amount generated by the PD 255 and the VAF 360.Furthermore, the task generation unit assigned to a single graphics processing pipeline 350 can generate a significantly larger amount of vertex data and vertex attributes than the task generation unit assigned to a different graphics processing pipeline 350, even in cases where both graphics processing pipelines 350 process the same amount of attributes at the beginning of the alpha phase pipeline. In such cases, the task allocation unit redistributes the attributes generated by the alpha phase pipeline so that each graphics processing pipeline 350 has approximately the same workload at the beginning of the beta phase pipeline.

[0068] It should also be noted that, as used here, references to shared memory include one or more technically feasible memories, including, but not limited to, local memory shared by one or more SM 310s, or memory accessible via memory interface 214, such as cache memory, parallel processing memory 204, or system memory 104. It should further be noted that, as used here, references to cache memory include one or more technically feasible memories, including, but not limited to, L1 cache, L1.5 cache, and L2 cache. Tiled caching

[0069] Fig. Figure 4 is a concept diagram of a cache tile 410(0), for which the graphics processing pipeline 350 of Fig. 3B is designed to generate and process these, according to an embodiment of the present invention. As shown, the cache tile 410(0) represents a section of a screen space 400 and is subdivided into several grid tiles 420.

[0070] The screen space 400 represents one or more buffer memories designed to store rendered image data and other data passed by the functional units within the graphics processing pipeline 350. In some embodiments, the one or more buffer memories are configured as one or more render targets. The screen space represents a buffer memory designed to store the image generated by the graphics processing pipeline. The screen space 400 can be associated with any number of render targets, each render target being configured independently of other render targets to contain any number of fields. Each field in a render target is configured independently of other fields to contain any number of bits. Each render target can contain multiple image elements (pixels), and each pixel can, in turn, contain multiple samples.In some embodiments, the size of each cache tile can be based on the size and configuration of the render targets allocated to the screen space. When image generation is complete during operation, the pixels in the one or more render targets can be sent to a display device to show the generated image.

[0071] For example, a group of render targets for screen space 400 can comprise eight render targets. The first render target can include four fields representing colors, such as red, green, and blue component colors and transparency information associated with a corresponding fragment. The second render target can include two fields representing depth information and stencil information associated with the corresponding fragment. The third render target can include three fields representing information about the surface normal vector, including normal vectors along the X-axis, Y-axis, and Z-axis, associated with the corresponding fragment. The remaining five render targets can be configured to store additional information belonging to the corresponding fragment.Such configurations can include: a memory for various information including, but not limited to, 3D position data, information for diffuse lighting and information for specular lighting.

[0072] Each cache tile 410 represents a section of the screen space 400. For clarity, only five cache tiles 410(0)-410(4) are shown in Fig. Figure 4 shows that in some embodiments, the cache tiles in the X and Y screen space have an arbitrary size. For example, if a cache tile resides in a cache memory that is also used to store other data, then the cache tile can be sized to occupy only a specific portion of the cache memory. The size of a cache tile can depend on a number of factors, including the quantity and configuration of the render targets allocated to screen space 400, the number of samples per pixel, and whether the data stored in the cache tile is compressed. In general, a cache tile is sized to increase the likelihood that the cache tile data will be retained in the cache memory until all graphic primitives allocated to the cache tile have been fully processed.

[0073] The raster tiles 420 represent a section of the cache tile 410(0). As shown, the cache tile 410(0) comprises 16 raster tiles 420(0)-420(15) arranged in an array that is four raster tiles 420 wide and four raster tiles 420 high. In systems that include multiple GPCs 208, the processing assigned to a given cache tile 410(0) can be distributed among the available GPCs 208. In the example shown, if the 16 raster tiles of the cache tile 410(0) are to be processed by many different GPCs 208, then each GPC 208 can be assigned to process four of the 16 raster tiles 420 in the cache tile 410(0). Specifically, the first GPC 208 can be assigned to process raster tiles 420(0), 420(7), 420(10), and 420(13). The second GPC 208 can be assigned to process raster tiles 420(1), 420(4), 420(11), and 420(14).The third GPC 208 can be assigned to process raster tiles 420(2), 420(5), 420(8), and 420(15). The fourth GPC 208 can then be assigned to process raster tiles 420(3), 420(6), 420(9), and 420(12). In other embodiments, the processing of the different raster tiles in a given cache tile can be distributed among the GPCs 208 or other processing units comprising the computer system 100 in any technically feasible manner. Piecewise Linear Irregular Rasterization

[0074] The graphics processing pipeline 350 can be configured to perform piecewise linear irregular rasterization, where such piecewise linear irregular rasterization approximates the ideal nonlinear rasterization needed to reverse the effects of VR headset distortion lenses or the artifacts of other widescreen rendering techniques. This piecewise linear irregular rasterization technique is now described.

[0075] Fig. Figure 5 is a detailed view of section 500 of the graphics processing pipeline 350. Fig. 3B according to an embodiment of the present invention. As shown, the illustrated section 500 comprises, without being limiting, a vertex processing unit 510, a geometry processing unit 520; a display field scaling, selection, and cutting unit (VPC) 370; a tiling unit 375, a setup unit 380, a raster unit 385; a fragment processing unit, also characterized as a pixel shading unit 390, and a raster operations unit (ROP) 395. The in Fig. The 5 elements shown work essentially the same way as in Fig. 3B described, except as further described below.

[0076] The Vertex Processing Unit 510 receives vertex data and vertex attributes from the VAF 360 and processes the received vertex data and vertex attributes. The Vertex Processing Unit 510 divides the scene into multiple regions, each region having a different x,y sampling factor and a different x,y offset, where x represents the horizontal dimension of the screen space associated with a display screen, and y represents the vertical dimension of the screen space associated with the display screen. In some embodiments, x and y can represent any two dimensions of a screen space, with the x dimension forming a right angle with the y dimension. The x,y sampling factor for a region determines the extent to which each pixel is to be stretched in the horizontal and vertical dimensions, respectively. The x,y offset determines where the region is to be located in the screen space in the horizontal and vertical dimensions, respectively.In general, regions with a greater horizontal distance from a specified location in screen space have a higher x-sampling factor, resulting in an increased pixel width relative to the region encompassing the specified location. Regions with a greater vertical distance from the specified location in screen space have a higher y-sampling factor, resulting in an increased pixel height relative to the region encompassing the specified location. Regions with a greater horizontal and vertical distance from the specified location in screen space have a higher x-sampling factor and a higher y-sampling factor, resulting in both an increased pixel width and pixel height relative to the region encompassing the specified location.

[0077] In some embodiments, the specified location may be the center of the screen space associated with the display screen. In other embodiments, the specified location may be a location on the screen space that is directly in line with the center of a corresponding distortion lens. This location may or may not be in the center of the screen space. In still other embodiments, the specified location may be any location on the screen. In these latter embodiments, the specified location may be a fixation point related to foveated rendering. Foveated rendering allows a fixation point to be specified, and the fixation point may be specified by a pointing device, including, but not limited to, a computer mouse or a digital pen device.Alternatively, the fixation point can be determined using an eye-tracking device that follows the location of the fovea, that is, the center of the retina of the observer's eye. In these embodiments, the x- and y-sampling factors can be based on the horizontal and vertical distance of a specific region from the fixation point.

[0078] In some embodiments, the x-sample factor and the y-sample factor can each be x- and y-scaling factors that specify the set of scale vertices and corresponding geometry objects that intersect a given region. Generally, a new vertex position (x',y') can be specified as x' = f(x,y,z,w) and y' = f(x,y,z,w), indicating that the new vertex position is a function of the homogeneous coordinates x, y, z, and w of the current vertex position. Any linear function of the homogeneous coordinates x, y, z, and w can be used, ensuring that lines associated with the current vertices of the geometry object remain lines after the vertices have been scaled to the new vertex positions.In one example, the new vertex position could be specified as x' = A*x and y' = B*y, where x and y specify the current vertex position, x' and y' specify the new vertex position, and A and B are horizontal and vertical scaling factors, respectively.

[0079] In some embodiments, the x,y sampling factor and the x,y offset can be applied after the geometric objects have been truncated. In other embodiments, the x,y sampling factor and the x,y offset can be applied before the geometric objects have been truncated. Similarly, in some embodiments, the x,y sampling factor and the x,y offset can be applied after the homogeneous coordinates x, y, z, and w have been normalized by dividing the coordinates by 'w'. In other embodiments, the x,y sampling factor and the x,y offset can be applied before the homogeneous coordinates x, y, z, and w have been normalized by dividing the coordinates by 'w'.

[0080] In a comparative example, the x-sampling factor can be the same for all regions in a given column, and the y-sampling factor is the same for all regions in a given row. In such examples, vertices between a region and an adjacent region can be aligned. As a result, geometric objects that truncate multiple regions can be rendered without discontinuity or seams at the region boundary. In other embodiments, the x-sampling factor can differ between a region and a vertically adjacent region, or the y-sampling factor can differ between a region and a horizontally adjacent region. In such embodiments, vertices between a region and an adjacent region cannot be aligned. As a result, geometric objects that truncate multiple regions can be rendered with discontinuity, seams, or other artifacts at the region boundary.In these embodiments, interpolation and filtering can be applied at such region boundaries to reduce or eliminate such artifacts. The vertex processing unit 510 generates x,y attributes for each received vertex and transfers the x,y attributes to the geometry processing unit 520.

[0081] The geometry processing unit 520 receives processed vertices from the vertex processing unit 510. In some embodiments, the geometry processing unit 520 can transfer replicated geometry objects via display field multicasting. In such embodiments, the geometry processing unit 520 can draw a geometry object once and transfer instances of the geometry object to multiple display fields, each display field having a given x,y sampling factor and x,y offset, as well as a scissor box that can define the portion of screen space for drawing for a given display field. The scene can be divided into multiple display fields, each display field corresponding to a different region of screen space, as described here.When the Geometry Processing Unit 520 receives a geometry object, it can create a clipping mask that causes the geometry object to be transferred only to the display areas corresponding to regions intersected by the geometry object. Alternatively, the Geometry Processing Unit 520 can transfer the geometry object to all display areas without creating a clipping mask, and each display area can determine whether the geometry object is drawn based on whether the geometry object intersects the corresponding region in screen space.

[0082] The VPC 370 receives processed vertices and a scissor box for each display area from the Geometry Processing Unit 520. The VPC 370 can also receive a clipping mask from the Geometry Processing Unit 520. If the VPC 370 receives a clipping mask from the Geometry Processing Unit 520, it generates an output vertex and a corresponding scissor box for each clipped display area, as determined by the clipping mask. If the VPC 370 does not receive a clipping mask from the Geometry Processing Unit 520, it generates an output vertex and a corresponding scissor box for each display area. In some embodiments, the VPC 370 determines which regions are of interest using different geometry objects, rather than the Geometry Processing Unit 520.

[0083] The tile unit 375 receives vertices and scissor boxes from the VPC 370 and transmits vertices and scissor boxes to one or more setup units 380, as further described herein.

[0084] Setup unit 380 receives vertices and corresponding scissor boxes from tile unit 375. Setup unit 380 applies the scissor box to the vertex to determine whether all, part, or none of the geometry object corresponding to the current vertex intersects the display area corresponding to the scissor box. Setup unit 380 computes and transmits linear plane equations and linear attribute boundary equations to raster unit 385.

[0085] The raster unit 385, a pixel shading unit 390, and the ROP 395 perform their standard functions without being consciously aware of the piecewise linear irregular rasterization. Because each region of screen space comprises vertices aligned in a regular linear grid, the raster unit 385, a pixel shading unit 390, and the ROP 395 rasterize and shade the incoming vertices and pixels correctly and efficiently without modifying the standard functionality of these processing elements. These processing elements rely on a linear grid space for efficiency. Because the plane equations and boundary equations generated by the setup unit 380 are linear, the raster unit 385, a pixel shading unit 390, and the ROP 395 maintain efficient processing. For example, the raster unit 385 could perform operations using linear boundary equations.One or more of the Raster Unit 385, a Pixel Shading Unit 390 and the ROP 395 could access attributes in the shared memory based on linear plane equations.

[0086] As described above, each region in screen space is assigned a different display field with a different x,y sampling factor and x,y offset. Consequently, each region in screen space corresponds to a different display field. For example, in a system comprising 16 display fields, the screen space could be divided in any way that does not exceed 16 regions. Thus, the screen space could be divided into regions of 2 rows of 8 columns, 3 rows of 5 columns, and 4 rows of 4 columns. In some embodiments, display fields can be separable into separate x-display fields and y-display fields, such as 16 x-display fields by 16 y-display fields. In such embodiments, each region in screen space could be assigned two display fields—one x-display field and one y-display field.For example, a system with 16 x-display fields by 16 y-display fields would have a total of 256 display fields. In other embodiments, regions can be defined using cache tiles instead of display fields. In such embodiments, each region in screen space could be assigned to a different cache tile, each cache tile representing a different 64-pixel by 64-pixel region in screen space, and each cache tile having a given x,y sampling factor and x,y offset. In still other embodiments, regions can be defined using either coarse raster tiles or fine raster tiles instead of display fields.In such embodiments, each region in screen space can be assigned to a different coarse raster tile, with each cache tile representing a distinct 16-pixel by 16-pixel region in screen space and each raster tile having a given x,y sampling factor and x,y offset. Alternatively, each region in screen space can be assigned to a different coarse raster tile, with each cache tile representing a distinct 8-sample by 8-sample region in screen space. By using cache tiles or raster tiles instead of display fields, more regions in screen space can be defined. As a result, piecewise linear irregular rasterization using cache tiles or raster tiles can more closely approximate the ideal nonlinear rasterization grid compared to the display field approach.

[0087] In some embodiments, the elements in the graphics processing pipeline 350 can invert the piecewise linear irregular plane equations back into a regular square grid before accessing certain attributes. For example, a texture in the texture memory might be mapped to square pixels. An element accessing the texture memory would invert the piecewise linear irregular plane equations back into a regular square grid. The element would then use the vertex position, based on the inverted plane equations, to access the texture memory and restore the correct texel.

[0088] Fig. Figures 6A-6C illustrate exemplary scanning grids that can be implemented during piecewise irregular rasterization according to various embodiments of the present invention.

[0089] As in Fig. As shown in Figure 6A, the scanning grid comprises nine regions 610(0) - 610(8), arranged as three rows by three columns. Region 610(4), near the center of the screen space, contains square pixels with a relatively fine resolution. Regions 610(3) and 610(5), located at a horizontal distance from the central region 610(4), contain pixels with a larger horizontal spacing relative to the central region 610(4). Regions 610(1) and 610(7), located at a vertical distance from the central region 610(4), contain pixels with a larger vertical spacing relative to the central region 610(4). Regions 610(0), 610(2), 610(6) and 610(8), located at a horizontal and vertical distance from the center region 610(4), include pixels with greater horizontal spacing and greater vertical spacing relative to the center region 610(4).As shown, regions in the same column all have the same horizontal resolution, and regions in the same row all have the same vertical resolution.

[0090] As in Fig. As shown in Figure 6B, triangle 620 is to be rendered onto the screen space represented by regions 610(0) - 610(8). Triangle 620 intersects four regions: 610(4), 610(6), 610(7), and 610(8). When rendered onto the piecewise linear irregular grid, triangle 620 appears as triangle 630, indicating discontinuity at the boundaries between adjacent regions. When projected into the VR headset, the distortion illustrated by triangle 630 and the optical distortion of the lenses in the VR headset interact, and the viewer sees a triangle that approximates triangle 620.

[0091] As in Fig. As shown in Figure 6C, the scanning grid comprises six regions 640(0)–640(5), arranged as two rows by three columns. Region 640(4) comprises five rows by eight columns of square pixels with a relatively high resolution. Regions 640(3) and 640(5), located horizontally from the central region 640(4), contain pixels with greater horizontal spacing relative to region 640(4). Region 640(1), located vertically from region 640(4), contains pixels with greater vertical spacing relative to region 640(4). Regions 640(0) and 640(2), located horizontally and vertically from the central region 640(4), contain pixels with greater horizontal and greater vertical spacing relative to region 640(4), respectively. As shown, regions 640(0) and 640(2) have different horizontal and vertical resolutions relative to the directly adjacent regions.Therefore, geometric objects located in one or both regions 640(0) and 640(2) and adjacent regions are likely to exhibit discontinuity, seams, or other artifacts. Such geometric objects should therefore be interpolated and filtered at region boundaries to reduce the effect of these artifacts.

[0092] As in Fig. As shown in Figure 6D, the scanning grid comprises six regions 650(0) - 650(5), arranged as two rows by three columns. Regions 650(1), 650(3), 650(4), and 650(5) comprise 8 rows by 8 columns of square pixels. Regions 650(0) and 650(2) comprise 6 rows by 6 columns of square pixels. All six regions 650(0) - 650(5) have the same horizontal and vertical resolution.

[0093] Fig. Figure 7 represents a flowchart of process steps for performing piecewise irregular rendering according to an embodiment of the present invention. Although the process steps are related to the systems of Fig. As described in 1-6D, those skilled in the art will understand that any system capable of carrying out the process steps in any order is within the scope of protection of this disclosure.

[0094] As shown, a process begins at step 702, where the Vertex Processing Unit 510 calculates x,y attributes, including the x,y sampling factor and x,y offset, for each received vertex. The calculated x,y sampling factor and x,y offset place the received vertex and the corresponding geometry object in the appropriate region of screen space. Each region of screen space is associated with a specific x and y sampling factor, which defines the region's horizontal and vertical resolution, respectively. Similarly, each region of screen space is associated with a specific x and y offset, which defines the region's horizontal and vertical position, respectively. In general, different regions have different x,y sampling factors and x,y offsets, so the pixel resolution can vary from region to region.In the case of rendering for VR headsets, regions near a specified location in the screen space have a relatively high resolution to provide sufficient resolution when these regions are magnified by the distortion lens. Regions horizontally removed from the specified location on the screen have a correspondingly lower resolution as the horizontal distance increases. Similarly, regions vertically removed from the specified location on the screen have a correspondingly lower resolution as the vertical distance increases. More generally, each region can be independently dimensioned, scaled, and offset. As a result, various configurations of regions are possible using the disclosed techniques.

[0095] In step 704, the geometry processing unit 520 calculates a display field clipping mask for each vertex. The display field clipping mask includes an indicator for each region, where the indicator for a given region is set to TRUE if a given geometry object intersects the region and is set to FALSE if the given geometry object does not intersect the region. In step 706, the VPC 370 performs clipping operations for each relevant display field. In some embodiments, the relevant display fields are those that clip the geometry object according to a given vertex. In these embodiments, the VPC 370 scans the mask and creates an instance of the geometry object for each region where the corresponding indicator in the display field clipping mask is set to TRUE. In other embodiments, any display field is considered to be a relevant display field.In these latter embodiments, the geometry processing unit 520 cannot generate a display field clipping mask. In such embodiments, the VPC 370 can round out an instance of the geometry object to each display field. The setup unit 380 can then clip or reject the triangle for those display fields that are not intersected by the triangle. These latter embodiments do not utilize the geometry processing unit 520, thereby improving the efficiency and performance of the graphics processing pipeline 350. Generating instances of geometry objects for regions that intersect the geometry object may slightly decrease the efficiency for stages of the graphics processing pipeline 350 that are downstream of the geometry processing unit 520.These reductions in efficiency can be more than compensated for by the increased efficiency of not using the geometry processing unit 520 to generate display field clipping masks.

[0096] In step 708, the VPC 370 replicates the geometry object for each relevant display area. As described above, the VPC 370 can create instances of the geometry object only for those display areas that truncate the geometry object, with such display areas being the relevant display areas. Alternatively, the VPC 370 can create instances of the geometry object and broadcast one instance for each display area, with all display areas being considered relevant display areas.

[0097] In step 710, the VPC 370 applies sample rates and offsets to the vertices to set the vertex positions for each of the corresponding display regions. In step 712, the VPC 370 tags each replicated geometry object with the corresponding relevant display region. This tag identifies each instance of the geometry object with the associated region, so that downstream stages of the 350 graphics processing pipeline render each instance of the geometry object correctly and properly based on the appropriate x,y sample rate and x,y offset for the region. In step 714, the setup unit 380 applies a scissor box to the tagged display region. Applying the scissor box simply causes the portion of the geometry object that intersects a given display region to be rendered in that display region.In step 716, the raster unit 385, the pixel shading unit 390, and the ROP unit 395 perform their normal functions. As further described herein, the disclosed techniques provide linear plane equations and boundary equations for each display area. As long as the plane equations and boundary equations remain linear, the raster unit 385, the pixel shading unit 390, and the ROP unit 395 do not require any special programming or functional changes to correctly render the instances of geometry for the various display areas. Procedure 700 then terminates.

[0098] In summary, 3D computer geometry objects for widescreen applications, such as VR headsets, are rendered using piecewise irregular rasterization. The screen space of the display is divided into multiple regions, with each region assigned a specific x,y sampling factor and x,y offset.

[0099] Geometry objects that span pixels in one or more screen-space regions are rendered according to the x,y sampling factor and x,y offset for the specific regions that clip the geometry object. For example, a geometry object spanning three screen-space regions might be rendered with one x,y sampling factor and x,y offset in the first region, another x,y sampling factor and x,y offset in the second region, and yet another x,y sampling factor and x,y offset in the third region. For widescreen rendering, regions located near a specified location in screen space might have relatively small square pixels.For regions that are located far from the specified location, where high resolution is typically not necessary, the x-sampling factor could increase as the horizontal distance from the specified location increases, and the y-sampling factor could increase as the vertical distance from the specified location increases.

[0100] At least one advantage of the disclosed method is that pixels located near the edge of the screen space are rendered at a lower resolution, resulting in improved performance relative to previously known widescreen rendering techniques. Another advantage of the disclosed technique is that pixels located near a specified location in the screen space are rendered at a higher resolution relative to previous techniques, resulting in improved video quality near that specified location in the screen space relative to previously known widescreen rendering techniques.

[0101] The descriptions of the various embodiments are presented for illustrative purposes only; however, they are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be obvious to those skilled in the art without altering the scope and nature of the described embodiments.

[0102] Aspects of the present embodiments may be embodied as a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of a purely hardware implementation, a purely software implementation (including firmware, resident software, microcode, etc.), or an embodiment that combines software and hardware aspects, all of which may herein be generally referred to as a "circuit," "module," or "system." Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media, the computer-readable program code embodied therein.

[0103] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signaling medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing.More specific examples (a non-exhaustive list) of computer-readable storage medium would include: an electrical connection with one or more wires, a portable computer disk, a hard disk, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, computer-readable storage medium can be any physical medium capable of containing or storing a program for use by or in conjunction with an instruction execution system, device, or apparatus.

[0104] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of processes, facilities (systems), and computer program products according to embodiments of the disclosure. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing device to create a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in the flowchart and / or block diagram block or blocks. Such processors may, without limitation, be general-purpose processors, special-purpose processors, application-specific processors, or field-programmable processors.

[0105] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, procedures, and computer program products according to various embodiments of the present disclosure. In this respect, each block in the flowchart or block diagrams can represent a module, segment, or section of code comprising one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions mentioned in the block may occur out of the order shown in the figures. For example, two consecutively shown blocks may actually be executed essentially simultaneously, or the blocks may be executed several times in reverse order, depending on the functionality involved.It should also be noted that each block of the block diagrams and / or flowchart illustration, as well as combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special-purpose hardware-based systems that perform the specified functions or actions, or combinations of special-purpose hardware and computer instructions.

Claims

[1] Method for rendering a geometry object in a computer-generated scene, wherein the method comprises: Dividing a screen space assigned to a display screen into multiple regions; for each region: Calculating a first sampling factor in a horizontal dimension, which represents a horizontal sampling factor for pixels localized in the region, Calculating a second sampling factor in a vertical dimension, representing a vertical sampling factor for pixels localized in the region, Calculating a first offset in the horizontal dimension, which represents a horizontal position assigned to the region, and Calculating a second offset in the vertical dimension, representing a vertical position assigned to the region; Determine that the geometry object intersects more than one region; Creating a first instance of the geometry object for a first region that intersects the geometry object; and Creating a second instance of the geometry object for a second region that intersects the geometry object, wherein the first region and the second region are arranged in the screen space adjacent to each other in at least one of the horizontal and vertical directions, wherein the first sampling factor of the first region differs from the first sampling factor of the second region, and the second sampling factor of the first region differs from the second sampling factor of the second region. [2] Method according to claim 1, wherein the first instance is associated with a first scissor box and the second instance is associated with a second scissor box, and further comprising: Applying the first sampling factor, the first offset, the second sampling factor, and the second offset assigned to the first region to the first instance; Applying the first scissors box to the first instance; Applying the first sampling factor, the first offset, the second sampling factor, and the second offset assigned to the second region to the second instance; and Applying the second scissors box to the second instance. [3] Method according to claim 1 or 2, wherein the first sampling factor for the first region is based on the horizontal distance of the first region from a location on the screen space, the second sampling factor for the first region is based on the vertical distance of the first region from the location on the screen space, the first sampling factor for the second region is based on the horizontal distance of the second region from the location on the screen space, and the second sampling factor for the second region is based on the vertical distance of the second region from the location on the screen space. [4] Method according to claim 3, wherein the location on the screen space corresponds to the center of a lens associated with the display screen. [5] Method according to one of the preceding claims, further comprising generating a mask that indicates that the first region and the second region cut off the geometry object. [6] Method according to one of the preceding claims, wherein each region contained in the plurality of regions is assigned a different display field in a plurality of display fields. [7] Method according to one of the preceding claims, wherein the first region is assigned to a first horizontal display field in a plurality of horizontal display fields and to a first vertical display field in a plurality of vertical display fields, and the second region is assigned to a second horizontal display field in the plurality of horizontal display fields and to a second vertical display field in the plurality of vertical display fields. [8] Method according to one of the preceding claims, wherein each region in the plurality of regions is assigned to a different cache tile which is contained in a plurality of cache tiles. [9] Method according to one of the preceding claims, wherein each region in the plurality of regions is assigned to a different raster tile which is contained in a plurality of raster tiles. [10] Method according to one of the preceding claims, wherein the regions contained in the plurality of regions are arranged in a set of rows and a set of columns, and wherein each region contained in a given row has the same second sampling factor, and each region contained in a given column has the same first sampling factor. [11] Method according to one of the preceding claims, wherein the regions contained in the plurality of regions are arranged in a set of rows and a set of columns, and the first region and the second region are contained in the same row, and wherein the first region has a vertical size that differs from a vertical size calculated for the second region, and the first region has a horizontal size that differs from a horizontal size calculated for a third region located in the same column as the first region. [12] Method according to any of the preceding claims, wherein the first sampling factor comprises a first scaling factor that specifies a first multiplier that is applied to an 'x' coordinate of each vertex associated with the geometry object, and the second sampling factor comprises a second scaling factor that specifies a second multiplier that is applied to a 'y' coordinate of each vertex associated with the geometry object. [13] A computer-readable storage medium comprising instructions which, when executed by a processor, cause the processor to render a geometry object in a computer-generated scene by performing the steps as mentioned in any one of claims 1 to 12. [14] Graphics processing pipeline, comprising: a vertex processor designed to: to divide a screen space assigned to a display screen into a plurality of regions; for each region: to calculate a first sampling factor in a horizontal dimension, which represents a horizontal sampling factor for pixels localized in the region, to calculate a second sampling factor in a vertical dimension, which represents a vertical sampling factor for the pixels localized in the region, to calculate a first offset in the horizontal dimension, which represents a horizontal position assigned to the region, and to calculate a second offset in the vertical dimension, representing a vertical position assigned to the region; and a display field scaling, selection, and slicing processor designed to: determine that a geometry object intersects more than one region; to create a first instance of the geometry object for a first region that intersects the geometry object; and to create a second instance of the geometry object for a second region that intersects the geometry object, wherein the first region and the second region are arranged in screen space adjacent to each other in one of the directions horizontally and vertically, wherein the first sampling factor of the first region differs from the first sampling factor of the second region, and the second sampling factor of the first region differs from the second sampling factor of the second region.

Images