Method, computer readable medium and computer system for graphics processing

Abstract

A method for graphics processing. The method includes rendering graphics for an application using a plurality of graphics processing units (GPUs). The method includes dividing responsibility for rendering geometry for the graphics among the plurality of GPUs based on a plurality of screen regions, each GPU having a corresponding division of responsibility known to the plurality of GPUs. The method includes generating information about a first screen region having a first division of responsibility for the geometry relative to a first GPU while rendering a geometry for an image at a second GPU. The method includes using the information to render the geometry at the first GPU.

Description

[0001] This application is a divisional application of the invention patent application with application number 202180018940.1 filed on January 29, 2021, and invention name "System and method for efficient multi-GPU rendering of geometric graphics through area testing during rendering". Technical Field

[0002] The present disclosure relates to graphics processing, and more particularly to multi-GPU collaboration in rendering images for applications. Background Art

[0003] In recent years, there has been a push for online services that allow online gaming or cloud gaming to be performed in a streaming format between a cloud gaming server and clients connected over a network. The streaming format has become increasingly popular due to the availability of game titles on demand, the ability to perform more complex games, the ability to network between players for multiplayer gaming, the sharing of assets between players, the sharing of instant experiences between players and / or spectators, allowing friends to watch friends play video games, allowing friends to join a friend's ongoing game play, and so on.

[0004] The cloud gaming server can be configured to provide resources to one or more clients and / or applications. In other words, the cloud gaming server can be configured with resources capable of high throughput. For example, the performance that a single graphics processing unit (GPU) can achieve is limited. In order to render more complex scenes or use more complex algorithms (such as materials, lighting, etc.) when generating scenes, it may be necessary to use multiple GPUs to render a single image. However, the equal use of these graphics processing units is difficult to achieve. In addition, even if there are multiple GPUs using traditional techniques to process images for applications, the corresponding increase in screen pixel count and geometric density cannot be supported (for example, four GPUs cannot write four times the pixels and / or process four times the vertices or primitives of the image).

[0005] It is against this background that embodiments of the present disclosure emerge. Summary of the invention

[0006] Embodiments of the present disclosure relate to collaborative use of multiple GPUs to render a single image, such as multi-GPU rendering of an application's geometry with region testing at render time.

[0007] Embodiments of the present disclosure disclose a method for graphics processing. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes dividing the responsibility for rendering geometry of the graphics between multiple GPUs based on multiple screen regions, each GPU having corresponding divisions of responsibility known to multiple GPUs. The method includes generating information about a first screen region having a first division of responsibility for the geometry relative to a first GPU while rendering a geometry for an image at a second GPU. The method includes using the information to render the geometry at the first GPU.

[0008] Other embodiments of the present disclosure disclose a non-transitory computer-readable medium for performing a method. The computer-readable medium includes program instructions for rendering graphics for an application using multiple graphics processing units (GPUs). The computer-readable medium includes program instructions for dividing the responsibility for rendering the geometry of the graphics between multiple GPUs based on multiple screen areas, each GPU having a corresponding responsibility division known to multiple GPUs. The computer-readable medium includes program instructions for generating information about a first screen area having a first responsibility division relative to a first GPU while rendering a geometry for an image at a second GPU. The computer-readable medium includes program instructions for using the information to render the geometry at the first GPU.

[0009] Still other embodiments of the present disclosure disclose a computer system comprising a processor and a memory, the memory being coupled to the processor and having instructions stored therein, which, if executed by the computer system, cause the computer system to perform a method. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes dividing the responsibility for rendering geometry of the graphics between multiple GPUs based on multiple screen areas, each GPU having corresponding responsibility divisions known to multiple GPUs. The method includes generating information about a first screen area having a first responsibility division relative to a first GPU while rendering a geometry for an image at a second GPU. The method includes using the information to render the geometry at the first GPU.

[0010] Embodiments of the present disclosure disclose a method for graphics processing. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes dividing the responsibility for rendering the geometry of the graphics between GPUs based on screen area, each GPU having a corresponding division of responsibility known to the GPU. The method includes determining a Z value of a geometry during a pre-pass phase of rendering an image at a first GPU, wherein the geometry overlaps a first screen area for which the first GPU has a division of responsibility. The method includes comparing the Z value with a Z buffer value of the geometry. The method includes generating information including the result of comparing the Z value with the Z buffer value for use by the GPU when rendering the geometry during a full rendering phase of rendering.

[0011] Other embodiments of the present disclosure disclose a non-transitory computer-readable medium for performing a method. The computer-readable medium includes program instructions for rendering graphics for an application using multiple graphics processing units (GPUs). The computer-readable medium includes program instructions for dividing the responsibility of rendering the geometry of the graphics between GPUs based on screen area, each GPU having a corresponding responsibility division known to the GPU. The computer-readable medium includes program instructions for determining a Z value of a geometry during a pre-pass phase of rendering an image at a first GPU, wherein the geometry overlaps a first screen area where the first GPU has a responsibility division. The computer-readable medium includes program instructions for comparing the Z value with a Z buffer value of the geometry. The computer-readable medium includes program instructions for generating information including the result of comparing the Z value with the Z buffer value for use by the GPU when rendering the geometry during a full rendering phase of rendering.

[0012] Still other embodiments of the present disclosure disclose a computer system comprising a processor and a memory coupled to the processor and having instructions stored therein, which, if executed by the computer system, cause the computer system to perform a method. The method includes rendering graphics for an application using multiple graphics processing units (GPUs). The method includes dividing the responsibility for rendering geometry of the graphics between GPUs based on screen area, each GPU having a corresponding division of responsibility known to the GPU. The method includes determining a Z value of a geometry during a pre-pass phase of rendering an image at a first GPU, wherein the geometry overlaps a first screen area having a division of responsibility for the first GPU. The method includes comparing the Z value with a Z buffer value of the geometry. The method includes generating information including the result of comparing the Z value with the Z buffer value for use by the GPU when rendering the geometry during a full rendering phase of rendering.

[0013] Other aspects of the disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present disclosure may be best understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 1 is a schematic diagram of a system for providing games between one or more cloud gaming servers over a network according to an embodiment of the present disclosure, wherein the cloud gaming server is configured to enable collaboration of multiple GPUs to render a single image, including multi-GPU (graphics processing unit) rendering of geometry for an application through region testing during rendering.

[0016] Figure 2 is a schematic diagram of a multi-GPU architecture in which multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure.

[0017] Figure 3 is a schematic diagram of multiple graphics processing unit resources configured for multi-GPU rendering of geometry for an application through region testing at render time according to one embodiment of the present disclosure.

[0018] Figure 4 is a schematic diagram of a rendering architecture implementing a graphics pipeline configured for multi-GPU processing such that multiple GPUs cooperate to render a single image according to one embodiment of the present disclosure.

[0019] Figure 5 is a flow chart illustrating a method for graphics processing according to one embodiment of the present disclosure, the method including multi-GPU rendering of geometry for an application through region testing at render time.

[0020] Fig. 6A is a schematic diagram of a screen subdivided into quadrants when performing multi-GPU rendering according to one embodiment of the present disclosure.

[0021] Figure 6B is a schematic diagram of a screen subdivided into a plurality of staggered regions when performing multi-GPU rendering according to one embodiment of the present disclosure.

[0022] Fig. 7A is a schematic diagram of a rendering command buffer shared by multiple GPUs collaborating to render a single image according to one embodiment of the present disclosure.

[0023] Figure 7B-1 An image including four objects rendered by multiple GPUs is illustrated according to one embodiment of the present disclosure, and the screen area responsibility of each GPU in rendering the objects of the image is shown.

[0024] Figure 7B-2 is a diagram illustrating a rendering according to an embodiment of the present disclosure Figure 7B-1 The four objects are tables of rendering performed by each GPU.

[0025] Figure 7C is a diagram illustrating rendering an image frame (eg, Figure 7B-1 Schematic diagram of the rendering of each object performed by each GPU when the image is rendered.

[0026] Fig. 8A Illustrated is object testing for screen areas when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure.

[0027] Figure 8B Illustrated is testing of multiple portions of an object for a screen area when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure.

[0028] Figure 9A-9C Various strategies for allocating screen areas to corresponding GPUs when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure are illustrated.

[0029] Fig. 10A is a schematic diagram illustrating multi-pass rendering when performing multi-GPU rendering of geometry for an application through region testing at rendering time according to one embodiment of the present disclosure.

[0030] Fig. 10B is a flow chart illustrating a method for graphics processing including performing a Z-only pre-pass stage of rendering for later use by the same GPU in rendering, according to one embodiment of the present disclosure.

[0031] Fig.11 Components of an example device that may be used to perform aspects of various embodiments of the present disclosure are illustrated. DETAILED DESCRIPTION

[0032] Although the following detailed description contains many specific details for illustrative purposes, those skilled in the art will appreciate that many variations and modifications of the following details are within the scope of the present disclosure. Therefore, the various aspects of the present disclosure described below are set forth without any loss of generality of the appended claims of this specification and without imposing any limitations on these claims.

[0033] In general, there are limits to the performance that a single GPU can achieve, such as from limitations on how large a GPU can be. In embodiments of the present disclosure, in order to render more complex scenes or use more complex algorithms (e.g., materials, lighting, etc.), it is desirable to use multiple GPUs to render a single image. In particular, various embodiments of the present disclosure describe methods and systems for multi-GPU rendering that are configured to perform geometry for an application through area testing at rendering time. Specifically, multiple GPUs collaborate to generate an image of an application. The responsibility for graphics rendering of an application is divided among multiple GPUs based on corresponding screen areas. When rendering geometry for an image, each of the multiple GPUs generates information about the geometry and its relationship to screen areas that other GPUs may be responsible for; this information allows other GPUs to render the geometry more efficiently or avoid rendering it altogether. As an advantage, for example, this allows multiple GPUs to render more complex scenes and / or images in the same amount of time.

[0034] With the above general understanding of various embodiments in mind, example details of the embodiments will now be described with reference to the various figures.

[0035] Throughout this specification, references to "applications" or "games" or "video games" or "gaming applications" are intended to refer to any type of interactive application that is directed by executing input commands. For purposes of illustration only, interactive applications include applications for gaming, word processing, video processing, video game processing, etc. Furthermore, the terms introduced above are interchangeable.

[0036] Throughout the specification, various embodiments of the present disclosure are described for multi-GPU processing or rendering of geometry for applications using an exemplary architecture with four GPUs. However, it should be understood that any number of GPUs (e.g., two or more GPUs) can collaborate when rendering geometry for an application.

[0037] Figure 1 1 is a schematic diagram of a system for performing multi-GPU processing when rendering images for an application according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the system is configured to provide games between one or more cloud gaming servers over a network, and more specifically configured for collaboration of multiple GPUs to render a single image of an application. Cloud gaming includes executing a video game on a server to generate a video frame rendered by the game, which is then sent to a client for display. In particular, the system 100 is configured for efficient multi-GPU rendering of geometry for an application through region testing at rendering time.

[0038] Although Figure 1The illustration illustrates an implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system, but other embodiments of the present disclosure provide efficient multi-GPU rendering of geometry for an application by performing region testing when rendering within a separate system (such as a personal computer or game console that includes a high-end graphics card with multiple GPUs).

[0039] It should also be understood that in various embodiments (e.g., in a cloud gaming environment or in a standalone system), multi-GPU rendering of geometric graphics can be performed using a physical GPU or a virtual GPU or a combination of the two. For example, a virtual machine (e.g., an instance) can be created using a hypervisor of host hardware (e.g., located in a data center) that utilizes one or more components of a hardware layer (such as multiple CPUs, memory modules, GPUs, network interfaces, communication components, etc.). These physical resources can be arranged in racks, such as CPU racks, GPU racks, memory racks, etc., where the physical resources in the racks can be accessed using a top-of-rack switch that facilitates the structure of components for assembling and accessing instances (e.g., when building virtualized components of instances). Typically, a hypervisor can present multiple client operating systems configured with multiple instances of virtual resources. That is, each operating system can be configured with a corresponding set of virtualized resources supported by one or more hardware resources (e.g., located in a corresponding data center). For example, each operating system can be supported by a virtual CPU, multiple virtual GPUs, virtual memory, virtualized communication components, etc. In addition, the configuration of an instance can be transferred from one data center to another to reduce latency. The GPU utilization defined for a user or game can be used when saving a user's game session. GPU utilization can include any number of configurations described herein to optimize fast rendering of video frames for a gaming session. In one embodiment, GPU utilization defined for a game or user can be transferred between data centers as a configurable setting. The ability to transfer GPU utilization settings can effectively migrate game play from data center to data center in situations where users are connected to different geographic locations to play the game.

[0040] According to one embodiment of the present disclosure, the system 100 provides a game via a cloud gaming network 190, wherein a client device 110 (e.g., a thin client) away from the corresponding user playing the game executes the game. The system 100 can provide game control to one or more users playing one or more games through the cloud gaming network 190 via the network 150 in a single player or multi-player mode. In some embodiments, the cloud gaming network 190 may include a plurality of virtual machines (VMs) running on a host's hypervisor, wherein one or more virtual machines are configured to execute a game processor module using hardware resources available to the host's hypervisor. The network 150 may include one or more communication technologies. In some embodiments, the network 150 may include a fifth generation (5G) network technology with an advanced wireless communication system.

[0041] In some embodiments, wireless technology can be used to facilitate communication. Such technology can include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. The 5G network is a digital cellular network in which the service area covered by the provider is divided into small geographical areas called cells. The analog signals representing sound and images are digitized in the phone, converted by an analog-to-digital converter and transmitted as a bit stream. Through the frequency channels allocated by the transceiver from the frequency pool reused in other cells, all 5G wireless devices in the cell communicate with the local antenna array and low-power automatic transceiver (transmitter and receiver) in the cell through radio waves. The local antenna is connected to the telephone network and the Internet through a high-bandwidth optical fiber or wireless backhaul connection. As in other cell networks, a mobile device that crosses from one cell to another will automatically go to the new cell. It should be understood that the 5G network is only an example type of communication network, and the embodiments of the present disclosure can utilize an earlier generation of wireless or wired communication, as well as a newer generation of wired or wireless technology after 5G.

[0042] As shown, the cloud gaming network 190 includes a game server 160 that provides access to multiple video games. The game server 160 can be any type of server computing device available in the cloud, and can be configured as one or more virtual machines executed on one or more hosts. For example, the game server 160 can manage a virtual machine that supports a game processor that instantiates a game instance for a user. In this way, multiple game processors of the game server 160 associated with multiple virtual machines are configured to execute multiple instances of one or more games associated with the game progress of multiple users. In this way, the back-end server supports the streaming of media (e.g., video, audio, etc.) of the game progress of multiple game applications provided to multiple corresponding users. That is, the game server 160 is configured to stream data (e.g., rendered images and / or frames of the corresponding game progress) back to the corresponding client device 110 through the network 150. In this way, computationally complex game applications can be executed at the back-end server in response to controller inputs received and forwarded by the client device 110. Each server is capable of rendering images and / or frames, which are then encoded (e.g., compressed) and streamed to the corresponding client device for display.

[0043] For example, multiple users can access the cloud gaming network 190 via the communication network 150 using a corresponding client device 110 configured to receive streaming media. In one embodiment, the client device 110 may be configured as a thin client that provides an interface with a back-end server (e.g., cloud gaming network 190) configured to provide computing functionality (e.g., including a game name processing engine 111). In another embodiment, the client device 110 may be configured with a game name processing engine and game logic for at least some local processing of the video game, and may also be utilized to receive streaming content generated by the video game executed at the back-end server, or other content provided by the back-end server support. For local processing, the game name processing engine includes basic processor-based functions for executing video games and services associated with the video game. In this case, the game logic can be stored on the local client device 110 and used to execute the video game.

[0044] Each client device 110 may request access to a different game from the cloud gaming network. For example, the cloud gaming network 190 may execute one or more game logics that are built on the game name processing engine 111, such as executed using the CPU resources 163 and GPU resources 365 of the game server 160. For example, game logic 115a that cooperates with the game name processing engine 111 may be executed on the game server 160 for one client, game logic 115b that cooperates with the game name processing engine 111 may be executed on the game server 160 for the second client, ... and game logic 115n that cooperates with the game name processing engine 111 may be executed on the game server 160 for the Nth client.

[0045] In particular, the client device 110 of the corresponding user (not shown) is configured to request access to the game through a communication network 150 such as the Internet, and to render a display image generated by the video game executed by the game server 160, wherein the encoded image is transmitted to the client device 110 for display in association with the corresponding user. For example, the user can interact with an instance of the video game executed on the game processor of the game server 160 through the client device 110. More specifically, the instance of the video game is executed by the game title processing engine 111. The corresponding game logic (e.g., executable code) 115 that implements the video game is stored and accessible through a data storage area (not shown) and used to execute the video game. The game title processing engine 111 is capable of supporting multiple video games using multiple game logics (e.g., game applications), each of which can be selected by the user.

[0046] For example, the client device 110 is configured to interact with a game title processing engine 111 associated with the game progress of the corresponding user, such as through input commands for driving the game progress. In particular, the client device 110 can receive input from various types of input devices, such as a game controller, a tablet computer, a keyboard, a gesture captured by a camera, a mouse, a touchpad, etc. The client device 110 can be any type of computing device having at least a memory and a processor module that can be connected to the game server 160 via a network 150. The back-end game title processing engine 111 is configured to generate a rendered image, which is transmitted via the network 150 to be displayed at a corresponding display associated with the client device 110. For example, through a cloud-based service, the game rendering image can be transmitted by an instance of the corresponding game (e.g., game logic) executed on the game execution engine 111 of the game server 160. That is, the client device 110 is configured to receive an encoded image (e.g., encoded from a game rendering image generated by executing a video game), and to display the rendered image on the display 11. In one embodiment, the display 11 includes an HMD (e.g., displaying VR content). In some embodiments, the rendered image may be downloaded wirelessly or wired, directly from a cloud-based service, or via a client device 110 (e.g., Remote Play) to your smartphone or tablet.

[0047] In one embodiment, the game server 160 and / or the game title processing engine 111 include basic processor-based functions for executing games and services associated with game applications. For example, the game server 160 includes a central processing unit (CPU) resource 163 and a graphics processing unit (GPU) resource 365, which are configured to perform processor-based functions, including 2D or 3D rendering, physical simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformation, artificial intelligence, etc. In addition, the CPU and GPU group can implement services for game applications, including memory management, multi-threading management, quality of service (QoS), bandwidth testing, social networks, social friend management, social network communications with friends, communication channels, texting, instant messaging, chat support, etc. In one embodiment, one or more applications share specific GPU resources. In one embodiment, multiple GPU devices can be combined to perform graphics processing for a single application executed on a corresponding CPU.

[0048] In one embodiment, the cloud game network 190 is a distributed game server system and / or architecture. In particular, the distributed game engine that executes the game logic is configured as a corresponding instance of the corresponding game. Typically, the distributed game engine uses each function of the game engine and distributes these functions for execution by multiple processing entities. Individual functions can be further distributed on one or more processing entities. The processing entity can be configured as different configurations, including physical hardware, and / or as a virtual component or virtual machine, and / or as a virtual container, wherein the container is different from the virtual machine because it virtualizes the instance of the game application running on the virtualized operating system. The processing entity can utilize and / or rely on the server and its underlying hardware on one or more servers (computing nodes) of the cloud game network 190, wherein the server can be located on one or more racks. The coordination, allocation and management of performing these functions on various processing entities are performed by the distribution synchronization layer. In this way, the execution of these functions is controlled by the distribution synchronization layer to generate media (e.g., video frames, audio, etc.) for the game application in response to the player's controller input. The distributed synchronization layer enables these functions to be efficiently performed (eg, via load balancing) across distributed processing entities so that key game engine components / functions are distributed and reassembled for more efficient processing.

[0049] Figure 2 2 is a schematic diagram of an exemplary multi-GPU architecture 200 in which multiple GPUs collaborate to render a single image of a corresponding application according to one embodiment of the present disclosure. It should be understood that in various embodiments of the present disclosure, many architectures are possible in which multiple GPUs collaborate to render a single image, although not explicitly described or shown. For example, multi-GPU rendering of geometry for an application by performing region testing at rendering time can be implemented between one or more cloud gaming servers of a cloud gaming system, or can be implemented within a standalone system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), etc.

[0050] The multi-GPU architecture 200 includes a CPU 163 and multiple GPUs configured to perform multi-GPU rendering for a single image of an application and / or each image in an image sequence of an application. In particular, the CPU 163 and GPU resources 365 are configured to perform processor-based functions including 2D or 3D rendering, physical simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformations, artificial intelligence, etc., as described above.

[0051] For example, four GPUs are shown in GPU resources 365 of multi-GPU architecture 200, although any number of GPUs may be used when rendering images for an application. Each GPU is connected to a corresponding dedicated memory, such as random access memory (RAM), via a high-speed bus 220. In particular, GPU-A is connected to memory 210A (e.g., RAM) via bus 220, GPU-B is connected to memory 210B (e.g., RAM) via bus 220, GPU-C is connected to memory 210C (e.g., RAM) via bus 220, and GPU-D is connected to memory 210D (e.g., RAM) via bus 220.

[0052] In addition, each GPU is connected to each other via bus 240, which, depending on the architecture, may be approximately equal to or slower in speed than bus 220 used for communication between the corresponding GPU and its corresponding memory. For example, GPU-A is connected to each of GPU-B, GPU-C, and GPU-D via bus 240. Likewise, GPU-B is connected to each of GPU-A, GPU-C, and GPU-D via bus 240. In addition, GPU-C is connected to each of GPU-A, GPU-B, and GPU-D via bus 240. Further, GPU-D is connected to each of GPU-A, GPU-B, and GPU-C via bus 240.

[0053] The CPU 163 is connected to each GPU via a lower speed bus 230 (e.g., bus 230 is slower than bus 220 used for communication between the corresponding GPU and its corresponding memory). Specifically, the CPU 163 is connected to each of GPU-A, GPU-B, GPU-C, and GPU-D.

[0054] In some embodiments, the four GPUs are discrete GPUs, each on their own silicon. In other embodiments, the four GPUs can share a single die in order to take advantage of the high-speed interconnects and other units on the die. In other embodiments, there is one physical GPU 250 that can be configured to function as a single more powerful GPU or as four less powerful "virtual" GPUs (GPU-A, GPU-B, GPU-C, and GPU-D). That is, GPU-A, GPU-B, GPU-C, and GPU-D each have enough power to operate a graphics pipeline (e.g., Figure 4 As shown), the chip as a whole can operate a graphics pipeline (such as Figure 4 ), and the configuration can flexibly switch between the two configurations (e.g. between rendering passes).

[0055] Figure 3is a schematic diagram of a graphics processing unit resource 365 configured for multi-GPU rendering of geometry for an application when rendering a single image through region testing at render time according to one embodiment of the present disclosure. For example, the game server 160 may be configured to Figure 1 The cloud gaming network 190 includes GPU resources 365. As shown, the GPU resources 365 include multiple GPUs, such as GPU 365a, GPU 365b...GPU 365n. As described above, various architectures may include multiple GPUs collaborating to render a single image by performing multi-GPU rendering of geometry for an application through region testing during rendering, such as implementing multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system, or implementing multi-GPU rendering of geometry in an independent system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), and the like.

[0056] In particular, in one embodiment, game server 160 is configured to perform multi-GPU processing when rendering a single image of an application, so that multiple GPUs collaborate to render a single image, and / or render each of one or more images of an image sequence when executing an application. For example, in one embodiment, game server 160 may include a CPU and a GPU group, which is configured to perform multi-GPU rendering to each of one or more images in an image sequence of an application, wherein a CPU and a GPU group can implement graphics and / or rendering pipelines of an application. CPU and a GPU group may be configured as one or more processing devices. As previously mentioned, GPU and a GPU group may include CPU 163 and GPU resources 365, which are configured to perform processor-based functions, including 2D or 3D rendering, physical simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformation, artificial intelligence, etc.

[0057] The GPU resources 365 are responsible for and / or configured to render objects (e.g., write the color or normal vector value of the object pixel to the multiple render target - MRT) and execute synchronous calculation kernels (e.g., full screen effect on the generated MRT); the synchronous calculation to be performed and the objects to be rendered are specified by the commands contained in the multiple rendering command buffers 325 to be executed by the GPU. In particular, the GPU resources 365 are configured to render objects and perform synchronous calculations (e.g., during the execution of synchronous calculation kernels) when executing commands from the rendering command buffers 325, where the commands and / or operations can be dependent on other operations so that they are executed in sequence.

[0058] For example, GPU resources 365 are configured to use one or more rendering command buffers 325 (e.g., rendering command buffer 325a, rendering buffer 325b ... rendering command buffer 325n) to perform synchronous calculations and / or rendering of objects. In one embodiment, each GPU in GPU resources 365 can have its own command buffer. Alternatively, when each GPU is rendering a substantially identical set of objects (e.g., due to the small size of the region), the GPUs in GPU resources 365 can use the same command buffer or the same set of command buffers. In addition, each GPU in GPU resources 365 can support the ability for commands to be executed by one GPU instead of by another GPU. For example, a flag on a drawing command or a prediction in a rendering command buffer allows a single GPU to execute one or more commands in a corresponding command buffer, while other GPUs will ignore these commands. For example, rendering command buffer 325a can support flag 330a, rendering command buffer 325b can support flag 330b ... rendering command buffer 325n can support flag 330n.

[0059] The performance of synchronous computation (e.g., execution of synchronous computation kernels) and rendering of objects is part of the overall rendering. For example, if a video game runs at 60 Hz (e.g., 60 frames per second), all object rendering and execution of synchronous computation kernels for an image frame must typically be completed within approximately 16.67 milliseconds (e.g., one frame at 60 Hz). As previously described, the operations performed when rendering objects and / or executing synchronous computation kernels are ordered such that operations may be dependent on other operations (e.g., commands in a rendering command buffer may need to complete execution before other commands in the rendering command buffer can execute).

[0060] In particular, each rendering command buffer 325 contains various types of commands, including commands that affect the configuration of the corresponding GPU (e.g., commands that specify the location and format of the rendering target), and commands that render objects and / or execute synchronous computing kernels. For illustrative purposes, the synchronous computing performed when executing synchronous computing kernels may include performing full screen effects when objects are all rendered to one or more corresponding multiple rendering targets (MRTs).

[0061] In addition, when GPU resources 365 are rendering objects for image frames, and / or executing synchronous computing kernels when generating image frames, GPU resources 365 are configured via registers of each GPU 365a, 365b ... 365n. For example, GPU 365a is configured via its registers 340 (e.g., registers 340a, registers 340b ... registers 340n) to perform the rendering or computing kernel execution in a certain manner. That is, when executing commands in rendering command buffer 325 for rendering objects and / or executing synchronous computing kernels for image frames, the values ​​stored in registers 340 define the hardware context (e.g., GPU configuration or GPU state) for GPU 365a. Each GPU in GPU resource 365 may be similarly configured such that GPU 365b is configured via its registers 350 (e.g., registers 350a, 350b, ... registers 350n) to perform the rendering or compute kernel execution in a certain manner; ... and GPU 365n is configured via its registers 370 (e.g., registers 370a, 370b, ... registers 370n) to perform the rendering or compute kernel execution in a certain manner.

[0062] Some examples of GPU configuration include the location and format of the render target (e.g., MRT). In addition, other examples of GPU configuration include operating procedures. For example, when rendering an object, the Z value of each pixel of the object can be compared to the Z buffer in various ways. For example, the object pixel is written only when the object Z value matches the value in the Z buffer, indicating that the object pixel is not occluded. Alternatively, the object pixel can be written only when the object Z value is equal to or less than the value in the Z buffer, indicating that the object pixel is not occluded. The type of test being performed is defined in the GPU configuration.

[0063] Figure 4 is a simplified diagram of a rendering architecture that implements a graphics pipeline 400 configured for multi-GPU processing so that multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure. The graphics pipeline 400 illustrates a general process for rendering an image using a 3D (three-dimensional) polygon rendering process. The graphics pipeline 400 for rendering an image outputs corresponding color information for each pixel in a display, where the color information may represent texture and shading (e.g., color, shading, etc.). The graphics pipeline 400 may be configured for multi-GPU processing such that multiple GPUs collaborate to render a single image. Figure 1 and Figure 3The multi-GPU rendering of the geometry is implemented in the client device 110, the game server 160, the game title processing engine 111 and / or the GPU resource 365. That is, various architectures may include multiple GPUs collaborating to render a single image by performing multi-GPU rendering of geometry for an application through region testing at rendering time, such as implementing multi-GPU rendering of geometry between one or more cloud game servers of a cloud game system, or implementing multi-GPU rendering of geometry in an independent system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), etc.

[0064] As shown, the graphics pipeline receives input geometry 405. For example, the geometry processing stage 410 receives the input geometry 405. For example, the input geometry 405 may include vertices within a 3D game world, and information corresponding to each vertex. A given object in the game world may be represented using polygons (e.g., triangles) defined by the vertices, where the surfaces of the corresponding polygons are then processed by the graphics pipeline 400 to achieve the final effect (e.g., color, texture, etc.). Vertex attributes may include normals (e.g., which direction is perpendicular to the geometry at that location), colors (e.g., RGB - red, green, and blue triples, etc.), and texture coordinate / mapping information.

[0065] The geometry processing stage 410 is responsible for (and can perform) both vertex processing (e.g., via a vertex shader) and primitive processing. In particular, the geometry processing stage 410 can output a set of vertices that define primitives and pass them to the next stage of the graphics pipeline 400, as well as the positions (homogeneous coordinates, to be precise) and various other parameters of these vertices. These positions are placed in a position cache 450 for access by the following shader stage. Other parameters are placed in a parameter cache 460, again for access by the following shader stage.

[0066] The geometry processing stage 410 can perform various operations, such as performing lighting and shading calculations on primitives and / or polygons. In one embodiment, because the geometry stage can process primitives, it can perform backface culling and / or clipping (e.g., testing against a view frustum), thereby reducing the load on downstream stages (e.g., rasterization stage 420, etc.). In another embodiment, the geometry stage can generate primitives (e.g., having equivalent functionality to a traditional geometry shader).

[0067] The primitives output by the geometry processing stage 410 are fed to the rasterization stage 420, which converts the primitives into a raster image composed of pixels. In particular, the rasterization stage 420 is configured to project objects in the scene into a two-dimensional (2D) image plane defined by a viewing position in the 3D game world (e.g., camera position, user eye position, etc.). At a simple level, the rasterization stage 420 looks at each primitive and determines which pixels are affected by the corresponding primitive. In particular, the rasterizer 420 divides the primitive into pixel-sized fragments, where each fragment corresponds to a pixel in the display. It is important to note that when the image is displayed, one or more fragments may affect the color of the corresponding pixel.

[0068] As previously described, the rasterization stage 420 may also perform additional operations such as clipping (identifying and ignoring fragments outside the viewing frustum) and culling (ignoring fragments occluded by closer objects) for the viewing position. With respect to clipping, the geometry processing stage 410 and / or the rasterization stage 420 may be configured to identify and ignore primitives outside the viewing frustum defined by the viewing position in the game world.

[0069] The pixel processing stage 430 uses the parameters created by the geometry processing stage and other data to generate values ​​such as the resulting color of the pixel. In particular, the pixel processing stage 430 performs shading operations on the fragments at its core to determine how the color and brightness of the primitives vary with the available lighting. For example, the pixel processing stage 430 can determine the depth, color, normal, and texture coordinates (e.g., texture details) of each fragment, and can further determine the appropriate brightness, darkness, and color levels of the fragment. In particular, the pixel processing stage 430 calculates the characteristics of each fragment, including color and other attributes (e.g., z depth represents the distance from the viewing position, and alpha value represents transparency). In addition, the pixel processing stage 430 applies lighting effects to the fragments based on the available lighting that affects the corresponding fragments. In addition, the pixel processing stage 430 can apply shadow effects to each fragment.

[0070] The output of the pixel processing stage 430 includes the processed fragments (e.g., texture and shading information) and is passed to the output merging stage 440 in the next stage of the graphics pipeline 400. The output merging stage 440 generates the final color for the pixel using the output of the pixel processing stage 430 and other data such as values ​​already in memory. For example, the output merging stage 440 can optionally blend the values ​​between fragments and / or pixels determined from the pixel processing stage 430 with the value of the MRT already written to the pixel.

[0071] The color values ​​of each pixel in the display may be stored in a frame buffer (not shown). When displaying a corresponding image of a scene, these values ​​are scanned to the corresponding pixels. In particular, the display reads the color values ​​from the frame buffer for each pixel row by row from left to right or right to left, from top to bottom or bottom to top, or in any other pattern, and uses these pixel values ​​to illuminate the pixels when displaying the image.

[0072] use Figure 1-Figure 3 A detailed description of the cloud gaming network 190 (e.g., in the game server 160) and GPU resources 365, Figure 5 Flowchart 500 illustrates a graphics processing method for implementing multi-GPU rendering of geometry for an application through region testing at rendering time according to one embodiment of the present disclosure. In this way, multiple GPU resources are used to perform rendering of objects and / or synchronize computing operations when executing an application. As previously described, various architectures may include multiple GPUs collaborating to render a single image by performing multi-GPU rendering of geometry for an application through region testing at rendering time, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), and the like.

[0073] At 510, the method includes rendering graphics for an application using multiple graphics processing units (GPUs) that collaborate to generate images. In particular, multi-GPU processing is performed when rendering a single image and / or each of one or more images in an image sequence for a real-time application.

[0074] At 520, the method includes dividing the responsibility of rendering the geometry of the graphics among the multiple GPUs based on the multiple screen regions. That is, each GPU has a corresponding division of responsibilities (eg, corresponding screen regions) known to all GPUs.

[0075] The GPUs render corresponding images simultaneously. When rendering geometry, the GPU generates information about the geometry and its relationship to the screen areas that other GPUs are responsible for. This allows other GPUs to render geometry more efficiently, and / or avoid rendering the geometry altogether. In other words, a GPU that is "ahead" of other GPUs in rendering can provide information to other GPUs, thereby speeding up their processing. In particular, at 530, the method includes generating information about a first screen area in which the geometry has a first responsibility division relative to the first GPU while rendering a geometry for an image at a second GPU. More specifically, each GPU is responsible for rendering geometry in a corresponding set of screen areas of a plurality of screen areas, wherein the corresponding set of screen areas includes one or more screen areas. For example, the first GPU has a first responsibility division for rendering objects in a first set of screen areas (e.g., including the first screen area). In addition, the second GPU has a second responsibility division for rendering objects in a second set of screen areas. In this way, each GPU starts rendering objects simultaneously, wherein a GPU that is ahead in rendering objects can generate a hint about a geometry that helps another GPU that may or may not be responsible for rendering the geometry.

[0076] For example, when rendering several geometries, a GPU may test the geometries relative to screen regions that other GPUs are responsible for. The test results (e.g., information) are sent to those other GPUs. In particular, when rendering, a second GPU may test the geometry of an image. The second GPU generates information for the first GPU about the geometry relative to one or more screen regions that the first GPU is assigned to be responsible for. For example, the test may include whether a geometry overlaps one or more screen regions that the first GPU is assigned to be responsible for.

[0077] At 540, the method includes using the information to render the geometry at the first GPU. Specifically, in some architectures, if the corresponding rendering GPU receives the corresponding information in time to use it, the GPU will use the information when deciding which geometry in the corresponding image to render. That is, the information can serve as a hint. Otherwise, the rendering GPU will process the geometry as usual. Using an example in which the information can indicate whether the geometry overlaps with one or more screen regions assigned to the first GPU (e.g., the rendering GPU), if the information indicates that the geometry does not overlap, the first GPU can completely skip rendering the geometry. In addition, if the information indicates that only a few geometries do not overlap, the first GPU can skip those geometries that do not overlap with one or more screen regions that the first GPU is assigned to be responsible for. On the other hand, the information may indicate that there is an overlap in the geometry, in which case the first GPU will render the geometry. In addition, the information may indicate that some geometries overlap with one or more screen regions that the first GPU is assigned to be responsible for. In this case, the first GPU will only render those overlapping geometries. In another embodiment, if there is no information, or if the information is not generated or received in time, the first GPU will perform rendering (e.g., rendering geometry) normally. Therefore, if the information provided as the prompt is received in time, the overall efficiency of the graphics processing system can be improved. In one embodiment, if the information is not received in time, the graphics processing system will still operate normally without this information.

[0078] Figure 6A-6B The rendering of the screen subdivided into regions is shown purely for illustrative purposes. It should be understood that the number of subdivided regions may be selected for efficient multi-GPU processing of each of the image and / or one or more images in the image sequence. That is, the plurality of subdivided regions may include two or more regions. In one embodiment of the present disclosure, the screen is subdivided into four quadrants, such as Fig. 6A In another embodiment of the present disclosure, the screen is subdivided into a larger number of staggered regions, such as Figure 6B The following Figure 6A-6B The purpose of this discussion is to illustrate the inefficiencies that arise when performing multi-GPU rendering on multiple screen regions that have multiple GPUs assigned to them; Figure 7A-7C and Figure 8A-8B More efficient rendering according to some embodiments of the present invention is shown.

[0079] In particular, Fig. 6A6 is a schematic diagram of a screen 610A that is subdivided into quadrants (e.g., four regions) when performing multi-GPU rendering. As shown, screen 610A is subdivided into four quadrants (e.g., A, B, C, and D). Each quadrant is assigned to one of four GPUs [GPU-A, GPU-B, GPU-C, and GPU-D] in a one-to-one relationship. For example, GPU-A is assigned to quadrant A, GPU-B is assigned to quadrant B, GPU-C is assigned to quadrant C, and GPU-D is assigned to quadrant D.

[0080] Geometry can be culled. For example, CPU 163 can check the bounding box of the frustum of each quadrant and request each GPU to render only the objects that overlap with its corresponding frustum. The result is that each GPU is responsible for rendering only a portion of the geometry. For illustrative purposes, screen 610 shows several geometries, each of which is a corresponding object, where screen 610 shows objects 611-617 (e.g., several geometries). GPU-A will not render any objects because no objects overlap with quadrant A. GPU-B will render objects 615 and 616 (since a portion of object 615 exists in quadrant B, the CPU's culling test will correctly conclude that GPU-B must render it). GPU-C will render objects 611 and 612. GPU-D will render objects 612, 613, 614, 615, and 617.

[0081] like Fig. 6A As shown, when screen 610A is divided into quadrants AD, the amount of work that each GPU must perform may be very different because, in some cases, a disproportionate amount of geometry may be in one quadrant. For example, quadrant A may not have any geometry, while quadrant D may have five geometries, or at least a portion of five geometries. As a result, GPU-A assigned to quadrant A will be idle, while GPU-D assigned to quadrant D will be unusually busy rendering objects in the corresponding image.

[0082] Figure 6B Another technique for subdividing the screen into regions is illustrated. In particular, screen 610B is not subdivided into quadrants, but rather into multiple interlaced regions when multi-GPU rendering is performed on each of a single image or one or more images in a sequence of images. In that case, screen 610B is subdivided into a larger number of interlaced regions (e.g., greater than four quadrants) while being rendered using the same number of GPUs (e.g., four). Objects (611-617) shown in screen 610A are also displayed in the same corresponding locations in screen 610B.

[0083] In particular, four GPUs [GPU-A, GPU-B, GPU-C, and GPU-D] are used to render images for corresponding applications. Each GPU is responsible for rendering the geometry that overlaps with the corresponding region. That is, each GPU is assigned to a group of corresponding regions. For example, GPU-A is responsible for each region marked as A in the corresponding group, GPU-B is responsible for each region marked as B in the corresponding group, GPU-C is responsible for each region marked as C in the corresponding group, and GPU-D is responsible for each region marked as D in the corresponding group.

[0084] In addition, these regions are staggered in a specific pattern. Due to the staggering of regions (and a greater number of regions), the amount of work that each GPU must perform may be more balanced. For example, the staggered pattern of screen 610B includes alternating rows, including regions ABAB, etc., and regions CDCD, etc. Other patterns of staggered regions are supported in embodiments of the present disclosure. For example, a pattern may include a repeated sequence of regions, evenly distributed regions, an uneven distribution of regions, repeated rows of a sequence of regions, a random sequence of regions, random rows of a sequence of regions, etc.

[0085] Choosing the number of regions is important. For example, if the distribution of regions is too fine (e.g., the number of regions is too large to optimize), each GPU must still process part or all of a large geometry. For example, it may be difficult to check an object bounding box against all of the regions that a GPU is responsible for. Furthermore, even if the bounding boxes can be checked in a timely manner, due to the small region sizes, the result will be that each GPU may have to process a large portion of the geometry because each object in the image overlaps with at least one region of each GPU (e.g., a GPU processes an entire object even if only a portion of the object overlaps with at least one region in the set of regions assigned to that GPU).

[0086] Therefore, it is important to choose the number of regions, interleaving mode, etc. Choosing too few or too many regions, or too few or too many regions for interleaving, or choosing an inefficient mode for interleaving may result in inefficiencies in GPU processing (e.g., each GPU processes most or all of the geometry). In these cases, even if multiple GPUs are used to render the image, the corresponding increase in screen pixel count and geometry density cannot be supported due to GPU inefficiencies (i.e., four GPUs cannot write four times as many pixels and process four times as many vertices or primitives). The following implementation is for the culling strategy ( Figure 7A-7C ) and rejection granularity ( Figure 8A-8B ) and other advances.

[0087] Figure 7A-7C1 is a schematic diagram illustrating the use of multiple GPUs to render a single image and / or each of at least one or more images in an image sequence in an embodiment of the present disclosure. Four GPUs are selected purely for the convenience of illustrating multi-GPU rendering when rendering images when executing an application, and it is understood that in various embodiments, any number of GPUs can be used for multi-GPU rendering.

[0088] In particular, Fig. 7A 700A is a schematic diagram of a rendering command buffer 700A shared by multiple GPUs collaborating to render a single image according to one embodiment of the present disclosure. That is, in this example, multiple GPUs each use the same rendering command buffer (e.g., buffer 700A), and each GPU executes all commands in the rendering command buffer. Multiple commands (complete sets) are loaded into rendering command buffer 700A and used to render the corresponding image. It is understood that one or more rendering command buffers can be used to generate the corresponding image. In one example, the CPU generates one or more draw calls for an image frame, where the draw call includes commands placed in one or more rendering command buffers for Figure 3 The CPU 163 may request one or more GPUs to generate all or some of the draw calls used to render the corresponding image. In addition, the entire command set may be executed in Fig. 7A and are all contained in rendering command buffer 700A, or Fig. 7A A portion of the entire command set contained within rendering command buffer 700A may be shown.

[0089] When performing multi-GPU rendering on an image or each of one or more images in an image sequence, the GPUs render simultaneously. The rendering of an image can be divided into multiple stages. In each stage, the GPUs need to be synchronized so that the faster GPU must wait until the slower GPU is finished. Fig. 7A The commands for rendering command buffer 700A shown in FIG. 7A show one stage, where rendering command buffer 700A may include commands for one or more stages in rendering an image. Fig. 7A The illustrated diagram shows a rendering command buffer 700A of a stage, in which there are four objects to be rendered (eg, object 0, object 1, object 2, and object 3). Figure 7B-1 shown.

[0090] As shown in the figure, Fig. 7AThe rendering command buffer 700A shown in includes commands for rendering objects and commands for configuring the state of one or more rendering GPUs that are executing commands from the rendering command buffer 700A. The commands can be further divided into command segments, where each segment includes one or more commands. The commands for object rendering and / or synchronous calculation are executed as part of the rendering of the corresponding image (e.g., image frame). In some embodiments, the object rendering and / or execution of the synchronous calculation kernel for the image must be performed within a frame period. For illustrative purposes only, Fig. 7A The rendering command buffer 700A shown in FIG. 7 includes eight command segments ( 710 - 717 ) for rendering objects and / or executing synchronous computing kernels when rendering a corresponding image for a corresponding application.

[0091] For example, Fig. 7A The rendering command buffer 700A shown in the figure includes four command segments 710, 712, 714 and 716, each of which is used to configure the state of one or more rendering GPUs that are executing the commands in the rendering command buffer 700A. As previously described, when the commands in the rendering command buffer 700A for rendering objects for corresponding images and / or executing synchronous computing kernels are executed, the values ​​stored in the registers define the hardware context (e.g., GPU configuration) of the corresponding GPU. As shown, the GPU state can be modified throughout the entire process of command processing in the rendering command buffer 700A, and each subsequent command segment can be used to configure the GPU state.

[0092] also, Fig. 7A The illustrated rendering command buffer 700A includes command segments 711, 713, 715, and 717 for rendering corresponding objects. For example, one or more commands in segment 711 are used to draw and / or render object 0, one or more commands in segment 713 are used to draw and / or render object 1, one or more commands in segment 715 are used to draw and / or render object 2, and one or more commands in segment 717 are used to draw and / or render object 3. Fig. 7A A segment for configuring the GPU state is shown preceding each of the command segments for object rendering, but it should be understood that the GPU state may be configured to render one or more objects.

[0093] Multiple GPUs share information (e.g., "hints") to help determine which objects need to be rendered by the corresponding GPU. Each GPU may be responsible for rendering within a specific area of ​​the screen (e.g., a set of areas assigned to the corresponding GPU for object rendering), where the corresponding GPU renders objects that completely and / or partially overlap the area or the set of areas. During rendering, the second GPU may generate hints about objects and their locations within the screen. As shown, hint 721 may be generated by the second GPU when performing a rendering operation for object 0, and may be provided to each of the other GPUs. Similarly, hint 723 may be generated by the second GPU when performing a rendering operation for object 1, and may be provided to each of the other GPUs. In addition, hint 725 may be generated by the second GPU when performing a rendering operation for object 2. Similarly, hint 727 may be generated by the second GPU when performing a rendering operation for object 3.

[0094] In particular, during rendering, the second GPU may test at least one of the objects (e.g., geometry) in the image relative to the screen area for which the other GPUs have rendering responsibility. Information including the test results may be sent to those other GPUs so that those other GPUs may use the information (if received in a timely manner) to reduce their own testing of the objects and / or render the objects efficiently. For example, the information may indicate that the first GPU should completely skip rendering a geometry (e.g., the geometry does not overlap with the screen area for which the first GPU is assigned responsibility for object rendering). In another example, the information is provided to the first GPU as a hint so that if the information is received before rendering of the first geometry at the first GPU begins, the first GPU considers the information. In one embodiment, if the information is not received in a timely manner, the first geometry is rendered completely by the first GPU.

[0095] For illustration purposes only, the four GPUs divide the corresponding screen into regions between them. The arrows show notification of other GPUs of region testing. For example, a second GPU generating a hint during rendering provides the hint (e.g., information) to other GPUs (e.g., the first GPU). As previously described, each GPU is responsible for rendering objects in a corresponding set of regions, where the corresponding set includes one or more regions.

[0096] In one embodiment, rendering command buffer 700A is shared by multiple GPUs that collaborate to render a single image. That is, GPUs used for multi-GPU rendering of each of a single image or one or more images in an image sequence share a common command buffer. In this way, a second GPU that provides object hints to another GPU (e.g., a first GPU) that may or may not render the same object each shares the same command buffer. In another embodiment, each GPU may have its own command buffer.

[0097] Alternatively, in yet another embodiment, each GPU may be rendering slightly different groups of objects. This may be the case when it can be determined that a particular GPU does not need to render a particular object because it does not overlap with its corresponding screen area in the corresponding group. Multiple GPUs can still use the same command buffer (e.g., share a command buffer) as long as the command buffer supports the ability of commands to be executed by one GPU rather than another GPU, as described above. For example, the execution of commands in the shared rendering command buffer 700A can be limited to one of the rendering GPUs. This can be achieved in a variety of ways. In another example, a flag can be used in the corresponding command to indicate which GPUs should execute it. In addition, bits can be used in the rendering command buffer to implement predictions to indicate which GPU does what under what conditions. Examples of predictions include-"If this is GPU-A, skip the following X commands".

[0098] In another embodiment, since each GPU is rendering substantially the same set of objects, multiple GPUs can still use the same command buffer. For example, when the area is relatively small, each GPU can still render all objects, as described above.

[0099] Figure 7B-1 Screen 700B is illustrated, showing a plurality of GPUs using Fig. 7A The rendering command buffer 700A renders an image of four objects. Multi-GPU rendering of geometry is performed for the application through region testing during the rendering of the geometry in the corresponding image.

[0100] In particular, rendering responsibility for geometry is divided among multiple GPUs by screen regions, where the multiple screen regions are configured to reduce rendering time imbalances among the multiple GPUs. For example, screen 700B illustrates the screen region responsibility of each GPU when rendering an image object. Four GPUs (GPU-A, GPU-B, GPU-C, and GPU-D) are used to render the objects in the image shown in screen 700B. Screen 700B, for example Fig. 6AThe quadrants are more finely divided as shown, in an effort to balance the pixel and vertex load between the GPUs. In addition, screen 700B is divided into staggered regions. For example, the staggering includes multiple rows of regions. Each of rows 731 and 733 includes region A alternating with region B. Each of rows 732 and 734 includes region C alternating with region D. More specifically, in one pattern, rows including regions A and B alternate with rows including regions C and region D.

[0101] As previously described, in order to achieve GPU processing efficiency, various techniques may be used when dividing the screen into regions, such as increasing or decreasing the number of regions (e.g., selecting the correct number of regions), interleaving regions, increasing or decreasing the number of regions for interleaving, selecting a specific mode when interleaving regions, etc. In one embodiment, the size of each of the plurality of screen regions is uniform. In one embodiment, each of the plurality of screen regions is non-uniform in size. In yet another embodiment, the number and size of the plurality of screen regions changes dynamically.

[0102] Each of the GPUs is responsible for rendering objects in a corresponding set of regions, where each set may include one or more regions. Thus, GPU-A is responsible for rendering objects in each A region in the corresponding set, GPU-B is responsible for rendering objects in each B region in the corresponding set, GPU-C is responsible for rendering objects in each C region in the corresponding set, and GPU-D is responsible for rendering objects in each D region in the corresponding set. There may also be GPUs that have other responsibilities such that they may not perform rendering (e.g., executing asynchronous compute kernels that execute over multiple frame cycles, performing culling for a rendering GPU, etc.).

[0103] The amount of rendering to be performed is different for each GPU. Figure 7B-2 FIG. 1 shows an example of a method for rendering a Figure 7B-1 3. The table below shows the rendering performed by each GPU for the four objects in the image. As shown in the table, GPU A renders object 2; GPU B renders objects 0, 2, and 3; GPU C renders objects 1 and 3; and GPU D renders objects 1, 2, and 3. There may still be some imbalanced rendering because GPU A only needs to render object 2, while GPU D needs to render objects 1, 2, and 3. However, overall, through the interleaving of screen areas, the rendering of objects within an image is reasonably balanced among multiple GPUs used for multi-GPU rendering of an image or rendering of each of one or more images in an image sequence.

[0104] Figure 7C is a diagram illustrating a method for rendering a video image in which multiple GPUs collaborate to render the video image according to an embodiment of the present disclosure. Figure 7B-1The single image shown is a schematic diagram of the rendering performed by each GPU for each object. In particular, Figure 7C The use of each of the four GPUs (eg, GPU-A, GPU-B, GPU-C, and GPU-D) is shown. Fig. 7A The shared rendering command buffer 700A performs the rendering process of objects 0-3.

[0105] In particular, two rendering timing diagrams are shown relative to timeline 740. Rendering timing diagram 700C-1 shows multi-GPU rendering of objects 0-3 of a corresponding image in one rendering phase, where each GPU performs rendering without any hints regarding overlap between objects 0-3 and a screen area. Rendering timing diagram 700C-2 shows multi-GPU rendering of objects 0-3 of a corresponding image in the same rendering phase, where hints are shared during each GPU's rendering regarding overlap between objects 0-3 and a screen area. Each of rendering timing diagrams 700C-1 and 700C-2 shows the time it takes each GPU to process each geometry. In one embodiment, a geometry is a complete object. In another embodiment, a geometry can be a portion of an object. For purposes of illustration, Figure 7C The example of shows the rendering of several geometries, where each geometry corresponds to an object (e.g., in its entirety). In each of the rendering timing diagrams 700C-1 and 700C-2, objects (e.g., several geometries) that do not have geometries (e.g., primitives of the object) that overlap with at least one screen region of the corresponding GPU (e.g., in a corresponding set of regions) are represented by boxes drawn with dashed lines. On the other hand, objects that have geometries that overlap with at least one screen region of the corresponding GPU (e.g., in a corresponding set of regions) are represented by boxes drawn with solid lines.

[0106] Rendering timing diagram 700C-1 shows the rendering of objects 0-3 using four GPUs (e.g., GPU-A, GPU-B, GPU-C, and GPU-D). Vertical line 755a indicates the beginning of the rendering phase of an object, while vertical line 755b shows the end of the rendering phase of the object in rendering timing diagram 700C-1. The starting and ending points of the rendering phases shown along timeline 740 represent synchronization points, where each of the four GPUs is synchronized when executing the corresponding GPU pipeline. For example, at vertical line 755b indicating the end of the rendering phase, all GPUs must wait for the slowest GPU (e.g., GPU-B) to complete the rendering of objects 0-3 through the corresponding graphics pipeline before entering the next rendering phase.

[0107] No hints are provided when rendering the objects shown in the rendering timing diagram 700C-1. Therefore, each GPU must process each object through the corresponding graphics pipeline. If there are no pixels to be drawn for the object in the area assigned to the corresponding GPU for object rendering (e.g., in a corresponding set of areas), the GPU may not fully render the object through the graphics pipeline. For example, when an object does not overlap, only the geometry processing stage of the graphics pipeline is executed. However, without hints, this still takes some time to process.

[0108] In particular, GPU-A does not completely render objects 0, 1, and 3 because they do not overlap any screen area assigned to GPU-A for object rendering. The rendering of these three objects is shown in the dashed boxes, indicating that at least the geometry processing stage is executed, but the graphics pipeline is not fully executed. GPU-A completely renders object 2 because the object overlaps at least one screen area assigned to GPU-A for rendering (e.g., in the corresponding group). The rendering of object 2 is shown in the solid box, indicating that all stages of the corresponding graphics pipeline are executed. Similarly, GPU-B does not completely render object 1 (indicated by the dashed box) (i.e., at least the geometry processing stage is executed), but completely renders objects 0, 2, and 3 (indicated by the solid box) because those objects overlap at least one screen area assigned to GPU-B for rendering (e.g., in the corresponding group). Similarly, GPU-C does not completely render objects 0 and 2 (indicated by the dashed boxes) (i.e., at least the geometry processing stage is executed), but completely renders objects (indicated by the solid box) because those objects overlap at least one screen area assigned to GPU-C for rendering (e.g., in the corresponding group). Furthermore, GPU-D does not fully render object 0 (represented by the dashed box) (i.e., performs at least the geometry processing stage), but does fully render objects 1, 2, and 3 (represented by the solid boxes) because those objects overlap at least one screen area allocated to GPU-D for rendering (e.g., in the corresponding group).

[0109] Rendering timing diagram 700C-2 shows the rendering of objects 0-3 when multi-GPU rendering is performed with shared hints. Specifically, the hints are shared between each GPU (e.g., GPU-A, GPU-B, GPU-C, and GPU-D) and are taken into account when rendering objects 0-3 through the corresponding GPU pipeline. Vertical line 750a indicates the beginning of the rendering phase of the object, while vertical line 750b shows the end of the rendering phase of the object in rendering timing diagram 700C-2. The starting and ending points of the rendering phases shown along timeline 740 in timing diagram 700C-2 represent synchronization points, where each of the four GPUs is synchronized when executing the corresponding GPU pipeline, as described above. For example, at vertical line 750b indicating the end of the rendering phase, all GPUs must wait for the slowest GPU (e.g., GPU-B) to complete the rendering of objects 0-3 through the corresponding graphics pipeline before entering the next rendering phase.

[0110] Vertical line 750a is aligned with vertical line 755a so that each of rendering timing diagrams 700C-1 and 700C-2 starts to render object 0-1 at the same time. However, the rendering of object 0-3 shown in rendering timing diagram 700C-2 is performed in a shorter time than the rendering shown in rendering timing diagram 700C-1. That is, vertical line 750b indicating the end of the rendering phase of lower timing diagram 700C-2 appears earlier than the end of the rendering phase of upper timing diagram 700C-1 as indicated by vertical line 755b. Specifically, when performing regional tests and providing prompts for the results of those tests, when performing multi-GPU rendering of the geometry of the image for the application, a speed increase 745 when rendering objects 0-3 is achieved. As shown, speed increase 745 is the time difference between vertical line 750b of timing diagram 700C-2 and vertical line 755b of timing diagram 700C-1.

[0111] The speed increase is achieved through the generation and sharing of hints. In particular, as the GPUs process objects through the graphics pipeline, each GPU examines the object and sends information about how the object relates to the screen responsibilities of those GPUs as a hint to the other GPUs. For example, three hints 761-763 are provided in the rendering timing diagram 700C-2, which reduces the total time used to render objects 0-3. Hints can be represented as arrows, where corresponding arrows show that one GPU notifies the second GPU that it can skip an object, and the second GPU successfully skips that object. Hints can be generated at any point during rendering (e.g., during the geometry processing stage) so that even if a GPU has not fully rendered an object (i.e., through all stages of the graphics pipeline), the GPU can still generate a hint about the object and provide the hint to other GPUs for rendering.

[0112] For example, GPU-A performs geometry processing on object 1 and determines that object 1 can be skipped by GPU-B because object 1 does not overlap with any area assigned to GPU-B for object rendering (e.g., in a corresponding group). In addition, object 1 is not completely rendered by GPU-A because it does not overlap with any area assigned to GPU-A for object rendering (e.g., in a corresponding group). Since it is determined that object 1 does not overlap with the area assigned to GPU-B before GPU-B starts geometry processing on object 1, GPU-B skips the rendering of object 1. In particular, GPU-A starts rendering object 1 (e.g., at the geometry processing level) and learns that object 1 does not overlap with any area assigned to GPU-B for object rendering. In this way, GPU-A can generate and pass hint 761 with information indicating that GPU-B does not need to render object 1. Hint 761 is passed in time, i.e., before GPU-B starts rendering object 1. For reference, hint 761 is provided even before GPU-B in the upper timing diagram 700C-1 performs rendering of object 1 (i.e., GPU-B is currently rendering object 0). In this case, GPU-B may completely skip the rendering process of Object 1, thereby not executing any stage of the corresponding graphics pipeline for the object (eg, not executing the geometry processing stage as the first stage).

[0113] Continuing with the example of rendering timing diagram 700C-2, GPU-B completely renders Object 2. During the rendering of Object 2, GPU-B determines (e.g., at the geometry processing stage) that Object 2 does not overlap with any area (e.g., in the corresponding group) allocated to GPU-C for object rendering. Because this determination is made before GPU-C begins geometry processing of Object 2, GPU-C is able to skip the rendering of Object 2. For example, GPU-B may generate and pass a hint 762 with information indicating that GPU-C does not need to render Object 2. Hint 762 is passed in time, i.e., before GPU-C begins rendering Object 2 (e.g., GPU-C is currently rendering Object 1). For reference, hint 762 is provided even before GPU-C in the upper timing diagram 700C-1 performs rendering of Object 2. In this case, GPU-C may completely skip the rendering process of Object 2, thereby not executing any stage of the corresponding graphics pipeline for the object (e.g., not executing the geometry processing stage as the first stage).

[0114] GPU-B also completely renders Object 3. During rendering, GPU-B determines (e.g., at the geometry processing stage) that Object 3 does not overlap with any area allocated to GPU-A for object rendering (e.g., in the corresponding group). Because this determination is made before GPU-A begins geometry processing of Object 3, GPU-A is able to skip rendering of Object 3. For example, GPU-B can generate and pass hint 763 with information indicating that GPU-A does not need to render Object 3. Hint 763 is passed in time, i.e., before GPU-A begins rendering Object 3 (e.g., GPU-A is currently rendering Object 2). In this case, GPU-A can completely skip the rendering process of Object 3, thereby not executing any stage of the corresponding graphics pipeline for the object (e.g., not executing the geometry processing stage as the first stage).

[0115] Figure 8A-8B Object testing for screen areas 820A and 820B is shown, where the screen areas can be interlaced areas (for example, screen areas 820A and 820B show a portion of a display). Specifically, when rendering an object in the screen, multi-GPU rendering of the object is performed for each of a single image or one or more images in an image sequence through area testing. GPU-A is assigned responsibility for rendering the object in screen area 820A. GPU-B is assigned responsibility for rendering the object in screen area 820B. Information is generated for "several geometries", where these geometries can be entire objects or parts of objects. For example, a geometry can be object 810, or several parts of object 810.

[0116] Fig. 8A The diagram illustrates object testing for screen areas when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure. As previously described, several geometries may be objects such that the several geometries correspond to geometries used or generated by corresponding draw calls. During testing, it may be determined that object 810 overlaps with area 820A. That is, portion 810A of object 810 overlaps with area 820A. In this case, GPU-A is tasked with rendering object 810. Similarly, during testing, it may be determined that object 810 overlaps with area 820B. That is, portion 810B of object 810 overlaps with area 820B. In this case, GPU-B is also tasked with rendering object 810.

[0117] Figure 8BThe test of multiple parts of an object for a screen area when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure is illustrated. That is, these several geometries can be parts of an object. For example, object 810 can be split into several pieces so that the geometries used or generated by the draw call are subdivided into smaller geometries. In one embodiment, each of the geometries is roughly the size of the position cache and / or parameter cache allocated for it. In that case, information (e.g., one or more hints) is generated for those smaller geometries, where the information is used by the rendering GPU as described above.

[0118] For example, object 810 is divided into multiple objects so that several geometries used for region testing and hinting correspond to these smaller objects. As shown, object 810 is divided into several geometries "a", "b", "c", "d", "e", and "f". After the region test, GPU-A only renders several geometries "a", "b", "c", "d", and "e". In other words, GPU-A can skip rendering geometry "f". Similarly, after the region test, GPU-B only renders several geometries "d", "e", and "f". In other words, GPU-B can skip rendering geometries "a", "b", and "c".

[0119] In one embodiment, since the geometry processing stage is configured to perform vertex processing and primitive processing, it is possible to perform a test of a geometry in the shader used in the geometry processing stage. For example, the geometry processing stage generates information (e.g., hints), such as by testing the boundary frustum of the geometry for the GPU screen area, which can be performed by software shader operations. In one embodiment, the test is accelerated by using special instructions or instructions implemented by hardware, thereby realizing a software / hardware solution. That is, one or more special instructions are used to accelerate the generation of information about the geometry and its relationship with the screen area. For example, the homogeneous coordinates of the primitive vertices of the geometry are provided as input to the area test instruction of the geometry processing stage. The test can generate a Boolean return value for each GPU, which indicates whether the primitive overlaps with any screen area assigned to the GPU (e.g., in a corresponding group). In this way, information about the corresponding geometry and its relationship with the screen area (e.g., hints) is generated by the shader in the geometry processing stage.

[0120] In another embodiment, the test for a geometry may be performed at the hardware rasterization level. For example, the hardware scan converter may be configured to perform multi-GPU region testing. In particular, since a GPU must test geometry against its own screen region, the hardware may be modified so that the scan converter generates information for screen regions of other GPUs. Thus, the hardware scan converter may be configured to generate information about the corresponding geometry and its relationship to the screen region for which a rendering GPU (e.g., another GPU) is responsible at the rasterization level.

[0121] In yet another embodiment, the geometric figures may be primitives. That is, the portion of the object used for the region test may be a primitive. Thus, the information generated by one GPU as a hint indicates whether each triangle (e.g., representing a primitive) needs to be rendered by another rendering GPU.

[0122] In one embodiment, the information provided as a hint includes the number of primitives (e.g., the surviving primitive count) that overlap with the screen area allocated to the corresponding GPU for object rendering (e.g., in the corresponding group). The information may also include the number of vertices used to construct or define these primitives. In other words, the information includes the surviving vertex count. Therefore, when rendering, the corresponding rendering GPU can use the provided vertex count to allocate space in the position cache and the parameter cache. For example, in one embodiment, unnecessary vertices do not have any allocated space, which can improve rendering efficiency.

[0123] In other embodiments, the information provided as a hint includes specific primitives (e.g., surviving primitives that are exact matches) that overlap the screen area assigned to the corresponding GPU for object rendering (e.g., in a corresponding group). That is, the information generated for the rendering GPU includes a specific set of primitives for rendering. The information may also include specific vertices for constructing or defining these primitives. That is, the information generated for the rendering GPU includes a specific set of vertices for rendering. For example, this information can save time for other rendering GPUs during their geometry processing stage when rendering geometry.

[0124] In other embodiments, there may be processing overhead (software or hardware) associated with generating information. In this case, it may be beneficial to skip generating information as a hint for certain geometries. That is, the information provided as a hint is generated for certain objects, rather than for other objects. For example, the geometry (e.g., objects or portions of objects) representing a skybox or a large area of ​​terrain may include large triangles. In this case, each GPU for multi-GPU rendering of each of one or more images in an image or image sequence may need to render these geometries. That is, information may or may not be generated according to the properties of the corresponding geometry.

[0125] Figure 9A-9C Various strategies for allocating screen regions to corresponding GPUs when multiple GPUs collaborate to render a single image according to one embodiment of the present disclosure are illustrated. In order to achieve GPU processing efficiency, various techniques can be used when dividing the screen into regions, such as increasing or decreasing the number of regions (e.g., selecting the correct number of regions), interleaving regions, increasing or decreasing the number of regions for interleaving, selecting a specific mode when interleaving regions, etc. For example, multiple GPUs are configured to perform multi-GPU rendering of geometry for an application through region testing when rendering objects in corresponding images. Figure 9A-9C The configuration of screen regions in is designed to reduce any rendering time imbalance between multiple GPUs. The complexity of the test (e.g. overlapping corresponding screen regions) varies depending on how the screen regions are allocated to the GPUs. Figure 9A-9C As shown in the schematic diagram shown in , the bold box 910 is the outline of the corresponding screen or display used when rendering the image.

[0126] In one embodiment, each of the multiple screen regions or multiple regions has a uniform size. In one embodiment, each of the multiple screen regions is non-uniform in size. In another embodiment, the number and size of the multiple screen regions change dynamically.

[0127] in particular, Fig.9A A simple mode 900A of screen 910 is illustrated. Each screen region is of uniform size. For example, the size of each region can be a rectangle whose dimensions are a power of 2 pixels. For example, the size of each region can be 256x256 pixels. As shown, the region allocation is a checkerboard pattern, where a row of A and B regions alternates with another row of B and C regions. Mode 900A can be easily tested. However, there may be some rendering inefficiencies. For example, the screen area allocated to each GPU is substantially different (i.e., screen region C and region D have less coverage in screen 910), which may result in an imbalance in rendering time for each GPU.

[0128] Fig. 9B A pattern 900B of screen regions of screen 910 is illustrated. The size of each screen or sub-region is uniform. Screen regions are allocated and distributed to reduce the imbalance of rendering time between GPUs. For example, allocating GPUs to screen regions in pattern 900B results in almost equal amounts of screen pixels allocated to each GPU across screen 910. That is, screen regions are allocated to GPUs to equalize the screen area or coverage in screen 910. For example, if the size of each region is 256x256 pixels, each region has approximately the same coverage in screen 910. Specifically, a set of screen regions A covers an area of ​​6x256x256 pixels, a set of screen regions B covers an area of ​​5.75x256x256 pixels, a set of screen regions C covers an area of ​​5.5x256x256 pixels, and a set of screen regions D covers an area of ​​5.5x256x256 pixels.

[0129] Fig. 9C Pattern 900C of the screen area of ​​screen 910 is illustrated. The size of each screen area is not uniform. That is to say, the size of the screen area for rendering objects assigned to the GPU may not be uniform. In particular, screen 910 is divided so that each GPU is assigned to the same number of pixels. For example, if a 4K display (3840x2160) is vertically evenly divided into four areas, the height of each area is 520 pixels. However, GPUs usually perform many operations in 32x32 pixel blocks, and 520 pixels are not a multiple of 32 pixels. Therefore, in one embodiment, pattern 900C can include blocks with a height of 512 pixels (a multiple of 32), and other blocks with a height of 544 pixels (also a multiple of 32). Other embodiments can use blocks of different sizes. Pattern 900C shows the equal amount of screen pixels assigned to each GPU by using non-uniform screen areas.

[0130] In yet another embodiment, the requirements of an application when performing image rendering vary over time, and the screen region is selected dynamically. For example, if it is known that most of the rendering time is spent in the lower half of the screen, it is advantageous to allocate the region in such a way that an almost equal number of screen pixels in the lower half of the display are allocated to each GPU used to render the corresponding image. In other words, the region allocated to each GPU used to render the corresponding image can be changed dynamically. For example, the application can change based on game mode, different games, screen size, mode selected for the region, etc.

[0131] Fig. 10A1 is a schematic diagram illustrating multi-pass rendering when performing multi-GPU rendering of geometry for an application with region testing at render time according to one embodiment of the present disclosure. The first pass performed by each GPU (GPU-A, GPU-B, GPU-C, and GPU-D) is a Z-only pre-pass 1000A, and similarly Figure 7C 700C-2. The first pass can be considered a first rendering stage. The second pass performed by each GPU is a rendering pass 1000B, in which the GPU fully renders the object while taking into account internal hints generated and used by each corresponding GPU. The second pass can be considered a second rendering stage.

[0132] In particular, information about geometry generated in one rendering phase (e.g., a first pass) is used in another rendering phase (e.g., a second pass). That is, internal hints generated by one GPU for one rendering phase are used by the same GPU for another rendering phase. Therefore, the same geometry is processed twice: once in the Z-only pre-pass phase 1000A and once in the full rendering phase 1000B. Specifically, the pre-pass phase 1000A writes only the Z values ​​of all geometry. Then during the full rendering phase 1000B, the pixel Z value is compared to the value in the previously stored Z buffer, as described above. In one embodiment, if there is an exact match, the pixel shader (e.g., a pixel processing stage) is executed for that pixel.

[0133] and Figure 7C Unlike one GPU that generates information for other GPUs, in this case the GPU can also generate information for itself to use in the next pass. That is, the GPU uses its own information so that the GPU can provide itself with hints to use in the next pass. Although it is useful to know whether a geometry overlaps at least one corresponding screen region for which it is responsible (as previously described in Figure 7C ), but in some embodiments, the GPU may generate more extensive information that can be used by itself in later rendering stages.

[0134] In one embodiment, during the Z-only prepass 1000A, the corresponding GPU rasterizes the primitives output by the geometry processing stage and performs a depth test. In the depth test, the GPU compares the Z value of its pixels with the corresponding Z buffer. The GPU generates (and later provides itself with) information about whether any pixel in the geometry survived the depth test during the first stage. If no pixel survived the test in the first stage, all rendering of the geometry can be skipped in the second stage.

[0135] In another embodiment, during the Z pre-pass phase 1000A only, the pixel shader is executed. For example, the alpha value of the texture is checked, and then pixels with too low alpha values ​​are killed. In particular, the GPU generates (and later provides itself with) information about whether any pixels in the geometry are written during the first phase, so that these pixels survive both the depth test and the pixel kill test. If no pixels are written in the first phase 1000A, all rendering of the geometry can be skipped in the second phase 1000B. For example, arrow 1020 shows that GPU-C passes information about object 3 to itself. This information indicates that GPU-C can skip the rendering of object 3 in the full rendering phase 1000B. In addition, arrow 1030 shows that GPU-D passes information about object 1 to itself. This information indicates that GPU-D can skip the rendering of object 1 in the full rendering phase 1000B. Note that this example focuses on a GPU passing information generated in one stage (e.g., Z pre-pass only stage 1000A) to itself for use in a subsequent stage (e.g., full rendering stage 1000B); in other embodiments, a GPU may also pass information generated in one stage (e.g., Z pre-pass only stage 1000A) to other GPUs so that they can use it in a subsequent stage (e.g., full rendering stage 1000B).

[0136] use Figure 1-Figure 3 A detailed description of the cloud gaming network 190 (e.g., in the game server 160) and GPU resources 365, and Figure 4 - Supporting description of Figure 9, Fig. 10B 1050 illustrates a graphics processing method according to one embodiment of the present disclosure, including multi-pass rendering when implementing multi-GPU rendering of geometry for an application by region testing during rendering. In one embodiment, flowchart 1050 is a method for implementing Fig. 10A In addition, flowchart 1050 may include Figure 5 One or more operations of flowchart 500. As previously described, various architectures may include multiple GPUs collaborating to render a single image by performing multi-GPU rendering of geometry for an application through region testing at render time, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), etc.

[0137] At 1060, the method includes rendering graphics for the application using multiple GPUs that collaborate to generate images. Specifically, multi-GPU processing is performed when rendering a single image and / or each of one or more images in an image sequence for a real-time application.

[0138] At 1065, the method includes dividing responsibility for rendering geometry of the graphics among the plurality of GPUs based on the plurality of screen regions. That is, each GPU has a corresponding division of responsibilities (eg, corresponding screen regions) known to all GPUs.

[0139] Each GPU performs a multi-pass rendering of the geometry of the image, including a first pass as a Z-only pre-pass rendering phase and a second pass as a full rendering phase of the rendering. When the geometry is rendered by the same GPU, the information generated by the GPU in the first pass of the geometry can be used in the second pass.

[0140] More specifically, the method includes determining a Z value of the geometry during a pre-pass rendering phase of the geometry of the image at the GPU at 1070. It may be determined that the geometry overlaps a screen region to which the GPU has responsibility for rendering the object.

[0141] At 1075, the method includes comparing the Z value to a Z buffer value for the geometry. As previously described, the GPU may rasterize primitives for the geometry (e.g., convert the primitives into a raster image composed of pixels). A depth test is performed in which the Z value of a pixel of the geometry is compared to a corresponding Z buffer value (e.g., a value stored therein). For example, upon comparison, it may be determined that the Z value of the geometry is less than or equal to the Z buffer value, indicating that the geometry is not occluded. In this case, any of the information generated below may indicate that the geometry is not occluded.

[0142] At 1080, the method includes generating information including the result of comparing the Z value with the Z buffer value for use by the GPU when rendering the geometry during a full rendering phase of rendering. Specifically, the information may indicate whether any pixel in the geometry survives the depth test. If no pixel survives the depth test, the geometry may be skipped in a second pass (e.g., a rendering phase). For example, the method may include rasterizing one or more primitives of the geometry into a plurality of pixels during a pre-pass phase. A depth test is performed by comparing a plurality of Z values ​​of a plurality of pixels with one or more corresponding Z buffer values, wherein the result of the depth test is included in the information. Pixels that did not survive the depth test in a plurality of pixels are not rendered based on the information. In one embodiment, when none of the plurality of pixels survives the depth test, rendering of the geometry may be skipped. On the other hand, if any pixel in the geometry survives the depth test, the geometry is rendered by the GPU in a second pass (e.g., rendering pixels that survived the depth test).

[0143] In yet another embodiment, the method may include rasterizing one or more primitives of the geometry into a plurality of pixels during a pre-pass phase. Checking alpha values ​​of textures of the plurality of pixels. Killing pixels of the plurality of pixels having alpha values ​​below a threshold, such that a result of the checking is included in the information. In one embodiment, rendering of the geometry may be skipped when none of the plurality of pixels has a corresponding alpha value equal to or above the threshold.

[0144] In some embodiments, the GPU also generates information about the geometry and its relationship to the screen areas that other GPUs are responsible for, and this information is used in subsequent stages (e.g., a full rendering stage of rendering). In other embodiments, no information is generated by comparing the Z value to the Z buffer, but instead the GPU only generates information about the geometry and its relationship to the screen areas that all GPUs (including itself) are responsible for, and this information is used in subsequent stages (e.g., a full rendering stage of rendering).

[0145] Fig.11 Components of an exemplary apparatus 1100 that can be used to perform aspects of various embodiments of the present disclosure are shown. For example, Fig.11 An exemplary hardware system is illustrated according to an embodiment of the present disclosure, which is suitable for multi-GPU rendering of geometry for applications with region testing while rendering objects for images. The block diagram shows a device 1100, which may be incorporated into or may be a personal computer, a server computer, a game console, a mobile device, or other digital device, each of which is suitable for practicing embodiments of the present invention. The device 1100 includes a central processing unit (CPU) 1102 for running software applications and optionally an operating system. The CPU 1102 can be composed of one or more homogeneous or heterogeneous processing cores.

[0146] According to various embodiments, CPU 1102 is one or more general purpose microprocessors having one or more processing cores. Additional embodiments may be implemented using one or more CPUs having a microprocessor architecture particularly suited for highly parallel and computationally intensive applications such as media and interactive entertainment applications configured for graphics processing during execution of a game.

[0147] Memory 1104 stores applications and data for use by CPU 1102 and GPU 1116. Storage 1106 provides non-volatile storage and other computer-readable media for applications and data and may include fixed disk drives, removable disk drives, flash memory devices and CD-ROM, DVD-ROM, Blu-ray discs, HD-DVD or other optical storage devices, as well as signal transmission and storage media. User input device 1108 communicates user input from one or more users to device 1100, examples of which may include keyboards, mice, joysticks, touch pads, touch screens, static or video recorders / cameras and / or microphones. Network interface 1109 allows device 1100 to communicate with other computer systems via an electronic communication network, and may include wired or wireless communications on local area networks and wide area networks such as the Internet. Audio processor 1112 is adapted to generate analog or digital audio output from instructions and / or data provided by CPU 1102, memory 1104 and / or storage 1106. Components of device 1100 , including CPU 1102 , graphics subsystem including GPU 1116 , memory 1104 , data storage device 1106 , user input device 1108 , network interface 1109 , and audio processor 1112 , are connected via one or more data buses 1122 .

[0148] The graphics subsystem 1114 is also connected to the data bus 1122 and the components of the device 1100. The graphics subsystem 1114 includes at least one graphics processing unit (GPU) 1116 and a graphics memory 1118. The graphics memory 1118 includes a display memory (e.g., a frame buffer) for storing pixel data for each pixel of an output image. The graphics memory 1118 may be integrated in the same device as the GPU 1116, connected to the GPU 1116 as a separate device, and / or implemented in the memory 1104. The pixel data may be provided directly from the CPU 1102 to the graphics memory 1118. Alternatively, the CPU 1102 provides the GPU 1116 with data and / or instructions defining the desired output image, and the GPU 1116 generates pixel data for one or more output images according to the data and / or instructions. The data and / or instructions defining the desired output image may be stored in the memory 1104 and / or the graphics memory 1118. In an embodiment, GPU 1116 includes 3D rendering capabilities for generating pixel data for an output image according to instructions and data that define the geometry, lighting, shading, texturing, motion, and / or camera parameters of a scene. GPU 1116 may also include one or more programmable execution units capable of executing shader programs.

[0149] Graphics subsystem 1114 periodically outputs pixel data of an image from graphics memory 1118 for display on display device 1110 or for projection by a projection system (not shown). Display device 1110 may be any device capable of displaying visual information in response to a signal from device 1100, including CRT, LCD, plasma, and OLED displays. Device 1100 may provide, for example, an analog or digital signal to display device 1110.

[0150] Other embodiments for optimizing the graphics subsystem 1114 may include multi-GPU rendering of geometry for applications with region testing while rendering objects for images. The graphics subsystem 1114 may be configured as one or more processing devices.

[0151] For example, in one embodiment, graphics subsystem 1114 may be configured to perform multi-GPU rendering of geometry for an application with region testing at render time, where multiple graphics subsystems may implement graphics and / or rendering pipelines for a single application. That is, graphics subsystem 1114 includes multiple GPUs for rendering each of one or more images in an image or sequence of images when executing an application.

[0152] In other embodiments, the graphics subsystem 1114 includes multiple GPU devices, which are combined to perform graphics processing for a single application executed on a corresponding CPU. For example, multiple GPUs can perform multi-GPU rendering of geometric graphics for an application by region testing when rendering an image object. In other examples, multiple GPUs can perform frame rendering in an alternating form, wherein in a sequential frame cycle, GPU 1 renders the first frame, and GPU 2 renders the second frame, and so on, until the last GPU is reached, then the initial GPU renders the next video frame (for example, if there are only two GPUs, GPU 1 renders the third frame). That is, the GPU rotates when rendering a frame. Rendering operations can overlap, wherein GPU 2 can start rendering the second frame before GPU 1 finishes rendering the first frame. In another embodiment, different shader operations can be assigned to multiple GPU devices in rendering and / or graphics pipelines. The main GPU is performing main rendering and synthesis. For example, in a group including three GPUs, master GPU 1 can perform the main rendering (e.g., a first shader operation) and synthesize the output from slave GPU 2 and slave GPU 3, where slave GPU 2 can perform a second shader (e.g., a fluid effect, such as a river) operation, and slave GPU 3 can perform a third shader (e.g., a particle smoke) operation, where master GPU 1 synthesizes the results from each of GPU 1, GPU 2, and GPU 3. In this way, different GPUs can be assigned to perform different shader operations (e.g., waving a flag, wind, smoke generation, fire, etc.) to render a video frame. In yet another embodiment, each of the three GPUs can be assigned to a different object and / or portion of a scene corresponding to a video frame. In the above embodiments and implementations, these operations can be performed in the same frame cycle (simultaneous parallelism) or in different frame cycles (sequential parallelism).

[0153] Thus, the present disclosure describes methods and systems configured for multi-GPU rendering of geometry for an application through region testing while rendering objects for each of one or more images in an image or sequence of images while executing the application.

[0154] It should be understood that the various embodiments defined herein can be combined or assembled into specific embodiments using the various features disclosed herein. Therefore, the examples provided are only some possible examples and are not limited to the various embodiments that are possible by combining various elements to define more embodiments. In some examples, some embodiments may include fewer elements without departing from the spirit of the disclosed or equivalent embodiments.

[0155] Embodiments of the present disclosure may be practiced with various computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked over wired or wireless networks.

[0156] In view of the above embodiments, it should be understood that the embodiments of the present disclosure may employ various computer-implemented operations involving data stored in a computer system. These operations are those operations that require physical manipulation of physical quantities. Any operation described herein that forms a part of the embodiments of the present disclosure is a useful machine operation. The embodiments of the present disclosure also relate to devices or equipment for performing these operations. The equipment may be specially constructed for the desired purpose, or the equipment may be a general-purpose computer selectively activated or configured by a computer program stored in a computer. Specifically, various general-purpose machines may be used together with computer programs written according to the teachings of this article, or it may be more convenient to construct more specialized equipment to perform the desired operations.

[0157] The present disclosure may also be implemented as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of computer-readable media include hard drives, network attached storage devices (NAS), read-only memories, random access memories, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable medium may include a computer-readable tangible medium distributed on a network-coupled computer system so that the computer-readable code is stored and executed in a distributed manner.

[0158] Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or the operations may be coordinated so that they occur at slightly different times, or may be distributed in a system that allows the processing operations to occur at various intervals associated with the processing, so long as the processing covering the operations is performed in the desired manner.

[0159] Although the foregoing disclosure has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Therefore, the embodiments of the present invention are to be considered as illustrative rather than restrictive, and the embodiments of the present disclosure are not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.

Claims

1. A method for graphics processing, comprising: Use multiple graphics processing units (GPUs) to render graphics for applications; dividing the responsibility for rendering the geometry of the graphics among the plurality of GPUs based on a plurality of screen regions, each of the plurality of GPUs having a corresponding division of responsibility known to the plurality of GPUs, wherein in the divided responsibility for rendering the geometry, each of the GPUs performs a Z-buffer test on one or more corresponding geometries overlapping a corresponding screen region for which each GPU has the corresponding division of responsibility; determining, at a GPU, one or more geometries overlapping a first screen region that the GPU has a division of responsibility for; determining a Z value for one geometry during a pre-pass phase of rendering the geometry for an image at the GPU, wherein the geometry overlaps the first screen region for which the GPU has the division of responsibility, wherein the GPU performs the Z-buffer test only on the one or more geometries in the image that overlap the first screen region; comparing the Z value to a Z-buffer value of the geometry in the Z-buffer test during the pre-pass phase; as well as generating information including the results of said comparing said Z value to said Z buffer value during said pre-pass phase for use by said GPU in rendering said geometry during a full rendering phase of rendering, Wherein before each of the plurality of GPUs performs a full rendering phase of the rendering, each of the plurality of GPUs completes a corresponding pre-pass phase of rendering of the one or more corresponding geometries in the image.

2. The method according to claim 1, further comprising: The GPU renders a plurality of pixels for the geometry during the full rendering phase of rendering when the geometry is not occluded based on the information.

3. The method of claim 1 , wherein comparing the Z value to the Z-buffer value comprises: determining that the Z value is less than or equal to the Z-buffer value, indicating that the geometry is not occluded, The information indicates that the geometric figure is not occluded.

4. The method according to claim 1, further comprising: rasterizing one or more primitives of the geometric figure into a plurality of pixels during the pre-pass phase; performing a depth test by comparing a plurality of Z values ​​of the plurality of pixels to the Z-buffer value, wherein a result of the depth test is included in the information; as well as Pixels of the plurality of pixels that survived the depth test are rendered based on the information.

5. The method according to claim 4, further comprising: When none of the plurality of pixels survives the depth test, rendering of the geometry is skipped.

6. The method according to claim 1, further comprising: rasterizing one or more primitives of the geometric figure into a plurality of pixels; checking alpha values ​​of the texture of the plurality of pixels; as well as Pixels having alpha values ​​below a threshold value among the plurality of pixels are killed, wherein a result of checking the alpha value is included in the information.

7. The method according to claim 6, further comprising: When none of the plurality of pixels has a corresponding alpha value equal to or above the threshold, rendering of the geometry is skipped.

8. The method of claim 1, wherein one or more of the plurality of GPUs are parts of a larger GPU configured as a plurality of virtual GPUs.

9. The method according to claim 1, further comprising: A Z-buffer is accessed to obtain the Z-buffer value for the geometry, wherein the Z-buffer includes one or more Z-buffer values.

10. A non-transitory computer-readable medium for performing a method, the computer-readable medium comprising: program instructions for rendering graphics for an application using a plurality of graphics processing units (GPUs); program instructions for dividing responsibility for rendering geometry of the graphics among the plurality of GPUs based on a plurality of screen regions, each of the plurality of GPUs having a corresponding division of responsibility known to the plurality of GPUs, wherein in the divided responsibility for rendering the geometry, each of the GPUs performs a Z-buffer test on one or more corresponding geometries that overlap with a corresponding screen region for which each GPU has the corresponding division of responsibility; program instructions for determining, at a GPU, one or more geometric figures overlapping a first screen region for which the GPU has a division of responsibility; program instructions for determining a Z value for one geometric figure during a pre-pass phase of rendering the geometric figure for an image at the GPU, wherein the geometric figure overlaps the first screen region for which the GPU has the division of responsibility, wherein the GPU performs the Z-buffer test only on the one or more geometric figures in the image that overlap the first screen region; program instructions for comparing the Z value to a Z-buffer value of the geometry in the Z-buffer test during the pre-pass phase; and program instructions for generating information including the results of said comparing said Z value to said Z-buffer value during said pre-pass phase for use by said GPU in rendering said geometry during a full render phase of rendering, Wherein before each of the plurality of GPUs performs a full rendering phase of the rendering, each of the plurality of GPUs completes a corresponding pre-pass phase of rendering of the one or more corresponding geometries in the image.

11. The non-transitory computer readable medium of claim 10, further comprising: Program instructions for rendering, by the GPU during the full rendering phase of rendering, a plurality of pixels for the geometry when the geometry is not occluded based on the information.

12. The non-transitory computer readable medium of claim 10, wherein the program instructions for comparing the Z value to the Z-buffer value comprise: program instructions for determining that the Z value is less than or equal to the Z-buffer value, indicating that the geometry is not occluded, The information indicates that the geometric figure is not occluded.

13. The non-transitory computer readable medium of claim 10, further comprising: program instructions for rasterizing one or more primitives of said geometric figure into a plurality of pixels during said pre-pass phase; program instructions for performing a depth test by comparing a plurality of Z values ​​of the plurality of pixels to the Z-buffer value, wherein a result of the depth test is included in the information; and Program instructions are provided for rendering pixels of the plurality of pixels that survived the depth test based on the information.

14. The non-transitory computer readable medium of claim 13, further comprising: Program instructions for skipping rendering of the geometry when none of the plurality of pixels survives the depth test.

15. The non-transitory computer readable medium of claim 10, further comprising: program instructions for rasterizing one or more primitives of the geometric figure into a plurality of pixels; program instructions for examining an alpha value of a texture of the plurality of pixels; and Program instructions for killing pixels of the plurality of pixels having alpha values ​​below a threshold, wherein a result of checking the alpha values ​​is included in the information.

16. The non-transitory computer readable medium of claim 15, further comprising: Program instructions for skipping rendering of the geometry when none of the plurality of pixels has a corresponding alpha value equal to or above the threshold. 17 . The non-transitory computer-readable medium of claim 10 , wherein in the method, one or more of the plurality of GPUs are part of a larger GPU configured as a plurality of virtual GPUs.

18. The non-transitory computer readable medium of claim 10, further comprising: Program instructions for accessing a Z-buffer to obtain the Z-buffer value for the geometry, wherein the Z-buffer includes one or more Z-buffer values.

19. A computer system comprising: processor; a memory coupled to the processor and having stored therein instructions, which, if executed by the computer system, cause the computer system to perform a method for implementing a graphics pipeline, the method comprising: Use multiple graphics processing units (GPUs) to render graphics for applications; dividing the responsibility for rendering the geometry of the graphics among the plurality of GPUs based on a plurality of screen regions, each of the plurality of GPUs having a corresponding division of responsibility known to the plurality of GPUs, wherein in the divided responsibility for rendering the geometry, each of the GPUs performs a Z-buffer test on one or more corresponding geometries overlapping a corresponding screen region for which each GPU has the corresponding division of responsibility; determining, at a GPU, one or more geometries overlapping a first screen region that the GPU has a division of responsibility for; determining a Z value for one geometry during a pre-pass phase of rendering the geometry for an image at the GPU, wherein the geometry overlaps the first screen region for which the GPU has the division of responsibility, wherein the GPU performs the Z-buffer test only on the one or more geometries in the image that overlap the first screen region; comparing the Z value to a Z-buffer value of the geometry in the Z-buffer test during the pre-pass phase; and generating information including the results of said comparing said Z value to said Z buffer value during said pre-pass phase for use by said GPU in rendering said geometry during a full rendering phase of rendering, Wherein before each of the plurality of GPUs performs a full rendering phase of the rendering, each of the plurality of GPUs completes a corresponding pre-pass phase of rendering of the one or more corresponding geometries in the image.

20. The computer system of claim 19, the method further comprising: When the geometry is not occluded based on the information, a plurality of pixels are rendered by the GPU for the geometry during the full rendering phase of rendering.

21. The computer system of claim 19, wherein in the method, comparing the Z value to the Z-buffer value comprises: determining that the Z value is less than or equal to the Z-buffer value, indicating that the geometry is not occluded, The information indicates that the geometric figure is not occluded.

22. The computer system of claim 19, the method further comprising: rasterizing one or more primitives of the geometric figure into a plurality of pixels during the pre-pass phase; performing a depth test by comparing a plurality of Z values ​​of the plurality of pixels to the Z-buffer value, wherein a result of the depth test is included in the information; as well as Pixels of the plurality of pixels that survived the depth test are rendered based on the information.

23. The computer system of claim 22, the method further comprising: When none of the plurality of pixels survives the depth test, rendering of the geometry is skipped.

24. The computer system of claim 19, the method further comprising: rasterizing one or more primitives of the geometric figure into a plurality of pixels; checking alpha values ​​of the texture of the plurality of pixels; as well as Pixels having alpha values ​​below a threshold value among the plurality of pixels are killed, wherein a result of checking the alpha value is included in the information.

25. The computer system of claim 24, the method further comprising: When none of the plurality of pixels has a corresponding alpha value equal to or above the threshold, rendering of the geometry is skipped.

26. The computer system of claim 19, the method further comprising: A Z-buffer is accessed to obtain the Z-buffer value for the geometry, wherein the Z-buffer includes one or more Z-buffer values.

Images