System and method for efficient multi-GPU execution of kernels via region-based dependencies

Abstract

The present application relates to a system and method for efficient multi-GPU execution of kernels through region-based dependencies. The method includes: dividing a first kernel into multiple parts during execution of an application; assigning a first part of the first kernel to a first graphics processing unit (GPU) for processing; assigning a second part of the first kernel to a second GPU for processing; executing by the first GPU to complete the first part of the first kernel; storing first data from executing to complete the first part of the first kernel on a first storage resource; generating dependency data after storing the first data on the first storage resource; and executing a second kernel after determining that the dependency data has been generated, wherein execution of the second kernel begins before the second part of the first kernel has been processed by the second GPU.

Description

[0001] This application is a divisional application of the invention patent application with application number 202180038200.4, application date April 28, 2021, and invention name "System and method for efficient multi-GPU execution of kernels through region-based dependencies". Technical Field

[0002] The present disclosure relates to graphics processing, and more particularly to kernel computing on a graphics processing unit (GPU). Background Art

[0003] In recent years, there has been a push for online services that allow online gaming or cloud gaming to be performed in a streaming format between a cloud gaming server and clients connected via a network. The streaming format has become increasingly popular due to the availability of game titles on demand, the ability to perform more complex games, the ability to network between players for multiplayer gaming, the sharing of assets between players, the sharing of instant experiences between players and / or spectators, the ability to allow friends to watch friends play video games, and for friends to join a friend's ongoing game play, etc.

[0004] A cloud gaming server can be configured to provide resources to one or more clients and / or applications. That is, a cloud gaming server can be configured with resources that have high throughput capabilities. For example, the performance that can be achieved by a single graphics processing unit (GPU) is limited, for example, derived from restrictions on how large a GPU can be. In order to render more complex scenes or use more complex algorithms (e.g., materials, lighting, etc.) when generating scenes, it may be necessary to use multiple GPUs to render a single image.

[0005] However, equal use of these GPUs is difficult to achieve. For example, it is difficult to evenly distribute workloads between GPUs, which can cause some GPUs to complete their workloads faster than other GPUs in a particular processing cycle. The GPUs that execute faster will wait (e.g., idle) for other GPUs to complete processing their respective workloads and copy their results to other GPUs, because the data generated by one GPU may be used by another GPU in the next processing cycle. In addition, GPUs connected via a lower speed bus have obvious disadvantages compared to GPUs connected to shared memory via a high-speed bus. As images or buffers become larger, the size of the copy increases and becomes a bottleneck. Due to this inefficiency (e.g., waiting for copies from other GPUs, idle time during synchronization, increased waiting time, etc.), it is difficult to process four times the data using traditional techniques, even if there may be four times the number of GPUs available. For example, in the past, even if there were multiple GPUs processing images for applications, the corresponding increase in screen pixel counts and geometric graphics density could not be supported (e.g., four GPUs could not write four times the pixels and / or process four times the vertices or primitives of the image).

[0006] It is against this background that embodiments of the present disclosure emerge. Summary of the invention

[0007] Embodiments of the present disclosure relate to using multiple GPUs to collaboratively process data or images.

[0008] Embodiments of the present disclosure disclose a method for graphics processing. The method includes: executing multiple kernels using multiple graphics processing units (GPUs), wherein the responsibility for executing the corresponding kernels is divided into one or more parts, each of which is assigned to the corresponding GPU. The method includes: generating multiple dependency data at the first kernel when each of the first multiple parts of the first kernel completes processing. The method includes: checking the dependency data from one or more parts of the first kernel before executing a part of the second kernel. The method includes: delaying the execution of the part of the second kernel as long as the corresponding dependency data of the first kernel is not satisfied.

[0009] Other embodiments of the present disclosure disclose a non-transitory computer-readable medium for performing a method. The computer-readable medium includes: program instructions for executing multiple kernels using multiple graphics processing units (GPUs), wherein the responsibility for executing the corresponding kernels is divided into one or more parts, each of which is assigned to the corresponding GPU. The computer-readable medium includes: program instructions for generating multiple dependency data at the first kernel when each of the first multiple parts of the first kernel completes processing. The computer-readable medium includes: program instructions for checking dependency data from one or more parts of the first kernel before executing a part of the second kernel. The computer-readable medium includes: program instructions for delaying execution of the part of the second kernel as long as the corresponding dependency data of the first kernel is not met.

[0010] Other embodiments of the present disclosure disclose a computer system comprising a processor and a memory, the memory being coupled to the processor and having instructions stored therein, which, if executed by the computer system, cause the computer system to perform a method. The method comprises: executing multiple kernels using multiple graphics processing units (GPUs), wherein the responsibility for executing the corresponding kernels is divided into one or more parts, each of which is assigned to a corresponding GPU. The method comprises: generating multiple dependency data at the first kernel when each of the first multiple parts of the first kernel completes processing. The method comprises: checking the dependency data from one or more parts of the first kernel before executing a part of the second kernel. The method comprises: delaying the execution of the part of the second kernel as long as the corresponding dependency data of the first kernel is not satisfied.

[0011] An embodiment of the present disclosure discloses a method, comprising: dividing a first kernel into multiple parts during execution of an application; assigning a first part of the first kernel to a first graphics processing unit (GPU) for processing; assigning a second part of the first kernel to a second GPU for processing; executing by the first GPU to complete the first part of the first kernel; storing first data from executing to complete the first part of the first kernel on a first storage resource; generating dependency data after storing the first data on the first storage resource; and executing a second kernel after determining that the dependency data has been generated, wherein execution of the second kernel begins before the second part of the first kernel has been completed by the second GPU.

[0012] Other embodiments of the present disclosure disclose a non-transitory computer-readable medium storing a computer program for execution by a processor to perform a method, the non-transitory computer-readable medium comprising: program instructions for dividing a first kernel into a plurality of parts during execution of an application; program instructions for assigning a first part of the first kernel to a first graphics processing unit (GPU) for processing; program instructions for assigning a second part of the first kernel to a second GPU for processing; program instructions for execution by the first GPU to complete the first part of the first kernel; program instructions for storing first data from execution to complete the first part of the first kernel on a first storage resource; program instructions for generating dependent data after storing the first data on the first storage resource; and program instructions for executing a second kernel after determining that the dependent data has been generated, wherein execution of the second kernel begins before the second part of the first kernel has been completed by the second GPU.

[0013] Still other embodiments of the present disclosure disclose a computer system comprising: a processor; a memory coupled to the processor and having instructions stored therein, which, if executed by the computer system, cause the computer system to perform a method, the method comprising: dividing a first kernel into multiple parts during execution of an application; assigning a first part of the first kernel to a first graphics processing unit (GPU) for processing; assigning a second part of the first kernel to a second GPU for processing; executing by the first GPU to complete the first part of the first kernel; storing first data from executing to complete the first part of the first kernel on a first storage resource; generating dependency data after storing the first data on the first storage resource; and executing a second kernel after determining that the dependency data has been generated, wherein execution of the second kernel begins before the second part of the first kernel has been completed by the second GPU.

[0014] Other aspects of the disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present disclosure may be best understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a schematic diagram of a system for providing games between one or more cloud gaming servers over a network, wherein the one or more cloud gaming servers are configured to enable multiple GPUs to collaborate to render a single image.

[0017] Figure 2 is a schematic diagram of a multi-GPU architecture in which multiple GPUs collaborate to process data or images according to one embodiment of the present disclosure.

[0018] Figures 3 to 5 A timeline illustrating kernel dependencies and data copying after the kernel completes processing and GPU inefficiencies when data dependencies are not used is illustrated.

[0019] Figure 6 Illustrated is a kernel call in N dimensions divided into multiple parts that are distributed to multiple GPUs for execution according to one embodiment of the present disclosure.

[0020] Fig. 7A An image resource segmented into regions according to one embodiment of the present disclosure is illustrated.

[0021] Figure 7B FIG. 1 illustrates a cache resource partitioned into regions according to one embodiment of the present disclosure.

[0022] Figure 8 is a flow chart illustrating steps in a method for graphics processing including processing kernels using multiple GPUs according to one embodiment of the present disclosure.

[0023] Fig. 9 A fixed or static allocation scheme for evenly distributing portions of a kernel across multiple GPUs according to one embodiment of the present disclosure is illustrated.

[0024] Fig.10 is a flow chart illustrating steps in a method for graphics processing according to one embodiment of the present disclosure, the method including processing kernels using multiple GPUs, wherein per-kernel portion dependency tracking is performed to allow early processing of dependent portions of subsequent kernels.

[0025] Fig.11A Dependency data based on a portion of a kernel according to one embodiment of the present disclosure is illustrated.

[0026] Fig. 11B Dependency data of kernel-based regions and portions is illustrated according to one embodiment of the present disclosure.

[0027] FIG. 12A to FIG. 12D Illustrated is the checking of dependency data by a portion of a kernel as a function of an index range of the portion according to an embodiment of the present disclosure.

[0028] FIG. 13A to FIG. 13C Various strategies for addressing when a function of the index range of a portion of a kernel is outside of a defined index space are illustrated in accordance with embodiments of the present disclosure.

[0029] Fig.14A A dynamic allocation scheme is illustrated according to one embodiment of the present disclosure, wherein different GPUs follow different space filling curves when allocating portions of a kernel to GPUs for execution.

[0030] Fig. 14B The diagram shows a method according to an embodiment of the present disclosure as defined in Fig.14A The kernel's parts are allocated in the order outlined in the GPU space filling curve in .

[0031] Fig.15 is a flow chart illustrating steps in a method for graphics processing including multi-GPU rendering of images for an application, wherein dependent data may be performed based on kernel processing and / or draw calls, according to one embodiment of the present disclosure.

[0032] Fig.16 Components of an exemplary apparatus that may be used to perform various aspects of the various embodiments of the present disclosure are illustrated. DETAILED DESCRIPTION

[0033] Although the following detailed description contains many specific details for illustrative purposes, those skilled in the art will appreciate that many variations and modifications of the following details are within the scope of the present disclosure. Therefore, the various aspects of the present disclosure described below are set forth without any loss of generality of the appended claims of this specification and without imposing any limitations on these claims.

[0034] In general, embodiments of the present disclosure disclose methods and systems for executing kernels in which multiple graphics processing units (GPUs) collaborate to process images or data. The kernel being processed is split into multiple parts. When processing an image or buffer, a GPU is assigned to multiple parts of the kernel and tracks dependency data between the parts, allowing fine-grained, region-based dependency data to be used between kernels to balance workloads across GPUs.

[0035] With the above general understanding of the various embodiments in mind, exemplary details of the embodiments will now be described with reference to the various figures.

[0036] Throughout this specification, references to "applications" or "games" or "video games" or "gaming applications" are intended to refer to any type of interactive application that is directed by executing input commands. For purposes of illustration only, interactive applications include applications for gaming, word processing, video processing, video game processing, etc. Furthermore, the terms introduced above are interchangeable.

[0037] Throughout the specification, various embodiments of the present disclosure are described for multi-GPU processing of kernels for applications using an exemplary architecture with four GPUs. However, it should be understood that any number of GPUs (e.g., two or more GPUs) can collaborate in generating images and / or data for an application.

[0038] Figure 1 1 is a schematic diagram of a system for executing a kernel when processing an application according to an embodiment of the present disclosure, wherein multiple graphics processing units (GPUs) collaborate to process images or data. In one embodiment, the system is configured to provide a game between one or more cloud gaming servers over a network. Cloud gaming includes executing a video game at a server to generate a game rendered video frame, which is then sent to a client for display.

[0039] Although Figure 1The diagram illustrates implementing multi-GPU execution of kernels between one or more cloud gaming servers of a cloud gaming system, but other embodiments of the present disclosure provide for executing kernels when processing applications, where multiple graphics processing units (GPUs) collaborate to process images or data within a standalone system (such as a personal computer or game console that includes a high-end graphics card with multiple GPUs).

[0040] It should also be understood that in various embodiments (e.g., in a cloud gaming environment or in a standalone system), the multi-GPU execution of the kernel can be performed using a physical GPU or a virtual GPU or a combination of the two. For example, a virtual machine (e.g., an instance) can be created using a hypervisor of host hardware (e.g., located in a data center) that utilizes one or more components of a hardware layer (such as multiple CPUs, memory modules, GPUs, network interfaces, communication components, etc.). These physical resources can be arranged in racks, such as CPU racks, GPU racks, memory racks, etc., where the physical resources in the rack can be accessed using a top-of-rack switch that facilitates the structure of components for assembling and accessing instances (e.g., when building virtualized components of instances). Typically, a hypervisor can present multiple client operating systems configured with multiple instances of virtual resources. That is, each operating system can be configured with a corresponding set of virtualized resources supported by one or more hardware resources (e.g., located in a corresponding data center). For example, each operating system can be supported by a virtual CPU, multiple virtual GPUs, virtual memory, virtualized communication components, etc. In addition, the configuration of an instance can be transferred from one data center to another to reduce waiting time. The GPU utilization defined for a user or game can be used when saving a game session for the user. The GPU utilization can include any number of configurations described herein to optimize the fast rendering of video frames for a game session. In one embodiment, the GPU utilization defined for a game or user can be transferred between data centers as a configurable setting. The ability to transfer GPU utilization settings can effectively migrate game play from data center to data center in the event that a user connects to play the game in different geographic locations.

[0041] According to one embodiment of the present disclosure, the system 100 provides a game via a cloud gaming network 190, wherein a client device 110 (e.g., a thin client) away from the corresponding user playing the game executes the game. The system 100 can provide game control to one or more users playing one or more games through the cloud gaming network 190 via the network 150 in a single player or multi-player mode. In some embodiments, the cloud gaming network 190 may include a plurality of virtual machines (VMs) running on a host's hypervisor, wherein one or more virtual machines are configured to execute a game processor module using hardware resources available to the host's hypervisor. The network 150 may include one or more communication technologies. In some embodiments, the network 150 may include a fifth generation (5G) network technology with an advanced wireless communication system.

[0042] In some embodiments, wireless technology can be used to facilitate communication. Such technology can include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. The 5G network is a digital cellular network in which the service area covered by the provider is divided into small geographical areas called cells. The analog signals representing sound and images are digitized in the phone, converted by an analog-to-digital converter and transmitted as a bit stream. Through the frequency channels allocated by the transceiver from the frequency pool reused in other cells, all 5G wireless devices in the cell communicate with the local antenna array and low-power automatic transceiver (transmitter and receiver) in the cell through radio waves. The local antenna is connected to the telephone network and the Internet through a high-bandwidth optical fiber or wireless backhaul connection. As in other cell networks, a mobile device that crosses from one cell to another will automatically go to the new cell. It should be understood that the 5G network is only an example type of communication network, and the embodiments of the present disclosure can utilize an earlier generation of wireless or wired communication, as well as a newer generation of wired or wireless technology after 5G.

[0043] As shown, the cloud gaming network 190 includes a game server 160 that provides access to multiple video games. The game server 160 can be any type of server computing device available in the cloud, and can be configured as one or more virtual machines executed on one or more hosts. For example, the game server 160 can manage a virtual machine that supports a game processor that instantiates a game instance for a user. In this way, multiple game processors of the game server 160 associated with multiple virtual machines are configured to execute multiple instances of one or more games associated with the game progress of multiple users. In this way, the back-end server supports the streaming of media (e.g., video, audio, etc.) of the game progress of multiple game applications provided to multiple corresponding users. In other words, the game server 160 is configured to stream data (e.g., rendered images and / or frames of the corresponding game progress) back to the corresponding client device 110 through the network 150. In this way, a computationally complex game application can be executed at the back-end server in response to a controller input received and forwarded by the client device 110. Each server is capable of rendering images and / or frames, which are then encoded (e.g., compressed) and streamed to the corresponding client device for display.

[0044] For example, multiple users can access the cloud gaming network 190 via the communication network 150 using a corresponding client device 110 configured to receive streaming media. In one embodiment, the client device 110 may be configured as a thin client that provides an interface with a back-end server (e.g., cloud gaming network 190) configured to provide computing functionality (e.g., including a game name processing engine 111). In another embodiment, the client device 110 may be configured with a game name processing engine and game logic for at least some local processing of the video game, and may also be utilized to receive streaming content generated by the video game executed at the back-end server, or other content provided by the back-end server support. For local processing, the game name processing engine includes basic processor-based functions for executing video games and services associated with the video game. In this case, the game logic can be stored on the local client device 110 and used to execute the video game.

[0045] Each client device 110 may request access to a different game from the cloud gaming network. For example, the cloud gaming network 190 may execute one or more game logics that are built on the game name processing engine 111, such as executed using the CPU resources 163 and GPU resources 265 of the game server 160. For example, game logic 115a that cooperates with the game name processing engine 111 may be executed on the game server 160 for one client, game logic 115b that cooperates with the game name processing engine 111 may be executed on the game server 160 for the second client, and game logic 115n that cooperates with the game name processing engine 111 may be executed on the game server 160 for the Nth client.

[0046] In particular, the client device 110 of the corresponding user (not shown) is configured to request access to the game via a communication network 150 such as the Internet, and to render a display image generated by a video game executed by a game server 160, wherein the encoded image is transmitted to the client device 110 for display in association with the corresponding user. For example, a user can interact with an instance of a video game executed on a game processor of the game server 160 through the client device 110. More specifically, the instance of the video game is executed by a game title processing engine 111. The corresponding game logic (e.g., executable code) 115 that implements the video game is stored and accessible through a data storage area (not shown) and used to execute the video game. The game title processing engine 111 is capable of supporting multiple video games using multiple game logics (e.g., game applications), each of which can be selected by a user.

[0047] For example, the client device 110 is configured to interact with a game title processing engine 111 associated with the game progress of the corresponding user, such as through input commands for driving the game progress. In particular, the client device 110 can receive input from various types of input devices, such as a game controller, a tablet computer, a keyboard, a gesture captured by a camera, a mouse, a touchpad, etc. The client device 110 can be any type of computing device having at least a memory and a processor module that can be connected to the game server 160 via the network 150. The back-end game title processing engine 111 is configured to generate a rendered image, which is transmitted over the network 150 to be displayed at a corresponding display associated with the client device 110. For example, through a cloud-based service, the game rendering image can be transmitted by an instance of the corresponding game (e.g., game logic) executed on the game execution engine 111 of the game server 160. That is, the client device 110 is configured to receive an encoded image (e.g., encoded from a game rendering image generated by executing a video game), and to display the rendered image on the display 11. In one embodiment, the display 11 includes an HMD (e.g., displaying VR content). In some embodiments, the rendered image may be downloaded wirelessly or wired, directly from a cloud-based service, or via a client device 110 (e.g., Remote Play) to your smartphone or tablet.

[0048] In one embodiment, the game server 160 and / or the game title processing engine 111 include basic processor-based functions for executing games and services associated with game applications. For example, the game server 160 includes a central processing unit (CPU) resource 163 and a graphics processing unit (GPU) resource 265, which are configured to perform processor-based functions, including 2D or 3D rendering, physical simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformation, artificial intelligence, etc. In addition, the CPU and GPU group can implement services for game applications, including memory management, multi-threading management, quality of service (QoS), bandwidth testing, social networks, social friend management, social network communications with friends, communication channels, texting, instant messaging, chat support, etc. In one embodiment, one or more applications share specific GPU resources. In one embodiment, multiple GPU devices can be combined to perform graphics processing for a single application executed on a corresponding CPU.

[0049] In one embodiment, the cloud game network 190 is a distributed game server system and / or architecture. In particular, the distributed game engine that executes the game logic is configured as a corresponding instance of the corresponding game. Typically, the distributed game engine uses each function of the game engine and distributes these functions for execution by multiple processing entities. Individual functions can be further distributed on one or more processing entities. The processing entity can be configured as different configurations, including physical hardware, and / or as a virtual component or virtual machine, and / or as a virtual container, wherein the container is different from the virtual machine because it virtualizes the instance of the game application running on the virtualized operating system. The processing entity can utilize and / or rely on the server and its underlying hardware on one or more servers (computing nodes) of the cloud game network 190, wherein the server can be located on one or more racks. The coordination, allocation and management of performing these functions on various processing entities are performed by the distributed synchronization layer. In this way, the execution of these functions is controlled by the distributed synchronization layer to generate media (e.g., video frames, audio, etc.) for the game application in response to the player's controller input. The distributed synchronization layer enables these functions to be efficiently performed (eg, via load balancing) across distributed processing entities so that key game engine components / functions are distributed and reassembled for more efficient processing.

[0050] Figure 2 2 is a schematic diagram of an exemplary multi-GPU architecture 200 in which multiple GPUs collaborate to generate data for corresponding applications and / or render a single image for corresponding applications according to one embodiment of the present disclosure. It should be understood that in various embodiments of the present disclosure, many architectures are possible in which multiple GPUs collaborate to generate data and / or render images, although not explicitly described or shown. For example, multiple GPUs collaborating to execute kernels when processing images and / or data can be implemented between one or more cloud gaming servers in a cloud gaming system, or can be implemented within a standalone system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), etc.

[0051] The multi-GPU architecture 200 includes a CPU 163 and multiple GPUs configured to perform multi-GPU rendering for a single image of an application and / or each image in an image sequence of an application. In particular, the CPU 163 and GPU resources 265 are configured to perform processor-based functions including 2D or 3D rendering, physical simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadows, culling, transformations, artificial intelligence, etc., as described above.

[0052] For example, four GPUs are shown in GPU resources 265 of multi-GPU architecture 200, although any number of GPUs may be used when generating data or rendering images for an application. Each GPU is connected to a corresponding dedicated memory, such as random access memory (RAM), via a high-speed bus 220. In particular, GPU-A is connected to memory 210A (e.g., RAM) via bus 220, GPU-B is connected to memory 210B (e.g., RAM) via bus 220, GPU-C is connected to memory 210C (e.g., RAM) via bus 220, and GPU-D is connected to memory 210D (e.g., RAM) via bus 220.

[0053] In addition, each GPU is connected to each other via bus 240, which, depending on the architecture, may be approximately equal to or slower in speed than bus 220 used for communication between the corresponding GPU and its corresponding memory. For example, GPU-A is connected to each of GPU-B, GPU-C, and GPU-D via bus 240. Likewise, GPU-B is connected to each of GPU-A, GPU-C, and GPU-D via bus 240. In addition, GPU-C is connected to each of GPU-A, GPU-B, and GPU-D via bus 240. In addition, GPU-D is connected to each of GPU-A, GPU-B, and GPU-C via bus 240.

[0054] The CPU 163 is connected to each GPU via a lower speed bus 230 (e.g., bus 230 is slower than bus 220 used for communication between the corresponding GPU and its corresponding memory). Specifically, the CPU 163 is connected to each of GPU-A, GPU-B, GPU-C, and GPU-D.

[0055] In some embodiments, the four GPUs are discrete GPUs, each on their own silicon. In other embodiments, the four GPUs can share a single die in order to take advantage of the high-speed interconnects and other units on the die. In other embodiments, there is one physical GPU 250 that can be configured to function as a single more powerful GPU or as four less powerful "virtual" GPUs (GPU-A, GPU-B, GPU-C, and GPU-D). That is, GPU-A, GPU-B, GPU-C, and GPU-D each have enough power to operate a graphics pipeline (e.g., Figure 4 As shown), the chip as a whole can operate a graphics pipeline (such as Figure 4 ), and the configuration can flexibly switch between the two configurations (e.g. between rendering passes).

[0056] Figures 3 to 5Illustrated is a possible scenario where a GPU having dedicated memory connected via a slower bus is left idle or has increased latency when a copy operation is performed from one GPU when dependent data is not used.

[0057] In particular, Figure 3 A timeline 305 showing kernel dependencies and data copying after a kernel has completed processing is illustrated, where a kernel (also referred to as a "compute kernel") is a program executed on a GPU that can read data from or write data to an image resource or buffer resource. For example, kernel A generates and writes data, and then kernel B reads and uses the data for processing. Kernel A and kernel B can be divided into work groups or parts that are executed separately by different GPUs. For illustration, kernel A can be divided into multiple parts, where GPU A is assigned one or more parts 320A of kernel A for execution, and GPU B is assigned one or more parts 320B of kernel A for execution. In addition, kernel B can be divided into multiple parts, where GPU A is assigned one or more parts 340A of kernel B for execution, and GPU B is assigned one or more parts 340B of kernel B for execution. In this way, each of kernel A and kernel B can be executed by more than one or more GPUs.

[0058] As shown, one or more parts of kernel B may depend on data from one or more parts of kernel A. Therefore, it is necessary to perform copy operation 330. In particular, if high-speed access to the results of kernel A is desired, because kernel B depends on previously executed kernel A, the memory written by kernel A needs to be copied to all other GPUs (e.g., GPU B) before kernel B can begin to execute on one or more GPUs. In other words, it is necessary to wait for the work from kernel A to complete and be copied before running kernel B. For example, synchronization point 310 provides completion of all parts of kernel A before copy operation 330 begins. Since there may be an unbalanced workload between the parts assigned to GPU A and / or GPU B, before copy operation 330 begins, GPU A or GPU B (or some execution units of GPU A or GPU B) may be idle or underutilized while waiting for other parts to complete processing at synchronization point 310.

[0059] Furthermore, no portion of kernel B can begin until the copying of memory written by kernel A to all other GPUs is complete at synchronization point 311, because it is not known which dependencies were satisfied during the execution of kernel A, and it is unclear whether the dependencies required by kernel B have been satisfied. As shown, the portion of kernel A on either GPU A or GPU B can complete the copying, and GPU A or GPU B is idle until all portions of kernel A complete their respective copy operations 330 at synchronization point 311.

[0060] Figure 4 A timeline 405 showing kernel dependencies and hiding of data copy costs after the kernel has completed processing during execution of a separate kernel is illustrated, wherein a kernel is a program executed on a GPU that can read data from or write data to an image resource or buffer resource. For example, kernel A generates and writes data, and then kernel C reads and uses the data for processing. A separate kernel B may also be required. Kernel A, kernel B, and kernel C can each be divided into work groups or parts that are executed separately by different GPUs. For illustration, kernel A can be divided into multiple parts, wherein GPU A is assigned one or more parts 420A for execution, and GPU B is assigned one or more parts 420B for execution. In addition, kernel B can be divided into multiple parts, wherein GPU A is assigned one or more parts 440A for execution, and GPU B is assigned one or more parts 440B for execution. In addition, kernel C can be divided into multiple parts, wherein GPU A is assigned one or more parts 450A for execution, and GPU B is assigned one or more parts 450B for execution. As such, each of Kernel A, Kernel B, and Kernel C may be executed by more than one or more GPUs.

[0061] As shown, one or more portions of kernel C may depend on data from one or more portions of kernel A. That is, kernel A writes data and kernel C reads the data, such as in situations where high-bandwidth access to the results of kernel A is desired. Therefore, copy operation 430 needs to be performed. In particular, because kernel C depends on previously executed kernel A, memory written by kernel A needs to be copied to all other GPUs (e.g., GPU A and / or GPU B) before kernel C can begin execution on one or more GPUs. As previously described, copying memory written by kernel A may have a cost because some GPUs may be idle, waiting for all portions of kernel A to complete, and / or kernel C may not begin execution until copy operation 430 is complete.

[0062] There may be ways to hide the cost of copy operation 430 by performing copy operation 430 in conjunction with another separate operation. For example, copy operation 430 may be performed while kernel B is executing. As shown, synchronization point 410 provides for completion of all portions of kernel A before copy operation 430 begins. In addition, since there may be an unbalanced workload between portions assigned to GPU A and / or GPU B, GPU A or GPU B may be idle before copy operation 430 begins, waiting for the other portion to complete processing at synchronization point 410. During copy operation 430, portion 440A of kernel B executed on GPU A and portion 440B of kernel B executed on GPU B may be completed.

[0063] No portion of kernel C can begin until the copying of memory written by kernel A to all other GPUs is complete at synchronization point 411, because it is not known which dependencies were satisfied during the execution of kernel A, and it is not clear whether the dependencies required by kernel C have been satisfied. As shown, the portion of kernel A on either GPU A or GPU B can complete the copying and sit idle until all portions complete their respective copy operations 430 at synchronization point 411. However, even though the cost of the copying is hidden in executing kernel B, there are additional costs. In particular, the start of executing kernel C is increased by latency because kernel B must execute to completion at synchronization point 411 before kernel C begins execution.

[0064] Figure 5 A timeline 505 is illustrated showing the execution of a kernel that is evenly divided across multiple GPUs, where the workload between the GPUs may be different. As shown, the kernel is evenly divided between four GPUs, including GPU A, GPU B, GPU C, and GPU D. For example, the kernel may perform a lighting function when rendering an image, and the kernel may be evenly divided by the number of pixels. Each GPU receives a portion of the kernel for execution and copies the results to the other GPUs between synchronization points 510 and 520 along timeline 505, as previously described in Figure 3 and Figure 4As shown, GPU A includes kernel instance 540A, which executes a unique portion of the kernel, after which a copy operation 545A is performed to copy the result to all other GPUs. The kernel instance can include values ​​associated with the independent variables in the corresponding portion, where the portion is defined by an index range in the index space of the kernel. In addition, GPU B includes kernel instance 540B, which executes a unique portion of the same kernel, after which a copy operation 545B is performed to copy the result to all other GPUs. GPU C includes kernel instance 540C, which executes a unique portion of the same kernel, after which a copy operation 545C is performed to copy the result to all other GPUs. Finally, GPU D includes kernel instance 540D, which executes a unique portion of the same kernel, after which a copy operation 545D is performed to copy the result to all other GPUs.

[0065] Application developers can perform load balancing of multiple GPUs to try to execute an even workload on all GPUs, otherwise the application may suffer some performance loss due to an unbalanced workload. However, predicting a balanced workload across all GPUs is difficult, especially for non-homogeneous GPUs. For example, partitioning the workload in advance or by the application developer may be inefficient because some workloads on some GPUs may take longer due to different inputs. Following the example where a kernel may perform a lighting function and is evenly divided between GPUs by the number of pixels, it may be difficult to predict the workload executed for each pixel or pixel tile (e.g., a portion of an image buffer) because there may be different inputs for different tiles (e.g., different numbers of lights, different shading models, etc.). This may cause some parts of the kernel to take longer to compute. While waiting for some GPUs to complete executing portions of the kernel and complete the copy, other GPUs that have completed executing portions of the kernel and copying the results are idle, waiting for all copy operations to complete. For example, GPU A, GPU B, and GPU D are all idle, waiting for GPU C to complete its copy operation, with GPU B being idle the longest between synchronization points 510 and 520.

[0066] like Figure 5As shown, due to these inefficiencies (e.g., time waiting for copies from all GPUs, idle time during synchronization, and increased waiting time), GPUs that are connected to dedicated memory via a lower speed bus and each connected to a dedicated memory may be at a significant disadvantage compared to GPUs that are connected to a shared memory via a high speed bus. As image resources or buffer resources become larger, the size and duration of the copies may increase, resulting in lower efficiency and may become a further bottleneck. Due to these inefficiencies and without the use of data dependencies of embodiments of the present disclosure, it becomes difficult to process N times the data, even though there may be N times the number of GPUs available.

[0067] In the embodiments of the present disclosure, a GPU can be implemented to perform a compute shader function or a graphics shader (e.g., a pixel or vertex shader) function. For example, in addition to kernel calls that can perform graphics or non-graphics related processing, a GPU may also be responsible for rendering an object (e.g., writing color or other data) to the pixels of an image or multiple images. One or more command buffers define actions for the GPU to perform. For example, an action performed by the GPU may include: rendering an object via a drawing command and the state information required for rendering the object. Another action performed by the GPU may include: performing a kernel call via a kernel call command and the state information required for executing the kernel. Other actions performed by the GPU may include: a synchronization command for waiting for a drawing command, or a kernel call, or a graphics pipeline or a certain other condition to be completed. In addition, other actions may include: configuring the GPU (including configuring a buffer or an image for kernel call), the position and format of a rendering target (e.g., MRT), scanning output, depth test status, etc.

[0068] The GPU executes commands, which can be executed by the GPU to perform graphics processing (such as rendering an object) or non-graphics functions (such as executing a kernel call). A "command" is data that the GPU reads and performs actions based on the command. A "kernel call command" is a specific command used to execute a kernel call. A "draw command" is a specific command used to render an object.

[0069] A "command buffer" is a container for one or more commands, wherein the GPU executes the commands by reading the commands from the corresponding command buffer. In particular, the GPU may be configured to execute commands from the corresponding command buffer. The commands and / or operations executed when rendering objects and / or executing kernels may be ordered, so that the commands and / or operations may depend on other commands and / or operations (e.g., commands in a command buffer may need to be completed before other commands in the command buffer can be executed). In addition, the commands and / or operations executed by a GPU may depend on other commands and / or operations executed by another GPU, so that they are executed in sequence by one or more GPUs. In one embodiment, each GPU may have its own command buffer. Alternatively, the GPU may use the same command buffer or the same set of command buffers (e.g., when each GPU is rendering a substantially identical set of objects).

[0070] In addition, command buffers may be limited to execution on all GPUs or a subset of GPUs in a multi-GPU architecture. In a multi-GPU architecture, it may be necessary to explicitly copy memory between GPUs using commands in command buffers. As will be further described, embodiments of the present disclosure do not synchronize GPUs via synchronization commands in command buffers, but rather minimize the use of synchronization commands by using dependent data. In addition, embodiments of the present disclosure can perform static and / or dynamic load balancing of workloads between multiple GPUs.

[0071] Many architectures are possible in which multiple GPUs collaborate to render images or execute kernels. For example, a multi-GPU architecture can be implemented between one or more cloud gaming servers of a cloud gaming system, or within a standalone system such as a personal computer or game console that includes a high-end graphics card with multiple GPUs. In one embodiment, each GPU of the multi-GPU architecture is able to access shared memory via a high-speed bus. In another multi-GPU architecture, each GPU can have local memory accessed via a high-speed bus, and access to the memory of other GPUs can be performed via a low-speed bus, as previously described in another embodiment. Figure 2 Described in the architecture shown.

[0072] Figure 6 Illustrated is a kernel call 600 in N dimensions according to one embodiment of the present disclosure. In particular, the "index space" is an N-dimensional space for the kernel call 600, where the kernel function is executed for each point or index in the index space. For illustration purposes only, the kernel call 600 can be represented by a 2-dimensional space (x-dimension and y-dimension) including 9x8 indices.

[0073] The parts of kernel call 600 are assigned to multiple GPUs for execution. As previously described, a kernel or computing kernel as executed by kernel call 600 is a program executed on a GPU, which can read or write an image or buffer, wherein the kernel can include independent variables and / or values ​​associated with the independent variables used. The kernel call 600 shown can be divided into multiple parts based on the corresponding N-dimensional index range, wherein each part can be the entire index space used by the kernel or its subset in each dimension. In other words, "index range" defines a part of the N-dimensional index space, wherein one or more index ranges can be used to divide the kernel call into multiple parts. Thus, "part" defines a part of the kernel or kernel call 600, wherein each part can be an index or index range in the N-dimensional index space. As a representative example, kernel call 600 is divided into six parts along two dimensions. Each part of kernel call 600 includes 12 indexes, and the kernel function will be evaluated at these indexes. For example, portion 610 includes indexes (6,0), (7,0), (8,0), (6,1), (7,1), (8,1), (6,2), (7,2), (8,2), (6,3), (7,3), and (8,3). One or more portions of a kernel as executed by kernel call 600 may be associated with some dependency data.

[0074] A portion of a kernel, as executed by a kernel call, may read from or write to an area of ​​a "resource," which may be a memory resource. In particular, a resource may include input data and output data used by the kernel. For example, a resource may be a buffer resource or an image resource, and may be configured in multiple dimensions or as a cubemap. In some embodiments, a resource may be defined in one, or two, or three dimensions, or may be defined in a cubemap.

[0075] A "region" is a portion of a resource and is associated with a portion of a kernel. For example, a region may define an image portion corresponding to a portion of a 2-dimensional kernel as executed by a corresponding kernel call. In one embodiment, a region corresponds to a subset of a resource and includes each dimension used by the kernel, such as a range in a tile or buffer of an image resource. For example, a "tile" may be a region type (e.g., an image resource type) that defines an image portion. An index range of a corresponding portion of a kernel may be used to determine which region of a resource to process.

[0076] For illustration purposes, Fig. 7A The image resource 710 is illustrated as being segmented into twenty-four regions. Furthermore, each region in the image resource 710 has sixty-four elements. For example, the enlarged image of region (2,2) includes 8x8 or sixty-four elements. Also for illustration purposes, Figure 7BThe buffer resource 720 is illustrated as being divided into four regions. In addition, each region in the buffer resource 720 has eight elements. For example, the enlarged image of region 2 includes eight elements.

[0077] With the help of previous Figure 1 A detailed description of the multi-GPU architecture and its implementation described in FIG. 7 , Figure 8 Flowchart 800 illustrates a method for graphics processing according to one embodiment of the present disclosure, the method comprising using multiple GPUs to process kernels, wherein the multiple GPUs collaborate to process images or data. As previously described, various architectures may include multiple GPUs that collaborate to process images or data, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), etc.

[0078] At 810, the method includes executing a plurality of kernels using a plurality of GPUs, wherein each kernel includes a program executing on a multi-GPU architecture that can read or write to an image or buffer resource. In addition, a kernel call is used to execute a corresponding kernel, wherein a kernel is a program executing on one or more GPUs that can read or write data in an image or buffer resource. The kernel call can be defined by an index space, wherein each index can include an argument and / or a value associated with the argument for executing a corresponding portion of the kernel, wherein the portion is defined by an index range in the index space.

[0079] At 820, the method includes dividing the responsibility of executing the kernel into multiple parts. There may be some GPUs that have other responsibilities and are not used to execute kernels.

[0080] At 830, the method includes: allocating multiple parts to multiple GPUs, wherein each part is allocated to a corresponding GPU. In particular, a kernel call can be divided into multiple parts, wherein the multiple parts are allocated to corresponding GPUs and then executed. The GPUs execute simultaneously. Each GPU can share a command buffer, or have a different command buffer (e.g., each GPU has one or more dedicated command buffers), wherein the command buffer can contain kernel call commands and other commands (e.g., draw call commands, etc.).

[0081] Fig. 9The fixed or static allocation scheme for evenly distributing the parts of kernel 910 for execution across multiple GPUs according to an embodiment of the present disclosure is illustrated. In particular, the parts of kernel 910 as executed by corresponding kernel calls are evenly divided to run on all GPUs. As shown in the figure, the two-dimensional index space of kernel 910 may include twenty-four indexes. The uniform distribution of kernel 910 may allocate equal amounts of indexes to each of four GPUs (e.g., GPU A, GPU B, GPU C, and GPU D) for execution, so that six indexes are allocated to each GPU. For example, in a fixed allocation scheme, GPU A may be allocated six indexes (0,0), (1,0), (2,0), (0,1), (1,1), and (2,1). In addition, GPU B may be allocated six indexes (3,0), (4,0), (5,0), (3,1), (4,1), and (5,1). In addition, GPU C may be allocated six indexes (0,2), (1,2), (2,2), (0,3), (1,3), and (2,3). GPU D may also be assigned six indices (3,2), (4,2), (5,2), (3,3), (4,3), and (5,3). Since the allocation scheme is fixed and the amount of execution time for each part may not be equal, this may cause an imbalance in the workload between GPUs. In other embodiments of the present disclosure, a dynamic allocation scheme may be implemented using dependent data, as described below in conjunction with Fig.10 and FIG. 11A to FIG. 11B Further described.

[0082] With the help of previous Figures 1 to 9 A detailed description of the multi-GPU architecture and its implementation described in, Fig.10 Flowchart 1000 illustrates a method for graphics processing according to one embodiment of the present disclosure, the method comprising using multiple GPUs to process kernels, wherein dependency tracking is performed per kernel portion or per region per portion to allow early processing of dependent portions of subsequent kernels, wherein multiple GPUs collaborate to process images or data.

[0083] As previously described, various architectures may include multiple GPUs that collaborate to process images or data, such as within one or more cloud gaming servers of a cloud gaming system, or within a standalone system (such as a personal computer or game console that includes a high-end graphics card with multiple GPUs), etc. The GPUs execute simultaneously. In an embodiment, multiple GPUs may share a command buffer, or each GPU may have one or more dedicated command buffers. The command buffer may contain kernel call commands as well as other commands (such as draw call commands).

[0084] At 1010, the method includes executing multiple kernels using multiple graphics processing units (GPUs). As previously described, each kernel includes a program that executes on a multi-GPU architecture, and the program can read data from or write data to an image or buffer resource. In addition, a kernel call is used to execute a corresponding kernel. The kernel call can be defined by an index space, where each index can include an independent variable and / or a value associated with the independent variable for executing a corresponding portion of the kernel, where the portion is defined by one or more index ranges in the index space.

[0085] In addition, the responsibility for executing the corresponding kernel is divided into one or more parts, each of which is assigned to the corresponding GPU. That is, the kernel call being processed is split or divided into multiple parts. Each part is assigned to the corresponding GPU for execution. As mentioned above, the GPU cooperates in executing one or more kernels to process images or data. In addition, the resources (e.g., memory) read by the kernel can be divided into one or more regions, wherein a part can read from and / or write to one or more regions of one or more resources.

[0086] At 1020, the method includes: generating a plurality of dependency data at the first kernel when each of the first plurality of parts of the first kernel completes processing. That is, the dependency data is generated when the corresponding part of the kernel completes processing, wherein the kernel part can write the dependency data before the kernel part completes execution (that is, before all instructions of the kernel part have completed execution). Dependency data is information that can be generated by each part of the kernel. For example, the information can indicate that the processing of the corresponding part of the kernel has been completed. For another example, the information can indicate that a write has been made to an area of ​​the resource by the corresponding part of the kernel. In particular, after a part of the kernel completes writing to an area of ​​the resource, the part can generate dependency data, the dependency data including that the writing of the part to the area has been completed and / or which GPU writes to the area. In this way, while processing image or buffer resources, the GPU is allocated parts of the kernel and can track dependency data between these parts, thereby allowing workload balancing across GPUs. In addition, fine-grained, area-based dependency data can be generated and used between kernels.

[0087] At 1030, the method includes: before executing a portion of the second kernel, checking dependency data generated from one or more portions of the first kernel. In particular, the subsequently executed portion of the second kernel will wait for the dependency data to ensure that each region of one or more resources that need to be read by the portion of the second kernel has been fully written by the first kernel. For example, the dependency data generated by the portion of the first kernel indicates the completion of one or more writes to one or more regions of resources or the completion of execution of the portion of the first kernel.

[0088] In one embodiment, once a region has been written to by a GPU (e.g., executing a portion of a first kernel), the written data is sent to other GPUs. In embodiments of the present disclosure, once all regions required by a portion of a second kernel have been written to by a previously executed first kernel (e.g., another portion of the first kernel), execution of the portion of the second kernel can begin without waiting for other portions of the first kernel to complete their copy operations and without using synchronization points for copying.

[0089] In another embodiment, the GPU can pre-fetch the resource region required by the kernel portion to be executed. GPU information (i.e., which GPU in the dependent data writes to the region) can be used to determine which GPU to read the data from. In some cases, the region data is likely to reside in the local memory of the requesting GPU.

[0090] At 1040, the method includes delaying execution of the portion of the second kernel as long as the corresponding dependency data of the first kernel is not satisfied. That is, the portion of the second kernel checks using the dependency data to ensure that all areas it requires have been written, after which the portion of the second kernel can begin processing. That is, the dependency data indicates whether a portion of a kernel can access a required area. In particular, the dependency data can be used to synchronize the execution of a later kernel call or a portion of a kernel (e.g., a second kernel).

[0091] Fig.11A The generation of dependency data based on a portion of a kernel according to one embodiment of the present disclosure is illustrated. In particular, kernel A and kernel B are each divided into three corresponding parts: part 0, part 1, and part 2. Resource 0 and resource 1 are each divided into three corresponding regions: region 0, region 1, and region 2.

[0092] In particular, a portion of a kernel may generate dependency data when its processing is completed or when writing to a resource area is completed. For example, portion 0 of kernel A may generate dependency data based on the portion, so that the dependency data is written only after portion 0 of kernel A is completed or when portion 0 of kernel A completes writing to region 0 of resource A and resource B. In particular, portion 0 of kernel A writes to region 0 of resource 0 along path 1110, and also writes to region 0 of resource 1 along path 1115. Upon completion of its processing or writing to the area, portion 0 of kernel A also writes dependency data DD0, as indicated by path 1120. In one embodiment, dependency data (e.g., DDO, or DD1, or DD2) may be stored in any array. For example, dependency data may be stored in an index range, wherein each dimension is shifted by an amount or divided by an amount, and the resulting value is used as an index into the array.

[0093] The portion 0 of kernel B depends on data from both resource 0 and resource 1, as indicated by the dependency data DD0. In particular, the portion 0 of kernel B waits for the dependency data DD0, and then after DD0 is generated, is able to read region 0 of resource 0, and read region 0 of resource 1. That is, the portion 0 of kernel B is able to check the state of the dependency data DDO, and is able to access DD0 as it is generated and stored, as indicated by path 1125. Because the portion 0 of kernel B can determine that all of its required resource regions are accessible via the dependency data, the portion can begin execution without waiting for the portion of kernel A to complete its processing and copying, and without using synchronization points (e.g., for copying).

[0094] Fig. 11B The diagram illustrates dependency data of resource-based regions and kernel portions according to one embodiment of the present disclosure. In particular, kernel A and kernel B are each divided into three corresponding portions: portion 0, portion 1, and portion 2. Resource 0 and resource 1 are each divided into three corresponding regions: region 0, region 1, and region 2.

[0095] Dependency data is generated for each part and each region of a resource. In particular, a part of a kernel may generate dependency data upon completion of all writes to a resource (e.g., a region of a resource). For example, part 0 of kernel A writes to region 0 of resource 0 along path 1130. Upon completion of all writes to region 0 of resource 0, part 0 of kernel A may generate dependency data DD0 and store dependency data DD0 in dependency data storage area 0 (e.g., an array) along path 1135. Additionally, part 0 of kernel A writes to region 0 of resource 1 along path 1140. Upon completion of all writes to region 0 of resource 1, part 0 of kernel A may generate dependency data DD0 and store dependency data DDO in dependency data storage area 1 (e.g., an array) along path 1145.

[0096] Core B's section 0 depends on data from both resource 0 and resource 1. Fig. 11B The dependency data generated in FIG. 11 can be finer in granularity than the dependency data generated in FIG. 11 because the dependency data is generated for each part and each region of the resource. In particular, part 0 of core B waits for two sets of dependency data, including dependency data DD0 of dependency data storage area 0 and dependency data DD0 of dependency data storage area 1.

[0097] In particular, portion 0 of kernel B waits for dependency data DD0 of dependency data storage area 0, and then after DD0 is generated, can read region 0 of resource 0 along path 1150. Portion 0 of kernel B can check the state of dependency data DD0 of data storage area 0, and can access the dependency data DD0 when it is generated and stored, as indicated by path 1155. In addition, portion 0 of kernel B waits for dependency data DD0 of dependency data storage area 1, and then after DD0 is generated, can read region 0 of resource 1 along path 1160. Portion 0 of kernel B can check the state of dependency data DD0 of data storage area 1, and can access the dependency data DD0 when it is generated and stored, as indicated by path 1165.

[0098] Because part 0 of kernel B can determine that all of its required resource areas are accessible via dependent data (DD0 of dependent data storage area 0 and DD0 of dependent data storage area 1), the part can start executing without waiting for part of kernel A to complete its processing and copying, and without using synchronization points (for example, for copying).

[0099] In another embodiment, once a portion of a kernel has completed writing to a corresponding region, the region data for the corresponding resource can be pushed to all GPUs (i.e., sent to the local memory of all GPUs). In this case, subsequent kernels using the region data can wait for the data to arrive at their local memory. In particular, when a portion of a first kernel is executed on a first GPU, the data generated by the first kernel at the first GPU is sent to the local memory of a second GPU.

[0100] As previously described, the GPU can pre-fetch resource regions required by the portion of the kernel to be executed. That is, the region data can be obtained before the data is needed. In particular, once a portion of a first kernel executed on a first GPU has completed writing to a region of resources and generated corresponding dependency data, a portion of a second kernel executed on a second GPU (which may be waiting for the dependency data) can then read the region. The second GPU (executing the portion of the second kernel) knows which GPU to read memory from because the information may be part of the dependency data. In some embodiments, efficient pre-fetching uses the known execution order of subsequent portions to determine which resource regions required by those portions are to be copied.

[0101] In one embodiment, the second GPU can read the region data directly from the local memory of the first GPU, such as via Figure 2In another embodiment, the second GPU may copy the local data from the local memory of the first GPU to the local memory of the second GPU before executing the portion of the second kernel. In this case, the portion of the second kernel may then read the local data from its local memory.

[0102] In another embodiment, the second GPU uses both the above-mentioned reading method and copying method to include: reading the regional data directly from the local memory of the first GPU; and copying the regional data from the corresponding local memory to the local memory of the second GPU. In particular, when the part of the second kernel is executed, the second GPU starts to copy the regional data from the local memory of the first GPU to the local memory of the second GPU. For example, the data generated by the first GPU and written to the local memory of the first GPU is accessed by the second GPU before the direct memory access (DMA) is completed, wherein the access is performed directly from the local memory of the first GPU through a normal read operation. While copying, the second GPU reads the regional data directly from the local memory of the first GPU. That is, the second GPU can start early processing of its part of the second kernel by performing a direct read from the local memory of the first GPU. For example, the data generated by the first kernel executed on the first GPU and written to the local memory of the first GPU is extracted via DMA to the local memory of the second GPU executing the second kernel. Before the copy is completed, the second GPU reads the data directly from the first GPU. After the copy is completed, the second GPU then reads the regional data directly from the local memory of the second GPU. For example, data generated by a first GPU and written to the local memory of the first GPU is accessed by a second GPU from the local memory of the second GPU after completing DMA. In this way, only the area required to execute the portion of the second kernel is read, thereby reducing the bandwidth across the multi-GPU architecture.

[0103] In another embodiment, if the execution order of the portion of the second kernel is known, the second GPU can use the order to predict which portions of the second kernel may be executed next. In this way, the second GPU can pre-fetch the corresponding regions of one or more resources that the portion of the second kernel will use as input before execution. That is, based on the predefined or predicted order of the second kernel at the second GPU, the data generated by the first kernel executed on the first GPU can be pre-fetched into the local memory of the second GPU. It is assumed here that dependency data for those regions has been generated, indicating that these regions have been written. If the regional data already exists in the local memory of the GPU that executes the kernel portion, the order can also be used to pre-fetch from the local memory to a faster local cache memory, thereby further increasing bandwidth and / or reducing latency.

[0104] In an embodiment, when a portion of a kernel examines corresponding dependency data, the portion may use various strategies to reference index ranges associated with itself (e.g., in a corresponding index space associated with a kernel invocation) and with the portion of the kernel that generated the dependency data. For example, FIG. 12A to FIG. 12D Representatively illustrated is a portion of a kernel checking dependency data as a function of the index range of the portion according to an embodiment of the present disclosure. It is understood that any function may be used to check dependency data.

[0105] For the sake of clarity and brevity, FIG. 12A to FIG. 12D Each of shows that each part of the kernel has a unit-sized index range. In addition, FIG. 12A to FIG. 12D In each of the above, kernel A (not shown) writes to a region of resource A and generates and stores associated dependency data. Kernel B's portion A with index range (2,2) or (1,1) reads resource A and writes to resource B. Four different strategies for checking dependency data are described in FIG. 12A to FIG. 12D As shown in FIG. 1 , part A of kernel B can check dependent data through these strategies.

[0106] In particular, one or more portions of a first kernel (e.g., kernel A) whose dependency data is checked prior to executing a portion of a second kernel (e.g., kernel B) depend on an index range for each dimension that includes the portion of the second kernel (e.g., kernel B). In one embodiment, dependency data associated with the portion of the first kernel (e.g., kernel A) is checked, wherein the portion includes an index range for each dimension or an offset thereof that corresponds to those ranges of the second kernel (e.g., kernel B). In another embodiment, dependency data associated with multiple portions of the first kernel (e.g., kernel A) are checked, wherein the portions include an index range for each dimension that, taken together, is a superset of the index range for each dimension of the second kernel. In yet another embodiment, one or more dependency data associated with a portion of the first kernel (e.g., kernel A) are checked, wherein the one or more portions include at least one index range for each dimension that is a function calculated using the index range of the second kernel (e.g., kernel B).

[0107] In particular, Fig. 12A FIG. 1 illustrates checking dependency data by portion A of kernel B, which corresponds to an index range (e.g., 2, 2) of the portion, according to one embodiment of the present disclosure. For example, portion A of kernel B is associated with the index range (2, 2) in the corresponding index space. In addition, portion A of kernel B writes to region (2, 2) of resource B, such as Fig. 12AThere is a one-to-one mapping between a region in resource A that is read from or written to by part A of kernel B, and a region in resource B to which the same data or some function thereof is written. That is, region (2,2) of resource B that receives output data is in the same location as region (2,2) of resource A from which data is read. In other words, there is a one-to-one relationship between the region index ((2,2) of resource A) and the part index ((2,2) of kernel B).

[0108] Fig. 12B The diagram shows that part A of kernel B checks multiple slices of dependency data associated with an index range according to one embodiment of the present disclosure. In particular, dependency data corresponding to an index range within a radius of one of the index range (2,2) or part IDs of part A of kernel B is checked. For example, part A of kernel B can be a filter function, in which multiple areas around a center pixel (e.g., corresponding to area (2,2) of resource A associated with the index range of part A of kernel B) are read and filtered, with an output for area (2,2) of resource B. That is, the filter function defines a sampling area around the index range (e.g., part ID) of part A of kernel B. As shown in the figure, the sampling areas from which data is read include areas (1,1), (2,1), (3,1), (1,2), (2,2), (3,2), (1,3), (2,3), and (3,3) of resource A. The output generated from the function is stored in area (2,2) of resource B.

[0109] Fig. 12C The diagram shows that according to one embodiment of the present disclosure, part A of kernel B checks dependency data, which is a function of the index range of the part. In particular, the dependency data is checked based on a function of the index range (1,1) or the part ID of part A of kernel B. In this case, the function is to double the index range (1,1) of part A of kernel B and take its neighbor in the lower right direction. As shown in the figure, the area sampled and read from it includes areas (2,2), (3,2), (2,3) and (3,3) of resource A. The output generated from the function is stored in area (1,1) of resource B. That is, the function represents a downsampling operation.

[0110] Fig.12DThe diagram shows that according to one embodiment of the present disclosure, part A of kernel B checks dependency data, which is a function of the index range of the part, wherein the function is an offset to the index range of the part. In particular, the dependency data is checked based on a function of the index range (2,2) or the part ID of part A of kernel B. In this case, the function is to offset the index range (2,2) of part A of kernel B in an upward direction. As shown, the area sampled and read from is area (2,1) of resource A. The output generated from the function is stored in area (2,2) of resource B.

[0111] Functions that use the index range of the kernel may result in indexes outside the defined index space. In these cases, further operations can be performed. In particular, FIG. 13A to FIG. 13C Various strategies for addressing when a function of the index range of a portion of a kernel is outside of a defined index space are illustrated in accordance with embodiments of the present disclosure.

[0112] For the sake of clarity and brevity, FIG. 13A to FIG. 13C Each of shows each portion of the kernel having a unit-sized index range (e.g., a "unit" index range). FIG. 13A to FIG. 13C In each of , kernel A writes to a region of resource A and generates and stores associated dependency data. The portion A of kernel B with index range (2,2) reads resource A and writes to resource B. Different strategies for the index range of the portion of the first kernel that is outside the index space of the first kernel are described in FIG. 13A to FIG. 13C Specifically, an offset of (-3,-1) is applied to the unit index range (2,2), which results in a unit index range (-1,1) (also in Fig. 13C ), which is outside the index space in the first dimension (eg, the horizontal direction or the X direction).

[0113] Fig.13A The diagram illustrates clamping of one dimension of an index range when an offset (e.g., function) of an index range applied to a portion A of kernel B falls outside of a defined index space according to one embodiment of the present disclosure. The offset is transformed into a corresponding region of a similar dimension in resource A. In particular, if an offset index range, a superset index range, or a calculated index range for the portion of kernel A is outside of the index space, dependency data generated by kernel A is checked for a portion of kernel A corresponding to the index range (which is clamped to a valid range in a dimension that is outside of the index space), wherein the clamping ensures that the resulting index range (0,1) is within the index space. For example, the index range is clamped to its first value 0 in the horizontal or X direction so that dependency data is checked for region (0,1) of resource A.

[0114] Fig. 13B The diagram illustrates the wrapping of an index range in one dimension when an offset (e.g., a function) of an index range applied to a portion of kernel B falls outside a defined index space according to one embodiment of the present disclosure. The offset is transformed into a corresponding region of a similar dimension in resource A. In particular, if the offset index range, superset index range, or calculated index range for the portion of kernel A is outside the index space, dependency data generated by kernel A is checked for a portion of kernel A corresponding to the index range (which is wrapped to a valid range in a dimension outside the index space). In particular, the index range is wrapped to (5, 1), wherein the wrapping ensures that the resulting index range (5, 1) is within the index space. In one embodiment, the value is selected as an unsigned modulus of the index having the size of the index space in the direction. Other methods are also applicable when performing the wrapping. For example, the index range is wrapped to its maximum value of 5 in the horizontal or X direction so that dependency data is checked for region (5, 1) of resource A.

[0115] Fig. 13C Illustrated is the ignoring of dependency data when an offset to an index range applied to a portion of a kernel (e.g., a function) falls outside of a defined index space, according to one embodiment of the present disclosure. In particular, dependency data generated by kernel A is ignored. Thus, the portion of kernel B may decide not to wait for the dependency data, or decide not to execute the portion of kernel B or process it in some other manner.

[0116] In some embodiments, the functions executed to read the index range of dependent data and the actions performed if the result is outside the defined index space may be different for each resource and for each kernel. That is, these functions may be based on the relevant resources and kernels.

[0117] Fig.14A A dynamic allocation scheme according to one embodiment of the present disclosure is illustrated, wherein different GPUs follow different space filling curves when allocating portions of a kernel to multiple GPUs for execution. In particular, portions of a kernel are dynamically allocated to GPUs, wherein a single GPU is granted an allocation for a unique portion of a kernel such that each portion is allocated to a corresponding GPU for execution. As shown, the two-dimensional index space 1410 of the kernel may include twenty-four indexes.

[0118] A dynamic allocation scheme may be used, in which portions of a kernel are allocated to a GPU as the GPU becomes capable of executing (e.g., available for execution). The responsibility for executing each portion of the kernel is assigned to one and only one GPU, and portions of the kernel are dynamically allocated to the GPU as the kernel is executed. That is, the allocation of portions of the kernel may refer to a predefined order (e.g., a space-filling curve) that may be different for each GPU. Fig.14A 1 represents one or more predefined orders or space filling curves, but it should be understood that other orders may be more efficient for other access patterns. In particular, the order may be different for each GPU in order to achieve locality of the portion assigned to each GPU. As an advantage, even when each GPU uses a different order, using the same order for multiple kernels (e.g., serially) on the same GPU reduces the need to obtain data from other local memories of other GPUs, because the data may reside in the GPU due to the locality of the portion. Fig.14A As shown, an attempt is made to allocate portions of the kernel to the corresponding GPU using a known predefined order for each GPU.

[0119] Fig.14A Multiple orders or space filling curves are shown, each corresponding to a GPU. For example, GPU A follows space filling curve 1420, which starts at index range (0,1) of index space 1410, checks the index range in an approximately clockwise manner, and ends at index range (0,2). For purposes of illustration, space filling curve 1420 attempts to allocate portions of the kernel in the following order: (0,1), then (0,0), then (1,0), then (1,1), then (2,1), etc. In addition, GPU B follows space filling curve 1425, which starts at index range (5,1) of index space 1410, checks the index range in an approximately counterclockwise manner, and ends at index range (5,2). For purposes of illustration, space filling curve 1425 attempts to allocate portions of the kernel in the following order: (5,1), then (5,0), then (4,0), then (4,1), then (3,1), etc. In addition, GPU C follows a space filling curve 1430 that starts at index range (0,2) of index space 1410, checks the index range in an approximately counterclockwise manner, and ends at index range (0,1). For purposes of illustration, space filling curve 1430 attempts to allocate portions of the kernel in the following order: (0,2), then (0,3), then (1,3), then (1,2), then (2,2), etc.

[0120] In addition, GPU D follows a space filling curve 1435 that starts at index range (5,2) of index space 1410, checks the index range in an approximately clockwise manner, and ends at index range (5,1). For purposes of illustration, space filling curve 1435 attempts to allocate portions of the kernel in the following order: (5,2), then (5,3), then (4,3), then (4,2), then (3,2), etc.

[0121] In one embodiment, when a portion of a kernel is used as input in multiple regions that are passed continuously (which are spatially close together in a buffer resource or an image resource), one or more space filling curves optimized for the locality of each GPU reduce the need to fetch data from the local memory of other GPUs. In particular, the predefined order referenced by the corresponding GPU when allocating portions of the kernel can be a space filling curve defined within the dimension of the kernel or the corresponding kernel call.

[0122] In one embodiment, the progress along the corresponding allocation order (e.g., the distance of the space-filling curve from the starting point) can be shared among multiple GPUs. This progress provides an efficient way to check which parts of the kernel have been attempted to be allocated to each GPU, so that the GPU does not need to try to allocate parts that have already been allocated.

[0123] Fig. 14B The diagram shows a method according to an embodiment of the present disclosure as defined in Fig.14A The portions of the kernel are allocated in the order of allocation outlined in the GPU space filling curves in step 1410 (e.g., curve 1420 for GPU A, curve 1425 for GPU B, curve 1430 for GPU C, and curve 1435 for GPU D). Each GPU is allocated a portion simultaneously, such as in steps. Because the space filling curves start at a different index range, each GPU is initially allocated a portion of the corresponding kernel for execution. For example, in step 1, GPU A is allocated portion (0,1) of index space 1410, GPU B is allocated portion (5,1), GPU C is allocated portion (0,2), and GPU D is allocated portion (5,2).

[0124] Because some portions may take longer to complete than others (e.g., depending on input values, etc.), some GPUs end up getting more portions allocated from index space 1410. For example, GPU C continues to execute portion (0, 2) through steps 1 through 3 and is not allocated any additional portions in steps 2 or 3, while each of GPU A, GPU B, and GPU D is allocated additional portions in each of steps 2 and 3.

[0125] In some cases, a GPU can allocate more than one portion. For example, in step 4, GPU A can allocate three additional portions, including portions (1,1), (2,1), and (2,0). In addition, in step 4, GPU B can allocate two additional portions, including portions (4,1) and (3,1).

[0126] like Fig. 14B As shown, each GPU is assigned a portion according to locality so that the portions are localized to each other. In some cases, a GPU may not be able to make an allocation according to the localization requirement. For example, in step 5, GPU B attempts to allocate two portions (3,0), (2,0), (2,1) ..., but each of these portions has been allocated by GPU A and / or GPU C. The first portion available to GPU B that has not yet been allocated according to the space filling curve 1425 is portion (1,2). That is, GPU B allocates the next portion that has not yet been allocated to the GPU along its predefined order or space filling curve 1425.

[0127] With the help of previous Figure 1 to Figure 1 4 and a detailed description of the multi-GPU architecture and its implementation, Fig.15 Flowchart 1500 illustrates a method for graphics processing according to one embodiment of the present disclosure, the method comprising multi-GPU rendering of an image for an application, wherein dependent data may be executed based on kernel processing and / or draw calls, wherein multiple GPUs collaborate to process images or data. As previously described, various architectures may include multiple GPUs that collaborate to process images or data, such as within one or more cloud gaming servers of a cloud gaming system, or within an independent system (such as a personal computer or game console including a high-end graphics card with multiple GPUs), and the like. The GPUs execute simultaneously. In an embodiment, multiple GPUs may share one or more command buffers, or each GPU may have one or more dedicated command buffers. The command buffer may include kernel call commands and other commands, such as draw call commands, or a combination of kernel call commands and draw call commands.

[0128] At 1510, the method includes rendering an image using multiple graphics processing units (GPUs). For example, multiple GPUs that collaborate to generate images and / or data can be used to render graphics for an application. Multi-GPU processing is performed when rendering a single image and / or each of one or more images in an image sequence for a real-time application.

[0129] At 1520, the method includes executing the plurality of kernels on the plurality of GPUs. The responsibility for executing the corresponding kernels is divided into one or more parts, where each part is assigned to a corresponding GPU.

[0130] At 1530, the method includes executing a plurality of draw calls on a plurality of GPUs. The responsibility for executing a corresponding draw call is divided into one or more parts, wherein each part is assigned to a corresponding GPU. In particular, when rendering an image via a draw command, the corresponding draw command may be divided into a plurality of parts (in a manner similar to how a call to a kernel is divided into a plurality of parts), wherein each part is assigned to a GPU such that each part is assigned to only one GPU. In this manner, each GPU renders the portion of the draw call that has been assigned to the same GPU. Furthermore, each portion of the draw call may generate dependency data (e.g., upon completion of the portion of the draw call).

[0131] In one embodiment, there may be dependency checks between portions of a draw call and / or portions of a kernel. That is, a portion of a draw call depends on one or more portions of a kernel, or a portion of a kernel depends on one or more portions of a draw call. The dependency data may be generated by a portion of a kernel, or may be generated by a portion of a draw call.

[0132] At 1540, the method (optionally) includes, prior to executing the portion of the kernel, checking dependency data for one or more portions of the draw call at the kernel. In particular, the portion of the kernel may check dependency data generated by the portion of the draw call and take appropriate action, such as suspending execution of the corresponding portion of the kernel until the corresponding portion of the draw call has completed.

[0133] At 1550, the method (optionally) includes checking dependency data for one or more portions of the kernel at the draw call prior to executing a portion of the draw call. In particular, each portion of the draw call may check dependency data (e.g., not process until dependencies are satisfied), and / or generate dependency data (generate dependency data when the portion of the draw call is completed). In one embodiment, a portion of the draw call may check dependency data generated by a portion of the kernel and take appropriate action, such as suspending execution of the corresponding portion of the draw call until the corresponding portion of the kernel has completed.

[0134] Fig.16 Components of an exemplary apparatus 1600 that may be used to perform aspects of various embodiments of the present disclosure are illustrated. For example, Fig.16An exemplary hardware system suitable for executing a kernel according to an embodiment of the present disclosure is illustrated, wherein multiple graphics processing units (GPUs) collaborate to process images or data. The block diagram shows a device 1600, which may be incorporated into or may be a personal computer, a server computer, a game console, a mobile device, or other digital device, each of which is suitable for practicing embodiments of the present invention. The device 1600 includes a central processing unit (CPU) 1602 for running software applications and optionally an operating system. The CPU 1602 may include one or more homogeneous or heterogeneous processing cores.

[0135] According to various embodiments, CPU 1602 is one or more general purpose microprocessors having one or more processing cores. Additional embodiments may be implemented using one or more CPUs having a microprocessor architecture that is particularly suitable for highly parallel and computationally intensive applications such as media and interactive entertainment applications configured for graphics processing during execution of a game.

[0136] Memory 1604 stores applications and data for use by CPU 1602 and GPU 1616. Storage 1606 provides non-volatile storage and other computer-readable media for applications and data and may include fixed disk drives, removable disk drives, flash memory devices and CD-ROM, DVD-ROM, Blu-ray disc, HD-DVD or other optical storage devices, as well as signal transmission and storage media. User input device 1608 communicates user input from one or more users to device 1600, examples of which may include keyboard, mouse, joystick, touch pad, touch screen, static or video recorder / camera and / or microphone. Network interface 1609 allows device 1600 to communicate with other computer systems via an electronic communication network, and may include wired or wireless communications on a local area network and a wide area network such as the Internet. Audio processor 1612 is adapted to generate analog or digital audio output from instructions and / or data provided by CPU 1602, memory 1604 and / or storage 1606. Components of device 1600 , including CPU 1602 , graphics subsystem including GPU 1616 , memory 1604 , data storage device 1606 , user input device 1608 , network interface 1609 , and audio processor 1612 , are connected via one or more data buses 1622 .

[0137] The graphics subsystem 1614 is also connected to the data bus 1622 and the components of the device 1600. The graphics subsystem 1614 includes at least one graphics processing unit (GPU) 1616 and a graphics memory 1618. The graphics memory 1618 includes a display memory (e.g., a frame buffer) for storing pixel data for each pixel of an output image. The graphics memory 1618 may be integrated in the same device as the GPU 1616, connected to the GPU 1616 as a separate device, and / or implemented in the memory 1604. The pixel data may be provided directly from the CPU 1602 to the graphics memory 1618. Alternatively, the CPU 1602 provides the GPU 1616 with data and / or instructions defining the desired output image, and the GPU 1616 generates pixel data for one or more output images according to the data and / or instructions. The data and / or instructions defining the desired output image may be stored in the memory 1604 and / or the graphics memory 1618. In an embodiment, GPU 1616 includes 3D rendering capabilities for generating pixel data for an output image based on instructions and data defining the geometry, lighting, shading, texturing, motion, and / or camera parameters of a scene. GPU 1616 may also include one or more programmable execution units capable of executing shader programs.

[0138] Graphics subsystem 1614 periodically outputs pixel data of an image from graphics memory 1618 for display on display device 1610 or for projection by a projection system (not shown). Display device 1610 may be any device capable of displaying visual information in response to a signal from device 1600, including CRT, LCD, plasma, and OLED displays. Device 1600 may provide, for example, an analog or digital signal to display device 1610.

[0139] Other embodiments of optimizing graphics subsystem 1614 may include using multiple GPUs to process kernels, where dependency tracking of each kernel portion is performed to allow early processing of dependent portions of subsequent kernels, where multiple GPUs collaborate to process images or data. Graphics subsystem 1614 may be configured as one or more processing devices.

[0140] For example, in one embodiment, graphics subsystem 1614 may be configured to perform multi-GPU rendering of geometry for an application by performing region testing while rendering, where multiple graphics subsystems may implement graphics and / or rendering pipelines for a single application. That is, graphics subsystem 1614 includes multiple GPUs for rendering each of one or more images in an image or sequence of images while executing an application.

[0141] In other embodiments, the graphics subsystem 1614 includes multiple GPU devices, which are combined to perform graphics processing for a single application executed on a corresponding CPU. For example, multiple GPUs can perform multi-GPU rendering of geometric graphics for an application by region testing when rendering an object of an image. In other examples, multiple GPUs can perform frame rendering in an alternating form, wherein in a sequential frame cycle, GPU 1 renders the first frame, and GPU 2 renders the second frame, and so on, until the last GPU is reached, then the initial GPU renders the next video frame (for example, if there are only two GPUs, GPU 1 renders the third frame). That is, the GPU rotates when rendering a frame. Rendering operations can overlap, wherein GPU 2 can start rendering the second frame before GPU 1 finishes rendering the first frame. In another embodiment, different shader operations can be assigned to multiple GPU devices in rendering and / or graphics pipelines. The main GPU is performing main rendering and synthesis. For example, in a group including three GPUs, master GPU 1 can perform the main rendering (e.g., a first shader operation) and synthesize the output from slave GPU 2 and slave GPU 3, where slave GPU 2 can perform a second shader (e.g., a fluid effect, such as a river) operation, and slave GPU 3 can perform a third shader (e.g., a particle smoke) operation, where master GPU 1 synthesizes the results from each of GPU 1, GPU 2, and GPU 3. In this way, different GPUs can be assigned to perform different shader operations (e.g., waving a flag, wind, smoke generation, fire, etc.) to render a video frame. In yet another embodiment, each of the three GPUs can be assigned to a different object and / or scene portion corresponding to a video frame. In the above embodiments and implementations, these operations can be performed in the same frame cycle (simultaneous parallelism) or in different frame cycles (sequential parallelism).

[0142] Thus, the present disclosure describes methods and systems configured for multi-GPU rendering of geometry for an application through region testing while rendering objects for each of one or more images in an image or sequence of images while executing the application.

[0143] It should be understood that the various embodiments defined herein can be combined or assembled into specific embodiments using the various features disclosed herein. Therefore, the examples provided are only some possible examples and are not limited to the various embodiments that are possible by combining various elements to define more embodiments. In some examples, some embodiments may include fewer elements without departing from the spirit of the disclosed or equivalent embodiments.

[0144] Embodiments of the present disclosure may be practiced with various computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked over wired or wireless networks.

[0145] In view of the above embodiments, it should be understood that the embodiments of the present disclosure may employ various computer-implemented operations involving data stored in a computer system. These operations are those operations that require physical manipulation of physical quantities. Any operation described herein that forms a part of the embodiments of the present disclosure is a useful machine operation. The embodiments of the present disclosure also relate to devices or equipment for performing these operations. The equipment may be specially constructed for the desired purpose, or the equipment may be a general-purpose computer selectively activated or configured by a computer program stored in a computer. Specifically, various general-purpose machines may be used together with computer programs written according to the teachings of this article, or it may be more convenient to construct more specialized equipment to perform the desired operations.

[0146] The present disclosure may also be implemented as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of computer-readable media include hard drives, network attached storage devices (NAS), read-only memories, random access memories, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable medium may include a computer-readable tangible medium distributed on a network-coupled computer system so that the computer-readable code is stored and executed in a distributed manner.

[0147] Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or the operations may be coordinated so that they occur at slightly different times, or may be distributed in a system that allows the processing operations to occur at various intervals associated with the processing, so long as the processing covering the operations is performed in the desired manner.

[0148] Although the foregoing disclosure has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Therefore, the embodiments of the present invention are to be considered as illustrative rather than restrictive, and the embodiments of the present disclosure are not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.

Claims

1. A method comprising: dividing the first kernel into a plurality of parts during execution of the application; Allocating a first portion of the first kernel to a first graphics processing unit (GPU) for processing; assigning a second portion of the first kernel to a second GPU for processing; being executed by the first GPU to complete the first portion of the first kernel; storing first data from executing to complete the first portion of the first kernel on a first storage resource; generating dependency data after storing the first data on the first storage resource; as well as executing a second kernel after determining that the dependency data has been generated, Wherein, executing the second kernel begins before the second portion of the first kernel has been processed by the second GPU.

2. The method according to claim 1, further comprising: storing second data from executing the second portion of the first kernel on a second storage resource, The execution of the second kernel begins before the storage of the second data is completed.

3. The method according to claim 1, in, The dependency data is based on a corresponding portion of a corresponding kernel and a corresponding region of a corresponding storage resource.

4. The method according to claim 1, in, The dependency data is generated based on each region of the first storage resource and each of the plurality of portions of the first kernel.

5. The method according to claim 1, further comprising: storing the dependent data in a second storage resource; as well as The second storage resource is checked to determine whether the dependency data has been generated prior to executing the second kernel.

6. The method according to claim 1, further comprising: Sending the first data to a local memory of a third GPU tasked with executing the second kernel; or The first data is fetched to the local memory of the third GPU via direct memory access (DMA).

7. The method according to claim 1, further comprising: The portions of the first kernel are allocated one-to-one to multiple GPUs while the first kernel is being executed by the first GPU.

8. A non-transitory computer-readable medium storing a computer program for execution by a processor to perform a method, the non-transitory computer-readable medium comprising: program instructions for partitioning the first kernel into a plurality of portions during execution of the application; program instructions for assigning a first portion of the first kernel to a first graphics processing unit (GPU) for processing; program instructions for allocating a second portion of the first kernel to a second GPU for processing; program instructions for execution by the first GPU to complete the first portion of the first kernel; program instructions for storing first data from executing to complete said first portion of said first kernel on a first storage resource; program instructions for generating dependency data after storing the first data on the first storage resource; as well as program instructions for executing a second kernel after determining that the dependency data has been generated, Wherein, executing the second kernel begins before the second portion of the first kernel has been processed by the second GPU.

9. The non-transitory computer-readable medium of claim 8, further comprising: program instructions for storing second data from executing said second portion of said first kernel on a second storage resource, The execution of the second kernel begins before the storage of the second data is completed.

10. The non-transitory computer readable medium of claim 8, in, In the method, the dependency data is based on a corresponding portion of a corresponding kernel and a corresponding region of a corresponding memory resource.

11. The non-transitory computer readable medium of claim 8, in, In the method, the dependency data is generated according to each region of the first storage resource and each of the plurality of portions of the first kernel.

12. The non-transitory computer-readable medium of claim 8, further comprising: program instructions for storing the dependent data in a second storage resource; as well as Program instructions for checking the second storage resource to determine whether the dependent data has been generated prior to executing the second kernel.

13. The non-transitory computer-readable medium of claim 8, further comprising: program instructions for sending the first data to a local memory of a third GPU tasked with executing the second kernel; or Program instructions for fetching the first data to the local memory of the third GPU via direct memory access (DMA).

14. The non-transitory computer-readable medium of claim 8, further comprising: Program instructions for allocating the portions of the first kernel to a plurality of GPUs on a one-to-one basis while the first kernel is being executed by the first GPU.

15. A computer system comprising: processor; a memory coupled to the processor and having instructions stored therein, the instructions, if executed by the computer system, causing the computer system to perform a method comprising: dividing the first kernel into a plurality of parts during execution of the application; Allocating a first portion of the first kernel to a first graphics processing unit (GPU) for processing; assigning a second portion of the first kernel to a second GPU for processing; being executed by the first GPU to complete the first portion of the first kernel; storing first data from executing to complete the first portion of the first kernel on a first storage resource; generating dependency data after storing the first data on the first storage resource; and executing a second kernel after determining that the dependency data has been generated, Wherein, executing the second kernel begins before the second portion of the first kernel has been processed by the second GPU.

16. The computer system of claim 15, wherein the method further comprises: storing second data from executing the second portion of the first kernel on a second storage resource, The execution of the second kernel begins before the storage of the second data is completed.

17. The computer system according to claim 15, in, In the method, the dependency data is based on a corresponding portion of a corresponding kernel and a corresponding region of a corresponding memory resource.

18. The computer system according to claim 15, in, In the method, the dependency data is generated according to each region of the first storage resource and each of the plurality of portions of the first kernel.

19. The computer system of claim 15, wherein the method further comprises: storing the dependent data in a second storage resource; and The second storage resource is checked to determine whether the dependency data has been generated prior to executing the second kernel.

20. The computer system of claim 15, wherein the method further comprises: Sending the first data to a local memory of a third GPU tasked with executing the second kernel; or The first data is fetched to the local memory of the third GPU via direct memory access (DMA).

Images