Scalable Game Console CPU / GPU Design for Home Console and Cloud Gaming

Abstract

In a multi-GPU simulation environment, frame buffer management may be implemented by rendering respective frames of a video by a plurality of GPUs (306, 402, 504, 600, 704) or by rendering respective portions of each frame of a video (900 / 902; 1000 / 1002; 1100 / 1102). One of the GPUs controls HDMI frame output by: receiving frame information from other GPUs; and reading out the complete frame through a physically connected HDMI output port (1200). Alternatively, the outputs of the GPUs may be multiplexed together (1302).

Description

Technical Field

[0001] The present application generally relates to scalable game console CPU / GPU designs for home console and cloud gaming. Background Art

[0002] Analog consoles, such as computer game consoles, typically use a single chip, called a "system on a chip" (SoC), that contains a central processing unit (CPU) and a graphics processing unit (GPU). Due to semiconductor scaling challenges and yield issues, multiple chiplets can be linked by high-speed coherent buses to form a large chip. While such scaling solutions are slightly less than optimal in terms of performance than building a large monolithic chip, they are less expensive. Summary of the invention

[0003] As understood herein, SoC technology may be applied to video simulation consoles, such as gaming consoles, and in particular, a single SoC may be provided for a "light" version of a console, while multiple SoCs may be used to provide a "high-end" version of the console having greater processing and storage capabilities than the "light" version. The "high-end" system may also include more memory such as random access memory (RAM) and other features, and may also be used to use a cloud-optimized version of the same gaming console chip with more performance.

[0004] However, as further understood herein, such "high-end" multi-SoC designs pose challenges to software and simulation (gaming) designs, which must be scaled accordingly. As examples, challenges arise with non-uniform memory access (NUMA) and thread management, as well as providing hints to software to use the hardware in an optimal manner. In the case of GPUs working in concert, frame buffer management and control of high-definition multimedia (HDMI) outputs may be addressed. Other challenges may also be addressed herein.

[0005] Thus, an apparatus includes at least a first graphics processing unit (GPU) and at least a second GPU communicatively coupled to the first GPU. The GPUs are programmed to render respective portions of a video such that the first GPU renders a first portion of the video and the second GPU renders a second portion of the video, wherein the first portion and the second portion are different from each other.

[0006] In other words, the first GPU may be programmed to render a first frame of a video to provide a first output, and the second GPU may be programmed to render some but not all frames of the video to provide a second output. The frames rendered by the second GPU are different from the frames rendered by the first GPU. The first output and the second output may be combined to render the video. Additionally, or alternatively, the first GPU may be programmed to render all of some but not all rows of a frame of the video to provide a first row output, and the second GPU may be programmed to render some but not all rows of a frame of the video to provide a second row output. The rows rendered by the second GPU are different from the rows rendered by the first GPU. The first row output and the second row output may be combined to render the frame.

[0007] In some embodiments, the first GPU and the second GPU are implemented on a common die. In other embodiments, the first GPU and the second GPU are implemented on respective first and second dies. The first GPU may be associated with a first central processing unit (CPU), and the second GPU may be associated with a second CPU.

[0008] In some implementations, the first memory controller and the first memory are associated with a first GPU, and the second memory controller and the second memory are associated with a second GPU. In other implementations, the GPUs share a common memory controller that controls the common memory.

[0009] In some examples, each GPU is programmed to render some but not all of the frames of the video that are different from the frames of the video rendered by another GPU to provide the respective output. The outputs of the GPUs may be combined to render the video. In other examples, each GPU is programmed to render some but not all of the rows of the frames of the video that are different from the rows of the frames rendered by another GPU to provide the respective output. The outputs of the GPUs may be combined to render the video.

[0010] In an exemplary technique, the first GPU includes at least one scanout unit pointing to at least one buffer managed by the second GPU. The first GPU can be programmed to cycle through the buffer to output a complete sequence of frames of the video. In another example, the first GPU includes at least one scanout unit pointing only to the buffer managed by the first GPU, and is programmed to receive frames of the video from the second GPU via direct memory access (DMA) to output a complete sequence of frames of the video.

[0011] In another exemplary technique, the first GPU includes at least one scanout unit that points to at least a first buffer managed by the first GPU and a second buffer managed by the second GPU. In this technique, the first GPU is programmed to cycle through the buffers to output a complete sequence of frames of the video using rows 1-N associated with the first buffer and rows (N+1)-M associated with the second buffer. The rows 1-N are different rows of the same frame associated with rows (N+1)-M.

[0012] Likewise, the first GPU may include at least one scan output unit that points to at least a first buffer managed by the first GPU and does not point to a second buffer managed by the second GPU. In this implementation, the first GPU is programmed to cycle through the buffers to output a complete sequence of frames of the video using 1-N rows associated with the first buffer and (N+1)-M rows associated with the second buffer received by the first GPU via direct memory access (DMA). The 1-N rows and (N+1)-M rows are different rows of the frame of the video.

[0013] In another technique, a first GPU includes at least one scan output unit directed to at least a first buffer in communication with a common memory controller. A second GPU includes a second buffer in communication with the common memory controller. The first GPU is programmed to render 1-N rows associated with the first buffer, and the second GPU is programmed to render (N+1)-M rows associated with the second buffer.

[0014] In some examples, the first GPU manages video data output from the first GPU and the second GPU. This can be achieved by physically connecting an HDMI port to the first GPU. In other examples, the GPU outputs video data to a multiplexer that multiplexes frames and / or lines from the corresponding GPUs together to output video.

[0015] In another aspect, in a multi-graphics processing unit (GPU) simulation environment, a method includes causing a plurality of GPUs to render respective frames of a video, or respective portions of each frame of the video, or both. The method includes controlling frame output by receiving frame information from at least one other of the GPUs using a first one of the GPUs, or multiplexing the outputs of the GPUs together, or receiving frame information from at least one other of the GPUs using a first one of the GPUs and multiplexing both the outputs of the GPUs together.

[0016] In another aspect, a computer simulation device includes: at least a first graphics processing unit (GPU), at least the first graphics processing unit is programmed to render a corresponding first portion of a simulated video; and at least a second GPU, at least the second GPU is programmed to render a corresponding second portion of the simulated video. At least the first GPU is programmed to combine the first portion and the second portion and render an output that establishes a complete simulated video.

[0017] The details of the present application, both as to its structure and operation, may be best understood with reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram of an exemplary system including an example according to the principles of the present invention;

[0019] Figure 2 is a schematic diagram of a cloud-based gaming system;

[0020] Figure 3 is a block diagram of an exemplary non-uniform memory access (NUMA) architecture in which two APUs are shown on a single fabric, it being understood that the NUMA architecture may be implemented by APUs on separate fabrics and may implement more than two APUs;

[0021] Figure 4 is a block diagram of a shared memory architecture showing two APUs where each processor is implemented on its own respective die, it being understood that the architecture may be implemented on fewer dies or even one die and more than two APUs may be implemented;

[0022] Figure 5 is a block diagram of a shared memory architecture in which two APUs are shown, wherein each APU is implemented on its own respective fabric, and wherein a shared memory controller is implemented on one of the fabrics, it being understood that the architecture may be implemented on one fabric, and more than two APUs may be implemented on one or more dies;

[0023] Figure 6 is a block diagram of an exemplary GPU having a scanout unit;

[0024] Figure 7 is a flow chart of exemplary logic for a NUMA implementation in which each GPU renders an entire frame, in which each GPU renders a different frame of the same video as another GPU, in which one of the GPUs has registers pointing to a buffer of the other (other) GPU;

[0025] Figure 8is a flow chart of exemplary logic for a NUMA implementation in which each GPU renders an entire frame, in which each GPU renders a different frame of the same video as another GPU, in which one of the GPUs receives a frame from another (other) GPU via DMA;

[0026] Fig. 9 is a flow chart of exemplary logic for a NUMA implementation in which each GPU renders a portion (e.g., a row) of a frame, wherein each GPU renders a different portion of the same frame as another GPU;

[0027] Fig.10 is a flow diagram of exemplary logic for a NUMA implementation in which each GPU renders a portion (e.g., a row) of a frame, in which each GPU renders a different portion of the same frame as another GPU, in which one of the GPUs receives the row from the other (other) GPU via DMA;

[0028] Fig.11 is a flow chart of exemplary logic for a shared memory implementation in which each GPU renders a portion (e.g., a row) of a frame, wherein each GPU renders a different portion of the same frame as another GPU;

[0029] Fig.12 is a flow chart of exemplary logic for controlling video output using a single GPU connected to an HDMI port;

[0030] Fig.13 is a flow chart of exemplary logic for controlling video output using a multiplexer; and

[0031] Fig.14 is with Fig.13 Associated block diagram. DETAILED DESCRIPTION

[0032] The present disclosure generally relates to a computer ecosystem that includes aspects of a network of consumer electronics (CE) devices, such as, but not limited to, a distributed computer game network, video broadcasting, a content delivery network, virtual machines, and machine learning applications. The system herein may include server and client components connected via a network so that data can be exchanged between the client and server components. The client component may include one or more computing devices, including, for example, a Sony The client devices may be game consoles and related motherboards, portable televisions (e.g., smart TVs, TVs with Internet capabilities), portable computers such as laptops and tablet computers, and other mobile devices including smart phones and additional examples discussed below. These client devices may operate in a variety of operating environments. For example, some client computers may use (for example) Orbis or Linux operating systems, operating systems from Microsoft or Unix operating systems, or operating systems produced by Apple Computer or Google. These operating environments may be used to execute one or more browsing programs, such as browsers made by Microsoft or Google or Mozilla, or other browser programs that can access websites hosted by Internet servers discussed below. Moreover, an operating environment according to the principles of the present invention may be used to execute one or more computer game programs.

[0033] The server and / or gateway may include one or more processors that execute instructions that configure the server to receive and transmit data via a network such as the Internet. Alternatively, the client and server may be connected via a local intranet or a virtual private network. The server or controller may be instantiated by a game console and / or its motherboard(s), such as a Sony Personal computer, etc.

[0034] Information may be exchanged between the client and the server via the network. To this end, and for security purposes, the server and / or client may include firewalls, load balancers, temporary storage devices and proxies, and other network infrastructure for reliability and security. One or more servers may form an apparatus that implements a method of providing a secure community (such as an online social networking site) to network members.

[0035] As used herein, instructions refer to computer-implemented steps for processing information in a system. Instructions may be implemented in software, firmware, or hardware, and include any type of programmed steps undertaken by components of the system.

[0036] The processor may be any conventional general purpose single or multi-chip processor that can execute logic through various lines such as address, data and control lines, as well as registers and shift registers.

[0037] The software modules described by the flowcharts and user interfaces herein may include various subroutines, programs, etc. Without limiting the present disclosure, the logic stated as being performed by a particular module may be redistributed to other software modules and / or combined together in a single module and / or made available in a shareable library.

[0038] The inventive principles described herein may be implemented as hardware, software, firmware, or a combination thereof; therefore, the illustrative components, blocks, modules, circuits, and steps are described in terms of their functionality.

[0039] In addition to what has been mentioned above, the logical blocks, modules and circuits described below may be implemented or executed with a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic devices designed to perform the functions described herein (such as application specific integrated circuits (ASICs), discrete gate or transistor logic, discrete hardware components or any combination thereof). The processor may be implemented by a controller or a state machine or a combination of computing devices.

[0040] The functions and methods described below when implemented in software may be written in an appropriate language such as, but not limited to, Java, C#, or C++, and may be stored on or transmitted through a computer-readable storage medium, such as a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM), or other optical disk storage devices (such as digital versatile disks (DVDs)), magnetic disk storage devices, or other magnetic storage devices including removable thumb drives, etc. A connection may establish a computer-readable medium. Such connections may include, for example, hardwired cables, including optical fiber and coaxial cables, as well as digital subscriber lines (DSL) and twisted pairs. Such connections may include wireless communication connections, including infrared and radio.

[0041] The components included in one embodiment may be used in other embodiments in any appropriate combination. For example, any of the various components described herein and / or depicted in the figures may be combined, interchanged, or excluded from other embodiments.

[0042] “A system having at least one of A, B, and C” (similarly, “a system having at least one of A, B, or C” and “a system having at least one of A, B, C”) includes the following systems: having only A; having only B; having only C; having both A and B; having both A and C; having both B and C; and / or having A, B, and C, etc.

[0043] Now specifically refer to Figure 1, an exemplary system 10 is shown, which may include one or more of the exemplary devices mentioned above and further described below according to the principles of the present invention. The first of the exemplary devices included in the system 10 is a consumer electronics (CE) device such as an audio video device (AVD) 12, such as but not limited to an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). However, the AVD 12 may alternatively be an appliance or household item, such as a computerized Internet-enabled refrigerator, washing machine, or drying machine. Alternatively, the AVD 12 may also be a computerized Internet-enabled ("smart") phone, a tablet computer, a notebook computer, a wearable computerized device (such as a computerized Internet-enabled watch, a computerized Internet-enabled bracelet), other computerized Internet-enabled devices, a computerized Internet-enabled music player, a computerized Internet-enabled headset, a computerized Internet-enabled implantable device (such as an implantable skin device), etc. Regardless, it should be understood that the AVD 12 is configured to employ the principles of the present invention (eg, communicate with other CE devices to employ the principles of the present invention, execute the logic described herein, and perform any other functions and / or operations described herein).

[0044] Therefore, in order to adopt such a principle, the AVD 12 can be composed of Figure 1 . For example, the AVD 12 may include one or more displays 14, which may be implemented by a flat screen of high definition or ultra high definition ("4K") or higher, and may be touch-enabled for receiving user input signals via touch on the display. The AVD 12 may include: one or more speakers 16 for outputting audio according to the present principles; and at least one additional input device 18 (such as an audio receiver / microphone) for inputting audible commands to the AVD 12, for example, to control the AVD 12. The exemplary AVD 12 may also include one or more network interfaces 20 for communicating via at least one network 22 (such as the Internet, WAN, LAN, etc.) under the control of one or more processors 24 (e.g., one or more central processing units (CPUs), graphics processing units (GPUs), and combinations thereof). It should be noted that a processing chip that acts as a central processing unit (CPU) and a graphics processing unit (GPU) on a single die may be referred to herein as an accelerated processing unit (APU).

[0045] Interface 20 may be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It will be understood that processor 24 controls AVD 12 to employ the principles of the present invention, including other elements of AVD 12 described herein, such as controlling display 14 to present images on the display and receiving input from the display. In addition, it should be noted that network interface 20 may be, for example, a wired or wireless modem or router or other suitable interface (such as a wireless telephone transceiver, or a Wi-Fi transceiver as mentioned above, etc.).

[0046] In addition to the foregoing, the AVD 12 may also include one or more input ports 26, such as a high-definition multimedia interface (HDMI) port or a USB port for physically connecting (e.g., using a wired connection) to another CE device and / or a headphone port for connecting headphones to the AVD 12 for presenting the audio from the AVD 12 to the user through the headphones. For example, the input port 26 may be connected to a wired or satellite source 26a of audio-video content via a wire or wirelessly. Thus, the source 26a may be, for example, a separate or integrated set-top box or satellite receiver. Alternatively, the source 26a may be a game console or disk player containing content that may be viewed by the user as a collection for the channel assignment purpose described further below. The source 26a may include some or all of the components described below with respect to the CE device 44 when implemented as a game console.

[0047] The AVD 12 may also include one or more computer memories 28 that are not transient signals, such as disk-based storage devices or solid-state storage devices, which are in some cases embodied as independent devices in the housing of the AVD, or as personal video recorders (PVRs) or video disk players for playing back AV programs inside or outside the housing of the AVD, or as removable storage media. In addition, in some embodiments, the AVD 12 may include a position or positioning receiver (such as, but not limited to, a cell phone receiver, a GPS receiver, and / or an altimeter 30), which is configured to receive geographic location information from, for example, at least one satellite or cell phone tower and provide the information to the processor 24 and / or determine the altitude at which the AVD 12 is located in conjunction with the processor 24. However, it will be understood that according to the principles of the present invention, another suitable position receiver other than a cell phone receiver, a GPS receiver, and / or an altimeter may be used to (for example) determine the location of the AVD 12 in (for example) all three dimensions.

[0048] Continuing the description of AVD 12, in some embodiments, according to the principles of the present invention, AVD 12 may include one or more cameras 32, which may be, for example, thermal imaging cameras, digital cameras (such as webcams), and / or cameras integrated into AVD 12 and capable of being controlled by processor 24 to collect pictures / images and / or videos. AVD 12 may also include a Bluetooth transceiver 34 and other near field communication (NFC) elements 36 for communicating with other devices using Bluetooth and / or NFC technology, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.

[0049] In addition, the AVD 12 may include one or more auxiliary sensors 37 (e.g., motion sensors such as accelerometers, gyroscopes, odometers or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or rhythm sensors, gesture sensors (e.g., for sensing gesture commands), etc.) that provide input to the processor 24. The AVD 12 may include a wireless TV broadcast port 38 for receiving OTH TV broadcasts that provide input to the processor 24. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering the AVD 12.

[0050] Still refer to Figure 1 In addition to the AVD 12, the system 10 may also include one or more other CE device types. In one example, a first CE device 44 may be used to control a display via commands sent by a server described below, while a second CE device 46 may include components similar to the first CE device 44 and therefore will not be discussed in detail. In the example shown, only two CE devices 44, 46 are shown, and it should be understood that fewer or larger devices may be used. As described above, the CE devices 44 / 46 and / or the source 26a may be implemented by a game console. Alternatively, one or more of the CE devices 44 / 46 may be implemented by devices sold under the trademarks Google Chromecast, Roku, Amazon FireTV.

[0051] In the example shown, for the purpose of illustrating the present invention, it is assumed that all three devices 12, 44, 46 are members of, for example, a home entertainment network, or at least are close to each other in a location such as a house. However, the present invention is not limited to the specific location shown by the dashed line 48 unless otherwise expressly required.

[0052] An exemplary non-limiting first CE device 44 may be established by any of the above devices (e.g., a portable wireless laptop or notebook computer or a game controller (also referred to as a "console")), and may therefore have one or more of the components described below. The second CE device 46 may be established by a video disk player such as a Blu-ray player, a game console, etc., without limitation. The first CE device 44 may be a remote control device (RC) for, for example, issuing AV play and pause commands to the AVD 12, or it may be a more complex device such as a tablet computer, a game controller that communicates with a game console implemented by the second CE device 46 via a wired or wireless link and controls the presentation of a video game on the AVD 12, a personal computer, a wireless phone, etc.

[0053] Thus, the first CE device 44 may include one or more displays 50 that may have touch functionality for receiving user input signals via touch on the display. The first CE device 44 may include: one or more speakers 52 for outputting audio according to the principles of the present invention; and at least one additional input device 54 (such as an audio receiver / microphone) for inputting audible commands to the first CE device 44 to control the device 44, for example. The exemplary first CE device 44 may also include one or more network interfaces 56 for communicating via the network 22 under the control of one or more CE device processors 58 (such as one or more CPUs, GPUs, and combinations thereof). Thus, the interface 56 may be (without limitation) a Wi-Fi transceiver, which is an example of a wireless computer network interface, including a mesh network interface. It should be understood that the processor 58 controls the first CE device 44 to employ the principles of the present invention, including other elements of the first CE device 44 described herein, such as controlling the display 50 to present images on the display and receiving input from the display. Furthermore, it should be noted that the network interface 56 may be, for example, a wired or wireless modem or router or other suitable interface (such as a wireless telephone transceiver, or a Wi-Fi transceiver as mentioned above, etc.).

[0054] In addition to the foregoing, the first CE device 44 may also include one or more input ports 60 (such as an HDMI port or a USB port) for physically connecting (e.g., using a wired connection) to another CE device and / or a headphone port for connecting headphones to the first CE device 44 for presenting audio from the first CE device 44 to the user through the headphones. The first CE device 44 may also include one or more tangible computer-readable storage media 62, such as a disk-based storage device or a solid-state storage device. Moreover, in some embodiments, the first CE device 44 may include a location or positioning receiver (such as, but not limited to, a cell phone and / or a GPS receiver and / or an altimeter 64), which is configured to (for example) receive geographic location information from at least one satellite and / or cell tower using triangulation, and provide the information to the CE device processor 58 and / or determine the altitude at which the first CE device 44 is located in conjunction with the CE device processor 58. However, it should be understood that another suitable location receiver other than a cell phone and / or a GPS receiver and / or an altimeter may be used, for example, to determine the location of the first CE device 44 in, for example, all three dimensions, in accordance with the principles of the present invention.

[0055] Continuing with the description of the first CE device 44, in some embodiments, in accordance with the principles of the present invention, the first CE device 44 may include one or more cameras 66, which may be, for example, thermal imaging cameras, digital cameras (such as webcams), and / or cameras integrated into the first CE device 44 and capable of being controlled by the CE device processor 58 to collect pictures / images and / or videos. A Bluetooth transceiver 68 and other near field communication (NFC) elements 70 may also be included on the first CE device 44 for communicating with other devices using Bluetooth and / or NFC technology, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.

[0056] In addition, the first CE device 44 may include one or more auxiliary sensors 72 (e.g., motion sensors such as accelerometers, gyroscopes, odometers or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., for sensing gesture commands), etc.) that provide input to the CE device processor 58. The first CE device 44 may include other sensors that provide input to the CE device processor 58, such as one or more climate sensors 74 (e.g., barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and / or one or more biometric sensors 76. In addition to the foregoing, it should be noted that in some embodiments, the first CE device 44 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 78, such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering the first CE device 44. The CE device 44 may communicate with the AVD 12 via any of the above-mentioned communication modes and related components.

[0057] The second CE device 46 may include some or all of the components shown for the CE device 44. Either or both CE devices may be powered by one or more batteries.

[0058] Referring now to the aforementioned at least one server 80, the at least one server includes at least one server processor 82, at least one tangible computer-readable storage medium 84 (such as a disk-based storage device or a solid-state storage device). In one implementation, the medium 84 includes one or more SSDs. In accordance with the principles of the present invention, the server also includes at least one network interface 86, the at least one network interface allowing communication with the server via the network 22. Figure 1 The server 80 may be configured to communicate with other devices of the server and may actually facilitate communication between the server and the client devices. It should be noted that the network interface 86 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface (such as a wireless telephone transceiver). The network interface 86 may be a remote direct memory access (RDMA) interface that directly connects the medium 84 to a network such as a so-called "fabric" without passing through the server processor 82. The network may include an Ethernet network and / or a Fibre Channel network and / or an Infiniband network. Typically, the server 80 includes multiple processors in multiple computers, which are called "blades" that can be arranged into a physical server "stack".

[0059] Thus, in some embodiments, server 80 may be an Internet server or an entire "server farm" and may include and perform "cloud" functions, such that the devices of system 10 may access the "cloud" environment via server 80 in an example embodiment of, for example, a network gaming application. Alternatively, server 80 may be provided by a server that is associated with a server. Figure 1The other devices shown may be implemented in the same room or nearby one or more game consoles or other computers.

[0060] The methods herein may be implemented as software instructions executed by a processor, a suitably configured application specific integrated circuit (ASIC) or field programmable gate array (FPGA) module, or any other convenient means as will be appreciated by those skilled in the art. Where employed, the software instructions may be embodied in a non-transitory device such as a CD ROM or flash drive. The software code instructions may alternatively be embodied as a transient arrangement such as a radio or optical signal, or via download over the Internet.

[0061] Figure 2 An exemplary application in the form of a cloud computer gaming environment is depicted, wherein one or more client game consoles 200 (also referred to as "gaming systems", "gaming devices") (such as Sony Microsoft The management server 202 communicates with a cloud-based game management server 202 (typically an Internet server) via wired and / or wireless links. In turn, the management server 202 communicates with a first game server 204 (which may be embodied by multiple server "blades"), which includes one or more solid-state memories 206, such as random access memory (RAM), including NVMe-enabled SSDs using solid-state memory, such as (but not limited to) flash or 3D Xpoint. The management server 202 communicates with up to "N" such servers, including an Nth game server 212 that includes one or more solid-state memories 214.

[0062] Example Architecture

[0063] Figure 3 An exemplary non-uniform memory access (NUMA) architecture is depicted, wherein a single fabric 300 holds two APUs 302, 304 on a single die or on respective dies, it being understood that the NUMA architecture may be implemented by more than two APUs. When implemented on respective die chips on the same fabric 300, a communication path may be established through the fabric's via layer, which may generally be referred to as a "bus".

[0064] As shown, each APU may include one or more CPUs 304 and one or more GPUs 306, typically one CPU and one GPU per APU. Each APU 302 may be associated with its own respective memory controller 308, which controls access to memory 310, such as random access memory (RAM). Communication between the APUs may be effected by one or more communication paths 312, which are referred to herein for convenience as "buses".

[0065] Therefore, each APU (or separate GPU) has its own memory controller and therefore its own dedicated memory, such as RAM. There may be a (cache-coherent) shared bus between the GPUs, allowing one GPU to access the memory of another GPU.

[0066] Figure 4 4 is a block diagram of a shared memory architecture in which two APUs are shown, each APU including a CPU 400 and a GPU 402, wherein each CPU and each GPU is implemented on its own respective die, it being understood that the architecture may be implemented on fewer dies or even one die and more than two APUs may be implemented. The APUs share a common memory controller 404 that controls memory 406, and the APUs may communicate with each other and with the memory controller via respective communication paths.

[0067] Figure 5 5 is a block diagram of a shared memory architecture, in which two APUs are shown (each APU includes a respective CPU 502 and GPU 504), wherein each APU is implemented on its own respective die 500, and wherein a shared memory controller 506 is implemented on one of the dies 500, it being understood that the architecture may be implemented on one die, and more than two APUs may be implemented. The shared memory controller 506 controls access to a memory 508, and the APUs may communicate with each other and with the memory controller 506 via one or more communication paths 510.

[0068] Figure 6 6 is a block diagram of an exemplary GPU 600 having a scanout unit 602. The scanout unit 602 may include a plurality of registers 604 pointing to corresponding memory buffers (or equivalently, buffer IDs) 606. A video encoder 608 may communicate with the scanout unit 602. The video encoder 608 is particularly useful in cloud gaming scenarios to allow encoding of the same images that would normally be output over HDMI.

[0069] Scanout unit 602 is responsible for outputting the pixels of each frame of video line by line, for example, to HDMI. As discussed more fully below, the scanout unit can be programmed to read the correct video frame. To this end, a set of registers 604 is maintained, each of which points to a different buffer 606, and wherein the scanout unit cycles through the buffers.

[0070] Framebuffer management

[0071] As will be discussed more fully herein, there are a variety of approaches to how multiple GPUs can work together to manage a frame buffer. Each GPU may render a different frame of video than another GPU. Alternatively, each GPU may render a different portion of the same frame, e.g., the top 1 to N rows of a frame may be rendered by a first GPU, and the bottom n+1 to M rows of the same frame may be rendered by a second GPU. Other patterns / portions between GPUs may be used.

[0072] Figure 7 7 is a flow chart of exemplary logic for a NUMA implementation, wherein each GPU renders an entire frame, wherein each GPU renders a different frame of the same video as another GPU, wherein one of the GPUs has a register pointing to a buffer of another (other) GPU. Beginning at block 700, an operating system and / or GPU driver assigns a memory area to be used as a frame buffer. Typically, a frame buffer consists of at least two buffers (more buffers increase latency). One buffer is used for (e.g.) a currently displayed frame to be output via HDMI, while a second buffer can be used to render the next frame. Additional depth buffers and other buffers may be present if desired.

[0073] exist Figure 7 In the example of FIG. 1 , a GPU driver and / or a simulation program (e.g., a computer game) sends rendering commands to alternate between the GPUs. The GPU driver or game manages this by commanding the first GPU to render every odd-numbered frame and commanding the second GPU to render every even-numbered frame.

[0074] In this implementation, the logic may move to block 702 to program the registers of the scan output unit so that each register points to a memory buffer managed by a different GPU. Proceeding to block 704, the GPU cycles through the buffers it manages and the buffers managed by other GPUs pointed to by the cycled GPU's registers to output all frames of the video, which may be in HDMI. It will be appreciated that when more than two GPUs are used, the number of frames rendered by each GPU may be reduced accordingly, for example, each of the N GPUs may render 1 / N frames of the video, with each GPU rendering a different frame than the frames rendered by the other GPUs.

[0075] Figure 8 An alternative approach is shown in which the scanout unit scans out frames only from the memory of the "local" GPU. Figure 7 As in the case of Figure 8 At block 800 in FIG. 8 , the operating system and / or GPU driver designates a memory region to be used as a frame buffer. However, moving to block 802, the registers of the first GPU are programmed to point only to a buffer local to that GPU, where the frame from the second GPU is copied to the first GPU via direct memory access (DMA) when rendering of the frame is completed at block 804. It should be noted that the "first" GPU can be established by the first GPU copying the frame based on an interrupt from the second GPU (notifying that the frame is complete). Proceeding to block 806, the first GPU cycles through the buffers it manages and the frames received from the second GPU via DMA at block 804 to output all frames of the video, which can be in HDMI.

[0076] It should be noted that in Figure 4 and Figure 5 In the shared memory controller architecture shown in , there is no need to copy frames because there is no problem of scan output units reading data. The timing is the same regardless of which GPU is rendering.

[0077] Fig. 9 9 is a flow diagram of exemplary logic for a NUMA implementation in which each GPU renders a portion of a frame (e.g., a row), wherein each GPU renders a different portion of the same frame as another GPU. Beginning at block 900, a first GPU renders a first portion of a frame, such as the first N rows (rows 1 to N), and at block 902, a second GPU renders a different portion of the same frame, such as rows N+1 to M (the last row). It will be appreciated that when more than two GPUs are used, the portion of the frame rendered by each GPU may be correspondingly reduced. In any case, at block 904, the entire frame (rows 1 to M) is output by the first GPU.

[0078] To achieve the above, the scanout unit can be modified to read from multiple buffers for each frame, each buffer being managed by a different GPU. The scanout unit can thus be programmed to generate the first "N" lines from a first buffer (which can be its own internal rendering), and the next N lines from a second buffer which can be associated with a second GPU.

[0079] Fig.10 Draws the Fig. 9Another alternative method is similar to the method of , except that the second GPU DMAs memory through a portion of the video buffer of the first GPU. Therefore, at block 1000, the first GPU renders a first portion of a frame, such as the first N rows (rows 1 to N), and at block 1002, the first GPU receives a different portion of the same frame from the second GPU via DMA, for example, rows N+1 to M (the last row). At block 1004, the entire frame (rows 1 to M) is output by the first GPU.

[0080] Fig.11 1 is a flow chart of exemplary logic for a shared memory implementation in which each GPU renders a portion of a frame (e.g., multiple rows), wherein each GPU renders a different portion of the same frame as another GPU. Thus, at block 1100, a first GPU renders a first portion of a frame to a buffer, and at block 1102, a second GPU renders a second portion of the same frame to the same buffer. At block 1104, the entire frame (rows 1 to M) is output by the shared buffer.

[0081] Determine which GPU controls HDMI output

[0082] Fig.12 A first method for determining which GPU manages output is illustrated including physically connecting only the HDMI (or DisplayPort) output to a specific GPU at manufacturing time, at block 1200. The mapping is thus controlled at manufacturing time.

[0083] Fig.13 and Fig.14 In another method, Fig.13 At block 1300 in FIG. 1 , each GPU 1400 (in Fig.14 1400) is implemented with its own corresponding video output. At block 1302, the output of GPU 1400 is multiplexed by one or more multiplexers 1402 that switch between two GPU output ports.

[0084] Recognizing that signals are often encrypted, an encryption chip 1404 may be provided to receive the output of the multiplexer 1402 for encryption. In essence, the multiplexed output may create a DisplayPort signal, which the encryption chip 1404 converts to HDMI.

[0085] Video synthesis

[0086] As understood herein, both user experience (UX) graphics and simulated (e.g., game) video may be rendered across different GPUs. Given that UX rendering is generally less demanding, only a single GPU needs to render the UX, typically the GPU selected to also handle the HDMI output in the previous section. This GPU composites the final frame buffer image to include the UX and the game. The game, its frame buffer may depend on the frame. The composition engine may read the memory directly from each GPU's memory or from a shared memory controller.

[0087] Power Management

[0088] Power management techniques can be implemented to reduce thermal load by constraining power consumption. Recognizing that power consumption changes linearly with frequency and changes with the square of voltage, computer simulation programs such as video games can be programmed to be responsible for maintaining power consumption within predetermined thresholds by automatically reducing frequency and / or voltage when approaching frequency / voltage / power thresholds. To this end, registers from hardware such as one or more GPUs can be read to determine the current usage allocation, thereby suppressing certain effects, such as particle effects, when necessary. The same principle can also be applied to mobile phones. Suppression can be implemented through over-limit technology, and the GPU can be suppressed independently of the CPU in the architecture. The resolution of the video can be reduced to maintain the simulation execution while remaining within the power consumption related threshold. Audio and / or visual alarms (such as activating LEDs) can be presented when approaching power consumption related thresholds.

[0089] Users may be permitted to pay extra for additional thermal budget. Similarly, users may be allocated more die (and therefore more APUs) on a cloud server by paying an additional fee, where less paying users are allocated only a single die. This may be done when an application begins programming an API to call system metrics and spawn threads and determine quality of service based on the metrics. System metrics may be filtered for less paying users who are allocated fewer die. More paying users who want the benefits of multi-threaded gaming with simultaneous processing may be allocated more die than less paying users.

[0090] It will be appreciated that while the principles of the invention have been described with reference to certain exemplary embodiments, these embodiments are not intended to be limiting and that various alternative arrangements may be used to implement the subject matter claimed herein.

Claims

1. A device, comprising: at least a first graphics processing unit GPU; at least a second GPU, at least the second GPU being communicatively coupled to the first GPU; The GPU is programmed to: rendering respective portions of a video such that the first GPU renders a first portion of the video and the second GPU renders a second portion of the video, the first portion and the second portion being different from each other; as well as The first GPU includes at least one scan output unit, wherein the at least one scan output unit includes multiple registers pointing to respective memory buffers, each register of the at least one scan output unit points to a different memory buffer so that the at least one scan output unit can cycle through the memory buffers.

2. The apparatus of claim 1, wherein the first GPU and the second GPU are implemented on a common die.

3. The apparatus of claim 1, wherein the first GPU and the second GPU are implemented on respective first and second dies.

4. The apparatus of claim 1, wherein the first GPU is associated with a first central processing unit (CPU), and the second GPU is associated with a second CPU.

5. The device of claim 1, comprising a first memory controller and a first memory associated with the first GPU, and a second memory controller and a second memory associated with the second GPU.

6. The apparatus of claim 1, wherein the GPUs share a common memory controller that controls a common memory.

7. The apparatus of claim 1, wherein each GPU is programmed to render some but not all of the frames of a video different from those of the video rendered by another GPU to provide a respective output, the outputs of the GPUs being combined to render the video.

8. The apparatus of claim 1 , wherein each GPU is programmed to render some but not all of all rows of a frame of video, the rows of a frame of video rendered by one GPU being different from the rows of the frame rendered by another GPU to provide respective outputs, the outputs of the GPUs being combined to render the video.

9. The apparatus of claim 7, wherein the at least one scanout unit points to at least one buffer managed by the second GPU, the first GPU being programmed to cycle through the buffers to output a complete sequence of frames of the video.

10. The apparatus of claim 7, wherein the at least one scanout unit points only to a buffer managed by the first GPU, the first GPU being programmed to receive frames of the video from the second GPU via direct memory access (DMA) and to output a complete sequence of frames of the video.

11. The apparatus of claim 1 , wherein the at least one scan output unit points to at least a first buffer managed by the first GPU and a second buffer managed by the second GPU, the first GPU being programmed to loop through the buffers to output a complete sequence of frames of a video using 1-N rows associated with the first buffer and (N+1)-M rows associated with the second buffer, the 1-N rows and (N+1)-M rows being different rows of the frames of the video.

12. The apparatus of claim 1 , wherein the at least one scan output unit points to at least a first buffer managed by the first GPU and does not point to a second buffer managed by the second GPU, the first GPU being programmed to loop through the buffers to output a complete sequence of frames of a video using 1-N rows associated with the first buffer and (N+1)-M rows associated with the second buffer received by the first GPU via direct memory access (DMA), the 1-N rows and (N+1)-M rows being different rows of the frames of the video.

13. The apparatus of claim 6, wherein the at least one scan output unit points to at least a first buffer in communication with the common memory controller, the second GPU includes a second buffer in communication with the common memory controller, the first GPU renders 1-N lines associated with the first buffer and the second GPU renders (N+1)-M lines associated with the second buffer, the 1-N lines and (N+1)-M lines being different lines of the frame of the video.

14. The apparatus of claim 1, wherein the first GPU manages video data output from the first GPU and the second GPU.

15. The apparatus of claim 1, wherein the GPU outputs video data to a multiplexer that multiplexes the frames and / or the lines from the corresponding GPU together to output video.

16. A method in a multi-graphics processing unit (GPU) simulation environment, the method comprising: causing the plurality of GPUs to render corresponding frames of the video or corresponding portions of each frame of the video or corresponding frames and corresponding portions of the frames of the video; Frame output is controlled by: using a first one of the GPUs to receive frame information from at least one other one of the GPUs, or multiplexing the outputs of the GPUs together, or both using the first one of the GPUs to receive frame information from at least one other one of the GPUs and multiplexing the outputs of the GPUs together, wherein the first one of the GPUs combines respective video components from the GPUs to render a combined video, the combined video being rendered by suppressing at least one effect based on a current usage allocation using one or more overrun techniques, wherein suppressing at least one effect includes suppressing at least one particle effect.

17. The method of claim 16, wherein the first of the GPUs also manages a High Definition Multimedia Interface (HDMI) output of the final frame buffer image.

Images