Sharing of graphical memory

Abstract

The computer implementation method of this disclosure includes the step of instantiating a memory management process that communicates with graphics processing hardware elements to control the use of shared memory by a number of different application instances. Furthermore, the memory management process receives a request from an application instance indicating that content associated with a specific resource implemented by the application instance should be stored in shared memory, determines that the content specified in the request has been previously stored at a specific location in shared memory, identifies a time span during which the specified content stored in shared memory will be immutable, and instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span. Various other methods, systems, and computer-readable media are also disclosed.

Description

Cross - reference to related applications

[0001] This application claims the benefit of priority of U.S. Patent Application No. 18 / 174,438, filed on February 24, 2023, which claims the priority of U.S. Provisional Patent Application No. 63 / 481,751, filed on January 26, 2023, the disclosures of which are hereby incorporated by reference in their entirety.

Technical Field

[0002] The present invention relates to the sharing of graphical memory.

Background Art

[0003] The popularity of cloud gaming is increasing. In a cloud gaming scenario, video game data (including graphics) is mainly rendered on a remote server and then transmitted to a gaming client, where the data is decoded and displayed on an electronic device. When processing this video game data, the remote server receives requests from a video game application and stores the data in a memory location that is easily accessible, typically referred to as random access memory (RAM) or video RAM (VRAM).

Summary of the Invention

Problems to be Solved by the Invention

[0004] VRAM is very fast but also expensive, and thus has limited capacity. When thousands or millions of video game sessions are simultaneously executed on a cloud server, the competition for access to this VRAM becomes high. Furthermore, simply reusing memory that may be duplicated among applications becomes a problem when considering very specialized graphics processing that can access content asynchronously and substantially without warning at any time.

Means for Solving the Problems

[0005] As will be described in great detail below, this disclosure describes a method and system for sharing memory among multiple application instances. In one example, a computer implementation includes the step of instantiating a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by multiple different application instances. These application instances may belong to a single software application or to multiple different applications or processes. The memory management process receives a request from at least one of the application instances indicating that it wants to store content associated with a specific resource implemented by the application instance in shared memory, determines that the content specified in the request has been previously stored at a specific location in shared memory, identifies a time span during which the specified content stored in shared memory will be immutable, and further instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span.

[0006] In some embodiments, the memory management process instructs the requesting application instance to access the specified content from a specific location in shared memory during a specified time span, instead of storing the specified content associated with a particular resource in shared memory.

[0007] In some cases, the identified content stored in shared memory is monitored by memory management processes. In some cases, the memory management processes are embedded in an application processing package file associated with the application instance that sent the request.

[0008] In some embodiments, the memory management process determines whether a particular resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from the application processing package file. In some cases, the application processing package file includes a game engine for video games. In other cases, the memory management process is loaded dynamically along with the application instance that submitted the request. In some examples, the memory management process is loaded dynamically using a Vulkan layer that enables dynamic interception of graphics application programming interface (API) calls.

[0009] In some cases, the memory management process determines whether a particular resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from at least one intercepted API call. In some cases, at least one of the resource identifiers obtained from the intercepted API call is a universally unique identifier (UUID). In some cases, the intercepted API call increments the reference count and further returns a shared file descriptor that points to the memory backing previously allocated to the resource. In some embodiments, the application instance uses the shared file descriptor when accessing a particular resource.

[0010] Furthermore, the corresponding system includes at least one physical processing unit and physical memory containing a computer executable instruction, which, when executed by the physical processing unit, causes the physical processing unit to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances. The memory management process receives a request from at least one of the application instances to store content associated with a specific resource implemented by the application instance in shared memory, determines that the content specified in the request has been previously stored at a specific location in shared memory, identifies a time span during which the specified content stored in shared memory will be immutable, and instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span.

[0011] In some embodiments, the memory management process preloads one or more resources for the application instance that sent the request. In some cases, after determining that a particular resource is no longer being used by a number of application instances, the memory management process stores the particular resource in shared memory for at least a certain length of time. In some examples, a particular resource is stored in shared memory for a certain length of time based on a determined amount of churn related to that particular resource.

[0012] In some cases, content identified as mutable is shared in a pool among multiple different application instances by the memory management process. In some embodiments, content identified as mutable is associated with a fence synchronization object used to track when an asynchronous task performed using the content has completed. The fence synchronization object is configured to protect against concurrent or non-exclusive use of the mutable content. In some examples, the memory management process redistributes the content identified as mutable to the pool once the asynchronous task is completed.

[0013] In some examples, the above method is encoded as a computer-readable instruction on a computer-readable medium. For example, the computer-readable medium contains one or more computer-executable instructions, which, when executed by at least one processing unit of a computer, cause the computer to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances. The memory management process receives a request from at least one of the application instances to store content associated with a specific resource implemented by the application instance in shared memory, determines that the content specified in the request has been previously stored at a specific location in shared memory, identifies a time span during which the specified content stored in shared memory will be immutable, and instructs the requesting application instance to access the specified content from that specific location in shared memory during the specified time span.

[0014] Features of any embodiment described herein may be used in combination with each other according to the general principles described herein. The embodiments, features, and advantages described herein and other herein will be better understood by reading the following detailed description together with the accompanying drawings and claims.

[0015] The accompanying drawings illustrate numerous exemplary embodiments and are part of this specification. These drawings, together with the following description, illustrate various principles of this disclosure.

[0016] Throughout the drawings, the same reference numerals and descriptions indicate similar components, but they are not necessarily identical components. While various modifications and alternative forms are possible for the exemplary embodiments described herein, specific embodiments are illustrated in the drawings and described in detail herein. Nevertheless, the exemplary embodiments described herein are not intended to be limited to any particular form disclosed. Rather, this disclosure covers all modifications, equivalents, and alternatives that fall within the scope of the appended claims. [Brief explanation of the drawing]

[0017] [Figure 1] This specification shows a computing environment designed for the embodiments described herein to operate. [Figure 2] This is a flowchart illustrating an exemplary method for sharing memory among multiple application instances. [Figure 3] This specification shows an alternative computing environment designed for the embodiments described herein to operate. [Figure 4A] This shows an embodiment in which different memory management processes are loaded and implemented. [Figure 4B] This shows an embodiment in which different memory management processes are loaded and implemented. [Figure 5] This illustrates an embodiment in which different application instances share a pool of variable content. [Figure 6] This is a block diagram of an exemplary content distribution ecosystem. [Figure 7] Figure 6 is a block diagram of an exemplary distributed infrastructure within the content distribution ecosystem shown. [Figure 8] Figure 6 is a block diagram of an exemplary content playback unit within the content distribution ecosystem shown in Figure 6. [Modes for carrying out the invention]

[0018] This disclosure generally relates to methods and systems for sharing video memory (i.e., Graphics Processing Unit (GPU) memory) among multiple application instances. As previously described, applications such as video games are often hosted remotely on a cloud computing system. These cloud computing systems can concurrently execute thousands or millions of gaming instances. In some cases, each of these application instances is executed separately and uses its own collection of resources or content. In other cases, some of these application instances use resources that may be identical to the resources used by other instances. For example, when multiple video game instances are running concurrently, at least some of those video game instances tend to use the same textures, the same shaders, the same buffers, or other similar types of content. This type of static content has the potential to be shared among various application instances.

[0019] However, such sharing is not straightforward because current Graphics Processing Units (GPUs) are designed to execute asynchronously and access different assets without warning at different times. GPU devices typically operate using their own hardware isolated from others. For example, each physical GPU device has its own processing unit (or set of processing cores) and allocated Video Random Access Memory (VRAM). This VRAM operates very fast and with a very high throughput. As a result, GPU devices end up with less VRAM than conventional RAM that a general-purpose processing computer would have. When multiple game instances are being run on a single GPU, as the number of concurrently running video game sessions increases, the VRAM becomes a contended resource and creates the first bottleneck.

[0020] To mitigate these issues, the embodiments described herein aim to share VRAM memory regions regardless of the asynchrony of GPUs. When running similar or identical copies of a particular video game (or other application), the systems described herein significantly reduce the VRAM required by reusing or deduplicating the same region of memory across those instances. These systems establish memory management processes that communicate directly with GPU hardware elements and APIs to identify which portions of the content are immutable (non-mutable) and which portions of the content overlap (are duplicable). For those portions of the immutable and overlapping content, the memory management process can perform deduplication to free the corresponding memory regions. For those portions of the content that can overlap when copied into memory by a video game instance, the memory management process determines which content can overlap and, instead of loading that content into memory, reuses the content already stored in memory.

[0021] The embodiments described herein also provide methods and systems for pooling VRAM resources for common processing even when the data in memory is mutable. For memory regions that are not identical across game instances but whose content is needed only temporarily or momentarily to render video frames of a game, the embodiments described herein reallocate those memory regions in an exclusive and time-shared ( "pooled") manner across video game instances. Next, the systems described herein provide virtual fence objects for those pooled resources. The virtual fence objects enable those resources to be used by multiple application instances within the virtual fence for a limited time. When the virtual fence is removed, those resources are unloaded from VRAM. These embodiments are described in more detail with reference to FIGS. 1 - 8.

[0022] Figure 1 shows a computing environment 100 including a computer system 101. The computer system 101 includes software modules, embedded hardware elements such as processors, or combinations of hardware and software. The computer system 101 includes substantially any type of computing system, including local computing systems or distributed (e.g., cloud) computing systems. In some cases, the computer system 101 includes at least one processing unit 102 and at least some system memory 103. The computer system 101 includes program modules for performing a variety of different functions. Program modules include hardware-based, software-based, or combinations of hardware and software. Each program module uses computing hardware and / or software to perform specific functions, including those described herein below.

[0023] Computer system 101 includes a communication module 104 configured to communicate with other computer systems. The communication module 104 includes any wired or wireless means capable of receiving data from and / or transmitting data to other computer systems. These communication means include hardware interfaces, including Ethernet adapters, Wi-Fi adapters, and hardware radios, which include, for example, a hardware receiver 105, a hardware transmitter 106, and a combined hardware transceiver capable of both sending and receiving data. The radios may be cellular radios, Bluetooth radios, Global Positioning System (GPS) radios, or other types of radios. The communication module 104 is configured to interact with databases, mobile computing devices (such as mobile phones or tablets), and embedded or other types of computing systems.

[0024] The computer system 101 also includes a processing instantiation module 107. The processing instantiation module 107 executes, runs, loads, or otherwise operates a memory management process 108. The memory management process 108 (also referred to herein as the GPU memory server (GMS)) is a daemon or other software process that communicates with the graphics processing hardware element 120 to manage shared memory 109 (among other possible operations). The graphics processing hardware element 120 includes a graphics processing unit, VRAM or other types of random access memory, data storage (e.g., solid-state or disk-based hard drives, flash memory, etc.), a hardware control unit, a serial bus, or other hardware elements. In addition, or instead, the memory management process 108 may communicate with an operating system, kernel, firmware module, or other software system. The memory management process 108 communicates with these hardware and / or software elements, monitoring API calls, memory requests, or other inter-element communications to determine when memory can be shared.

[0025] In the embodiments described herein, content in shared memory 109 may be shared when the memory locations or memory regions are identical, when the content in those regions is consistent over time (i.e., immutable), and when the existence of the memory locations overlaps over time. Memory management processing 108 makes these determinations, including or having access to different hardware or software modules. For example, in some cases, receiving module 111 receives a memory request 123 from application instance 122, which may be one of several different application instances 121. Each of these application instances may be part of a video game session for a different user. While application instances 121 are often described herein in conjunction with video games, these embodiments should be understood to be applicable to substantially any type of software application and not limited to video games or graphics processing hardware implementations.

[0026] A memory request 123 sent by application instance 122 may include one indicating that specific content (e.g., content 113) should be stored in shared memory 109 (e.g., VRAM). The content may include any kind of digital data that should be stored in shared memory, at least temporarily. This may include static assets such as textures, mesh vertex buffers, shader programs, or other types of static content. In addition, or instead, content 113 may include variable assets such as collectible items, skins, graphical user interface (GUI) data, or other variable content that should be stored in shared memory 109.

[0027] Upon receiving this memory request 123, the decision module 112 determines whether the content 113 is already stored in the shared memory 109. The identification module 114 determines whether the content 113 is variable over time (or will become variable), and if it is variable, it identifies the time span 115 during which the content remains unchanged. During the time span 115 during which the content 113 remains unchanged or is identical to previously stored content (for example, at memory location 110), the memory management process 108 determines that the previously stored content is shareable or reusable, and instructs the instruction module 116 to send an instruction 124 to the application instance 122 indicating the memory location 110 where the previously stored content exists. Instruction 124 also indicates the time span 115 during which the data remains unchanged. Next, the application instance 122 uses the previously stored content 113 at memory location 110 when executing.

[0028] In this way, the memory management process 108 avoids storing the requested content in shared memory (for each request 123) and instead directs the application instance 122 to previously stored data. This reserves the space in shared memory 109 for other content and (at least partially) eliminates the bottleneck that previously had to be overcome by the asynchronous nature of the GPU, which required storing each part of the application content in VRAM. In some cases, this process is adapted or modified based on input 118 from user 117 (or multiple users across multiple application instances). For example, as will be described in more detail below, some data may be retained in memory even after the application instance has stopped using it because the data is frequently requested directly or indirectly by user 117's input 18. These embodiments of method 200 in Figure 2 are further described below.

[0029] Figure 2 is a flowchart of an exemplary computer implementation method 200 for sharing video memory. The steps shown in Figure 2 can be performed by any suitable computer executable code and / or a computing system including the system shown in Figure 1. In one example, each step shown in Figure 2 may have a structure that includes a number of substeps and / or represent an algorithm represented by a number of substeps, more detailed examples of which are provided below.

[0030] As shown in Figure 2, in step 210, one or more systems described herein instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances. In step 220, the memory management process receives a request from at least one application instance indicating that it wants to store content associated with a specific resource implemented by the application instance in shared memory. In step 230, the memory management process determines that the content specified in the request has been previously stored at a specific location in shared memory. In step 240, the memory management process identifies a time span during which the specified content stored in shared memory will be immutable, and in step 250, it instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span.

[0031] Therefore, in at least some embodiments, the processing instantiation module 107 of the computer system 101 in Figure 1 is configured to communicate with a graphics processing hardware element 120, which includes at least a portion of the shared memory 109, to control the use of the shared memory by a number of different application instances 121. The memory management process 108 receives requests from these application instances 121 and stores content (i.e., digital data) in the shared memory 109. Storing the content 113 in the shared memory facilitates access to the content and speeds up its loading for use by the application instances.

[0032] Instead of simply storing content 113 in shared memory 109 based on the application instance's memory request 123, the memory management process 108 determines whether existing stored content can be reused. Because the graphics processing hardware element 120 (which may be the same as or different from the processing unit 102 and memory 103 of the computer system 101) accesses shared memory 109 in an unpredictable and asynchronous manner, various other processes are performed to ensure that content already stored in shared memory 109 is reused. Alternatively, if duplicate copies already exist in shared memory, at least one of the copies is deduplicated or removed from shared memory.

[0033] Therefore, the decision module 112 determines whether the content 113 identified in the memory request 123 is already loaded into the shared memory 109. If it is loaded, the identification module 114 determines the time span 115 or time frame in which the content remains immutable within the shared memory 109. Once this time frame is determined, the instruction module 116 then generates an instruction 124 indicating the memory location 110 of the previously stored (identical) data and the time span 115 in which the data remains immutable, and sends it to the application instance (e.g., instance 122). During this time span, although for a short period, the application instance 122 may then access this content 113, and the memory management process 108 may avoid storing the requested content 113 in the shared memory 109.

[0034] As described above, in order to share the same memory area across video game (or other application) instances, the memory management process 108 determines: 1) which memory areas are identical (i.e., which memory areas contain the same digital data), 2) whether these memory areas are consistent over time (i.e., whether the content is immutable or variable over time), and 3) whether there is any temporal overlap of the same memory area.

[0035] The embodiments described herein are configured to obtain this information from the GPU's video game engine. In conventional graphics rendering scenarios, neither the GPU driver associated with the GPU, nor even the GPU itself, has direct knowledge of this information. Instead of determining which memory regions are identical, whether the data is variable, or whether memory requests overlap in time by looking at abstract memory regions or memory pages, the embodiments described herein are configured to directly access the game's graphical resources. In this context, “resources” may refer to specialized resources such as textures (images), generic buffers, or shader programs. Such resources are typically supported by one or more VRAM memory regions. Some memory regions may support many different resources. Furthermore, some resources are read-only, while others are writable.

[0036] To ensure the temporal consistency of resources, maximize content sharing within shared memory 109, and further minimize the complexity of memory sharing, at least one embodiment described herein identifies read-only and non-writable resources. This is particularly applicable to video games, as they tend to upload large amounts of immutable resources such as static assets, textures, mesh vertex buffers, and shader programs to VRAM.

[0037] Furthermore, the benefits of deduplication or reuse of memory content include the fact that, since game resources are already in VRAM, memory management processes can reduce the number of video game loads and potentially reduce PCI-e bus usage. Indeed, if video game resources are already loaded in VRAM, there is no need to re-upload those resources to VRAM when a new game instance starts or when a different game instance needs to access those resources. Moreover, it avoids delays or costs associated with uploading resources to VRAM, such as the need to rearrange or optimize the placement of resources within VRAM (e.g., tiling memory models for textures).

[0038] In some embodiments, the memory management process 108 may be a daemon process referred to herein as the GPU Memory Server (GMS). The GMS daemon “fully owns” or can fully manage resources between the shared memory 109 and the graphics processing hardware elements 120. The GMS is a centralized process with which all (or at least some subset) of the game processing communicates. In at least some cases, these communications occur through inter-process communication (IPC). Such IPC may include communication via an operating system socket, for example, through a GPU Memory Client (GMC) library that acts as an aid in the communication. The GMC may be embedded in one of the video game processing package files or binary files, or it may be dynamically loaded using a loading mechanism such as a Vulkan layer that enables dynamic interception of graphics API calls. Both the GMS and the video game processing ultimately interact with an installable client driver (ICD) from the GPU vendor that implements the graphics API calls. In this way, the embodiments described herein are independent of the GPU vendor and can therefore work with any GPU or processor provider.

[0039] Figure 3 shows an embodiment 300 in which the GMS 304 mediates between the GPU 307 and different game processes 301A, 301B, and 301C. Each game process 301A to 301C may be associated with a gaming session or other application instance. Each game process 301A to 301C includes its own instance (302A to 302C) of the GPU memory client. As described above, the GPU memory client communicates with the GMS 304, and the GMS 304 communicates with the GPU 307 (and its associated VRAM 308) via an installable client driver (ICD 305) and a GPU kernel-mode driver 306. By intercepting or otherwise accessing memory calls from game processes 301A-301C and / or ICD305, GMS304 can determine which content has been loaded into VRAM308 and how long that content will remain immutable. During this immutable period, the memory location of that content is shared and reused among the various game processes 301A-301C.

[0040] Furthermore, requests from game processes 301A-301C to store known immutable content in VRAM 308 are blocked, and instead, the requests are satisfied using known, previously stored content. Since GMS 304 is in virtually constant communication with GPU 307, GMS remains up-to-date on which content is stored in VRAM 308, where that content is stored, and the length of time each part of the content remains immutable. Therefore, despite asynchronous access by GPU 307, the stored content can be securely shared.

[0041] In at least some embodiments, the GMC (and therefore, the GMS) provides an API endpoint to determine whether a portion of the content or resource is already managed by the GMS. In some cases, the content or resource has an associated universally unique identifier (UUID). The GMS then uses that UUID to determine resource characteristics and other information obtained from game processes 301A-301C. The UUID and other information are determined either by intercepting graphics API calls or through explicit integration with the underlying game engine that performs various game processes. In some examples, these API calls optionally return a shared file descriptor or handle that increments a reference count and points to a memory backing already allocated for that resource. The game processes 301A-301C can then directly use that previously allocated memory for their operations.

[0042] In some cases, game processing 301A-301C and / or GPU memory clients 302A-302C are configured to add new resources tracked by the GMS to memory. In such cases, the game processing or GPU memory client provides the resource UUID and resource characteristics obtained from the game engine. The GMS then returns a shared file descriptor or handle pointing to the memory backing allocated for that resource, which the game can then use directly. This memory backing is owned by the GMS 304. The game processing then uses a secondary API call to indicate when the resource is fully set up or when the memory backing is filled with data. This also indicates that other GPU memory clients may start referencing the backing.

[0043] When a given resource is no longer needed, the game process and / or the GPU memory client notifies the GMS304 with a termination notification indicating that the specific resource is no longer required. This process then decrements the reference count of the memory backing. This, in turn, allows the GMS304 to determine when to remove the memory backing from its cache or collection of resident resources. In some cases, this occurs after the last resource reference was decremented. For example, the GMS304 might determine that a resource is in high demand and has been repeatedly used by different game processes. In such cases, the GMS decides to keep the resource stored in VRAM, even though it is not currently being used. In this instance, the GMS304 retains the resource in VRAM308, overriding the termination notification. Indirect termination notifications are also supported. For example, if game processing terminates unexpectedly, the GMS304 can track the IPC connection status to automatically decrement the reference count, indicating that the resource may be unloaded from memory.

[0044] While the embodiments described above are carried out with immutable and / or read-only resources, the embodiments described herein also intend to use variable resources via temporal sharing, including resources that are common in size and nature across instances. Indeed, in a multi-tenant scenario where many video game (or other application) instances run concurrently on the same GPU307, some resources used for rendering, such as frame buffers, do not need to persist frame-over-frame but have content that remains the same size. Furthermore, games utilize their memory backing for a time shorter than the overall target frame interval. The games and applications described herein often render faster or render frames after a period shorter than the target frame interval. This, in turn, creates an opportunity to use the same resources, or at least their memory backing, from frame rendering of one instance to frame rendering of another instance.

[0045] For memory regions that are not identical throughout game processing, but whose content is needed only temporarily or momentarily to render game frames, and for which memory is not otherwise needed or used contiguously, the embodiments described herein reallocate those memory regions in an exclusive and temporally shared manner (referred to herein as “pooled”) throughout game processing. By pooling and tracking these resources so that they can be reused temporally using separate sharing processes, the embodiments described herein provide reduced memory usage, reduced memory fragmentation over time, and faster resource allocation.

[0046] Furthermore, resource pooling and ownership by other common processes offers the additional benefit of allowing game processing to generate frame buffers that can be retained in memory for longer periods for purposes other than game rendering throughout the rest of the system. This frees video game processing from having to manage its persistence and eliminates the need to copy those resources (which in some cases can be very CPU and memory-intensive processes).

[0047] For example, in one instance, once game processing finishes rendering, the rendered framebuffer can persist throughout the entire encoding pipeline until the buffer is no longer needed. Without creating wait points in the game rendering logic (e.g., swap-chain image retrieval), the encoding process itself can return the framebuffer resource to a common resource pool when game processing needs to use the same framebuffer again. Once game processing is performed using the framebuffer, the game processing no longer needs to synchronize with or deal with its delayed use. In other words, the embodiments described herein are designed to eliminate the relationship between backend video game processing and client-side encoding processing in cloud gaming scenarios. Furthermore, these embodiments simultaneously make better use of memory by immediately recycling and reusing resources that are no longer needed through pooling.

[0048] In at least some cases, the implementation generally implements the same topology used for deduplication, as shown in Figure 3. The GMS304 is used to own, pool, and exclusively distribute resources (i.e., each resource has a single write operation at any given time). When communication is performed via IPC, each GPU memory client 302A-302C acts as an interface library or as an intermediary with the GMS304. With respect to writable resources, the embodiments described herein ensure that all asynchronous tasks performed on the GPU-writable resources are completed before the resources are redistributed for reuse.

[0049] To achieve this, each resource is associated with a semaphore or fence synchronization object. In different scenarios using GPUs from different vendors, the fence synchronization object may have different names or characteristics. The GMS (or other pooling mechanism) is configured to wait on its fence synchronization object before redistributing a given resource. Additionally, or instead, the GMS304 may prioritize resources whose associated fence has been signaled and completed its work before redistribution. This consequently minimizes propagation delays between processes or across different uses of resources. Client processes (e.g., 301A-301C) signal when they have completed their work through their associated fence object. In this way, the GMS304 and other components are implemented to reuse even variable resources across different game processes.

[0050] For immutable and / or read-only resources, the memory management process (e.g., GMS304) instructs the requesting application instance (e.g., game processes 301A-301C) to access the requested content from a specific location in shared memory. The instruction may also specify a time frame, or time span, during which the content is immutable. The game process then accesses the previously stored content, and GMS304 may avoid storing the content associated with that resource in memory. In such embodiments, GMS304 is said to own the resource, meaning it manages its content and monitors and controls its use across different game processes (across other application instances).

[0051] Figure 4A shows an embodiment 400A in which the memory management process is incorporated into an application processing package file associated with the application instance that sent the request. For example, in some cases, the memory management process 403 (similar to or the same as GMS304 in Figure 3) is incorporated into an application processing package file 402 which is part of the application instance 401. Thus, the memory management process 403 is loaded as part of the application instance 401. In such cases, the memory management process 403 can determine whether a particular resource is managed by the memory management process using one or more resource identifiers 404 or resource characteristics obtained from the application processing package file 402.

[0052] In such an example, the memory management process 403 is incorporated as part of the application instance 401. If the application instance 401 is a video game instance, the application processing package file may be part of the game engine or may be used by the game engine when the video game instance is executed. The application processing package file may provide resource identifiers or resource characteristics obtained through runtime video game processing from the video game package file or other application resource files. The memory management process 403 has low-level access to each memory call or memory operation performed by the video game instance. Using the UUID, as well as resource characteristics such as resource size, resource file type, read-only or write-only status, mutable or immutable status, or other features, the memory management process 403 can determine when and how often the resource is used, and whether the resource can be shared among various application instances.

[0053] In other examples, memory management processes are dynamically loaded along with the application instance that sent the request. For example, as shown in Embodiment 400B of Figure 4B, the memory management process 411 is dynamically loaded using a Vulkan layer 412 that enables dynamic interception of graphics API calls 414. The memory management process 411 uses various resource identifiers or resource characteristics obtained from at least one intercepted API call to determine whether a particular resource is already managed by the memory management process 411. As described above, the resource identifier obtained from an intercepted API call may be a UUID. In some cases, the intercepted graphics API call 414 returns a shared file descriptor 416 that increments a reference count and points to a previously allocated memory backing for the resource. The application instance 413 then uses the shared file descriptor 416 when accessing a particular resource stored in the memory of the graphics processing hardware 415 of the underlying computer system 410.

[0054] In some embodiments, the memory management process 411 assumes complete ownership or complete management control of the interaction between the application instance 413 and the graphics processing hardware 415. As part of this management, the memory management process 411 preloads various resources for the application instance when the application instance sends a memory storage request. For example, dynamically loaded using the Vulkan layer 412, the memory management process 411 may intercept graphics API calls 414 and determine whether a certain resource is in use or likely to be used in the future. In such cases, it preloads some of the specific resources into memory (e.g., VRAM) before the resources are actually requested by the application instance. This reduces access time when the application instance finally requests to load the resources into memory.

[0055] Furthermore, in some cases, the memory management process 411 stores a particular resource in shared memory for a specific length of time after determining that the resource is no longer being used by a large number of application instances. Thus, even if a resource is no longer being used by an application instance 413, the memory management process 411 may determine that the resource should be retained in memory. In some cases, the memory management process 411 looks at the churn associated with the resource. If a resource is continuously loaded and unloaded from memory (i.e., "churn"), the memory management process 411 determines that the churn level is high for that resource and therefore it should be retained in memory even if it is not currently being used by an application instance. In some cases, a particular resource is stored in shared memory for a length of time based on or proportional to the determined churn amount associated with that particular resource.

[0056] Figure 5 shows an embodiment 500 in which a memory management process 501 identifies certain content as mutable over time. This mutable content 505 is placed in a shared pool 504 where other application instances (e.g., 502A-502C) have access to that content (or other temporary content). The memory management process 501 shares the content identified as mutable using a fence synchronization object 506. The fence synchronization object 506 is used to determine when asynchronous tasks (e.g., 503A-503C) performed using the content have been completed. In this way, the memory management process 501 knows, with up-to-date information, which asynchronous tasks have been completed relative to parts of the mutable content. While the content is being used to perform asynchronous tasks 503A-503C, the content is removed from the shared pool 504, and when the use of the content is complete, the memory management process 501 returns the mutable content 505 to the shared pool 504 by changing the state of the fence synchronization object 506. In addition to the variable content 505, the shared pool 504 may also include immutable content 507. This immutable content can be shared among various application instances 502A to 502C together with, or instead of, the variable content 505.

[0057] Regardless of whether the content is mutable or immutable, at least some of the embodiments described herein may add copy-on-write protection to the content. Copy-on-write protection can create another copy of a given file or resource when an application instance writes to that resource, ensuring that the changes are stored and the original version is also preserved. Then, even if the resource, or some memory location, is mistakenly identified as immutable, the resource will be preserved in its original form because it is marked as "copy-on-write". Furthermore, another (private) copy of the resource is created before any changes are made. This ensures that data integrity is preserved across different application instances.

[0058] In addition to the above computer implementation method, a corresponding system is also described. The system includes at least one physical processing unit and physical memory containing a computer executable instruction, which, when executed by the physical processing unit, causes the physical processing unit to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances. The memory management process receives a request from at least one of the application instances to store content associated with a specific resource implemented by the application instance in shared memory, determines that the content specified in the request has been previously stored at a specific location in shared memory, identifies a time span during which the specified content stored in shared memory will be immutable, and instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span.

[0059] Furthermore, the system provides a non-temporary computer-readable medium containing one or more computer-executable instructions, which, when executed by at least one processing unit of the computer, causes the computer to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances, the memory management process receiving a request from at least one of the application instances to store content associated with a specific resource implemented by the application instance in shared memory, determining that the content specified in the request has been previously stored at a specific location in shared memory, identifying a time span during which the specified content stored in shared memory will be immutable, and instructing the requesting application instance to access the specified content from that specific location in shared memory during the specified time span.

[0060] Figures 6 to 8 illustrate cloud gaming systems, infrastructure, and clients that may implement the embodiments described herein. Referring, for example, Figure 6 provides a detailed description of an exemplary ecosystem that delivers content to end nodes and directs requests for content to specific end nodes. The descriptions corresponding to Figures 7 and 8 provide an overview of an exemplary distributed infrastructure and an exemplary content playback unit used during a playback session, respectively.

[0061] Figure 6 is a block diagram of a content distribution ecosystem 600, which includes a distributed infrastructure 610 that communicates with a gaming client 620, a content playback unit, or other software applications designed to provide rendered graphics to a user. In some embodiments, the distributed infrastructure 610 is configured to encode data at a specific data rate and to transfer the encoded data to the gaming client 620. The gaming client 620 is configured to receive the encoded data via the distributed infrastructure 610 and to decode the data for playback to the user. The data provided by the distributed infrastructure 610 includes, for example, audio, video, text, images, animations, interactive content, haptic data, virtual or augmented reality data, location data, gaming data, or any other type of data provided via streaming.

[0062] The distributed infrastructure 610 generally represents any service, hardware, software, or other infrastructure components configured to deliver content to end users. For example, the distributed infrastructure 610 includes content aggregation systems, media transcoding and packaging services, network elements, and / or various other types of hardware and software. In some cases, the distributed infrastructure 610 is used as a highly complex distributed system, a single media server or device, or any device between them. In some examples, regardless of size or complexity, the distributed infrastructure 610 includes at least one physical processing unit 612 and at least one memory 614. As described herein, one or more modules 616 are stored in or loaded into memory 614 to enable adaptive streaming.

[0063] The gaming client 620 generally represents any type or form of device or system capable of playing audio, video, and / or other gaming content provided via the distributed infrastructure 610. Examples of the gaming client 620 include, but are not limited to, mobile phones, tablets, laptop computers, desktop computers, televisions, set-top boxes, digital media playback units, virtual reality headsets, augmented reality glasses, and / or any other type or form of device capable of rendering digital content. Similar to the distributed infrastructure 610, the gaming client 620 includes a physical processing unit 622, memory 624, and one or more modules 626. Some or all of the adaptive streaming processing described herein is performed or enabled by the modules 626, and in some examples, the modules 626 of the distributed infrastructure 610 work in conjunction with the modules 626 of the gaming client 620 to provide adaptive streaming of multimedia content.

[0064] In one embodiment, one or more modules 616 and / or 626 in Figure 6 represent one or more software applications or programs that, when executed by a computer, cause the computer to perform one or more tasks. For example, as will be described in much greater detail below, one or more modules 616, 626 represent modules that are stored in and configured to run on one or more general-purpose computers. One or more modules 616, 626 in Figure 6 also represent all or part of one or more dedicated computers configured to perform one or more tasks.

[0065] Furthermore, one or more modules, processes, algorithms, or steps described herein transform data, physical devices, and / or representations of physical devices from one form to another. For example, one or more modules described herein receive audio data to be encoded, transform the audio data by encoding it, output the encoded result for use in an adaptive audio bitrate system, transmit the transformed result to a content playback unit, and further render the transformed data for consumption by an end user. In addition, or instead, one or more modules described herein transform processing units, volatile memory, non-volatile memory, and / or any other parts of a physical computer from one form to another by executing on a computer, storing data in a computer, and / or interacting with a computer in other ways.

[0066] The physical processing units 612 and 622 generally represent processing units using any type or form of hardware capable of interpreting and / or executing computer-readable instructions. In one example, each of the physical processing units 612 and 622 accesses and / or modifies one or more modules 616 and 626. In addition, or instead, the physical processing units 612 and 622 execute one or more modules 616 and 626 to facilitate adaptive streaming of multimedia content. Examples of physical processing units 612 and 622 include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) with soft-core processors, application-specific integrated circuits (ASICs), one or more parts thereof, one or more variations or combinations thereof, and / or any other suitable physical processing units.

[0067] Memory 614, 624 generally represents any kind or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, memory 614, and / or 624 stores, loads, and / or maintains one or more modules 616, 626. Examples of memory 614, and / or 624 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drives (HDDs), solid-state drives (SSDs), optical disk drives, caches, one or more variations or combinations thereof, and / or any other suitable memory device or system.

[0068] Figure 7 is a block diagram of exemplary elements of a content distribution infrastructure 610 according to one embodiment. The distributed infrastructure 610 includes a storage unit 710, a service unit 720, and a network 730. The storage unit 710 generally represents any device, set of devices, and / or system capable of storing content for delivery to end users. The storage unit 710 includes a central repository having devices that store terabytes or petabytes of data, and / or includes a distributed storage system (e.g., devices that mirror or cache content at Internet interconnect locations to enable faster access to mirrored content within a region). The storage unit 710 may also be configured in any other suitable form.

[0069] Therefore, the storage unit 710 may store a variety of different items, including content 712, user data 714, and / or log data 716. Content 712 includes television programs, movies, video games, user-generated content, and / or any other appropriate type or form of content. User data 714 includes personally identifiable information (PII), payment information, preference settings, language and accessibility settings, and / or any other information associated with a particular user or content playback unit. Log data 716 includes viewing history information, network throughput information, and / or any other metrics associated with a user's connection to or interaction with the distributed infrastructure 610.

[0070] The service unit 720 includes a personalization service unit 722, a transcoding service unit 724, and / or a packaging service unit 726. The personalization service unit 722 tailors recommendations, content streams, and / or other aspects of the user's experience with the distributed infrastructure 610 to the user's individual needs. The transcoding service unit 724 compresses the media at different bitrates and enables real-time switching between different encodings, as described in much greater detail below. The packaging service unit 726 packages the encoded video before placing it on a transmission network, such as network 730, for streaming.

[0071] Network 730 generally represents any medium or architecture that facilitates communication or data transfer. Network 730 facilitates communication or data transfer using wireless and / or wired connections. Examples of Network 730 include, but are not limited to, intranets, wide area networks (WANs), local area networks (LANs), personal area networks (PANs), the Internet, power line communications (PLCs), cellular networks (e.g., Global System for Mobile Communications (GSM) networks), one or more of these, one or more variations or combinations of these, and / or any other suitable network. For example, as shown in Figure 7, Network 730 includes an Internet backbone 732, an Internet service provider 734, and / or a local network 1836. As will be described in much more detail below, video and / or audio bitrates are adjusted in response to bandwidth limitations and bottlenecks in one or more of these network segments.

[0072] Figure 8 is a block diagram of an exemplary use case of the gaming client 620 shown in Figure 6. The gaming client 620 generally represents any type or form of computing device capable of reading computer executable instructions. The gaming client 620 includes, but is not limited to, laptops, tablets, desktops, servers, mobile phones, multimedia playback units, embedded systems, wearable devices (e.g., smartwatches, smart glasses, etc.), smart cars, gaming consoles, smart devices and other Internet of Things (IoT) devices, one or more variations or combinations thereof, and / or any other suitable computing device.

[0073] As shown in Figure 8, in addition to the processing unit 622 and memory 624, the gaming client 620 includes a communication interface 822 connected to the communication infrastructure 802 and the network connection 824. The gaming client 620 also includes a graphics interface 826 connected to the graphics device 828, an input interface 834 connected to the input device 836, and a storage interface 838 connected to the storage device 840.

[0074] The communication infrastructure 802 generally represents any type or form of infrastructure that facilitates communication between one or more components of a computing device. Examples of the communication infrastructure 802 include, but are not limited to, any type or form of communication bus (e.g., peripheral component interconnect (PCI) bus, PCI Express (PCIe) bus, memory bus, frontside bus, integrated drive electronics (IDE) bus, control or register bus, host bus, etc.).

[0075] As described above, memory 624 generally represents any kind or form of volatile or non-volatile storage device or medium capable of storing data and / or other computer-readable instructions. In some examples, memory 624 stores and / or loads the operating system 808 for execution by the processing unit 622. In one example, the operating system 808 includes and / or represents software that manages computer hardware and software resources, and further, in addition to / or instead, provides common services to computer programs and / or applications on the gaming client 620.

[0076] The operating system 808 performs various system management functions, such as managing hardware elements (e.g., graphics interface 826, audio interface 830, input interface 834, and / or storage interface 838). The operating system 808 also provides a processing and memory management model for the playback application 810. The modules of the playback application 810 include, for example, a content buffer 812, an audio decoder 818, and a video decoder 820.

[0077] The playback application 810 is configured to retrieve digital content via the communication interface 822 and play that digital content through the graphics interface 826. The graphics interface 826 is configured to transmit the rendered video signal to the graphics device 828. In normal operation, the playback application 810 receives a request from a user to play a specific title or specific content. The playback application 810 then identifies one or more encoded video and audio streams associated with the requested title. After finding the encoded streams associated with the requested title, the playback application 810 downloads the sequence header index associated with each encoded stream associated with the requested title from the distributed infrastructure 610. The sequence header index associated with the encoded content contains information related to the encoded data sequence contained in the encoded content.

[0078] In one embodiment, the playback application 810 initiates the download of content associated with a requested title by downloading sequence data encoded to the lowest audio and / or video playback bitrate to minimize playback startup time. The requested digital content file is then downloaded to a content buffer 812 configured to function as a first-in, first-out queue. In one embodiment, each unit of downloaded data includes either a video data unit or an audio data unit. When a video data unit associated with a requested digital content file is downloaded to the gaming client 620, the video data unit is pushed into the content buffer 812. Similarly, when an audio data unit associated with a requested digital content file is downloaded to the gaming client 620, the audio data unit is pushed into the content buffer 812. In one embodiment, video data units are stored in a video buffer 816 within the content buffer 812, and audio data units are stored in an audio buffer 814 within the content buffer 812.

[0079] The video decoding unit 820 reads video data units from the video buffer 816 and outputs video data units of a video frame sequence whose duration corresponds to a certain length of playback time. By reading video data units from the video buffer 816, the video data units are effectively extracted from the video buffer 816. Next, the video frame sequence is rendered by the graphics interface 826 and sent to the graphics device 828 for display to the user.

[0080] The audio decoding unit 818 reads audio data units from the audio buffer 814 and outputs the audio data units as an audio sample sequence, which is generally time-synchronized with the decoded video frame sequence. In one embodiment, the audio sample sequence is sent to an audio interface 830 that converts the audio sample sequence into an electro-audio signal. The electro-audio signal is then sent to the speaker of an audio device 832 that generates an acoustic output accordingly.

[0081] In situations where the bandwidth of the distributed infrastructure 610 is limited and / or fluctuates, the playback application 810 downloads and buffers contiguous portions of video data and / or audio data from video encoded data at different bitrates based on various factors (e.g., scene complexity, audio complexity, network bandwidth, device capabilities, etc.). In some embodiments, video playback quality takes precedence over audio playback quality. Furthermore, audio playback quality and video playback quality are balanced against each other, and in some embodiments, audio playback quality takes precedence over video playback quality.

[0082] The graphics interface 826 is configured to generate video frame data and transmit the video frame data to the graphics device 828. In one embodiment, the graphics interface 826 is included as part of an integrated circuit together with the processing unit 622. Alternatively, the graphics interface 826 is configured as a hardware accelerator separate from (i.e., not integrated into) the chipset containing the processing unit 622.

[0083] The graphics interface 826 generally represents any type or form of device configured to send images for display to the graphics device 828. For example, the graphics device 828 is fabricated using liquid crystal display (LCD) technology, cathode ray tube (CRT) technology, and (organic or inorganic) light-emitting diode (LED) display technology. In some embodiments, the graphics device 828 also includes a virtual reality display unit and / or an augmented reality display unit. The graphics device 828 includes any technically feasible means for generating images for display. In other words, the graphics device 828 generally represents any type or form of device capable of visually displaying information sent by the graphics interface 826.

[0084] As shown in Figure 8, the gaming client 620 also includes at least one input device 836 connected to the communication infrastructure 802 via an input interface 834. The input device 836 generally represents any type or form of computing device capable of providing computer or human-generated input to the gaming client 620. Examples of the input device 836 include, but are not limited to, keyboards, pointing devices, voice recognition devices, touchscreens, wearable devices (e.g., gloves, watches, etc.), control units, one or more variations or combinations thereof, and / or any other type or form of electronic input mechanism.

[0085] The gaming client 620 also includes a storage device 840 connected to the communication infrastructure 802 via a storage interface 838. The storage device 840 generally represents any type or form of storage device or medium capable of storing data and / or other computer-readable instructions. For example, the storage device 840 could be a magnetic disk drive, a solid-state drive, an optical disk drive, a flash drive, etc. The storage interface 838 generally represents any type or form of interface or device for data transfer between the storage device 840 and other elements of the gaming client 620.

[0086] Numerous other devices or subsystems are included in or connected to the gaming client 620. Conversely, one or more components and devices shown in Figure 8 do not need to be present to carry out the embodiments described herein and / or illustrated. The devices and subsystems are interconnected in ways different from those shown in Figure 8. The gaming client 620 can also be employed in any number of software, firmware, and / or hardware configurations. For example, one or more exemplary embodiments disclosed herein are encoded as computer programs (also referred to as computer software, software applications, computer-readable instructions, or computer control logic) on a computer-readable medium. The term “computer-readable medium” generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission media such as carrier waves, and non-transient media such as magnetic storage media (e.g., hard disk drives, tape drives, etc.), optical storage media (e.g., compact discs (CDs), digital video discs (DVDs), and Blu-ray discs), and electronic storage media (e.g., solid-state drives and flash media), as well as other digital storage systems.

[0087] A computer-readable medium containing a computer program is loaded into the gaming client 620. Then, all or part of the computer program stored on the computer-readable medium is stored in memory 624 and / or storage device 840. When executed by the processing unit 622, the computer program loaded into memory 624 causes the processing unit 622 to perform and / or become a means to perform one or more functions of the exemplary embodiments described herein and / or illustrated herein. Further or alternatively, one or more exemplary embodiments described herein and / or illustrated herein are implemented in firmware and / or hardware. For example, the gaming client 620 is configured as an application-specific integrated circuit (ASIC) adapted to implement the embodiments disclosed herein.

[0088] Exemplary Embodiments Example 1: A computer implementation method comprising the step of instantiating a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances, wherein the memory management process receives a request from at least one of the application instances indicating that it wants to store content associated with a specific resource implemented by the application instance in shared memory, determines that the content specified in the request has been previously stored in a specific location in shared memory, identifies a time span in which the specified content stored in shared memory will be immutable, and instructs the requesting application instance to access the specified content from the specific location in shared memory during the specified time span. Example 2: The computer implementation described in Example 1, in which the memory management process instructs the requesting application instance to access the specified content from a specific location in shared memory during a specified time span, instead of storing the specified content associated with a specific resource in shared memory. Example 3: The computer implementation method described in Example 1 or 2, wherein the identified content stored in shared memory is monitored by memory management processes. Example 4: A computer implementation described in any one of Examples 1 to 3, in which the memory management process is incorporated into an application processing package file associated with the application instance that sent the request. Example 5: A computer implementation method described in any one of Examples 1 to 4, wherein the memory management process determines whether a particular resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from the application processing package file. Example 6: The computer implementation method described in any one of Examples 1 through 5, wherein the application processing package file contains a game engine for a video game. Example 7: A computer implementation of any one of Examples 1 through 6, wherein the memory management process is dynamically loaded along with the application instance that sent the request. Example 8: A computer implementation of any one of Examples 1 through 7, wherein memory management operations are dynamically loaded using a Vulkan layer that enables dynamic interception of graphics application programming interface (API) calls. Example 9: A computer implementation of any one of Examples 1 to 8, wherein the memory management process determines whether a particular resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from at least one intercepted API call. Example 10: A computer implementation of any one of Examples 1 through 9, wherein at least one of the resource identifiers obtained from the intercepted API call contains a universally unique identifier (UUID). Example 11: A computer implementation described in any one of Examples 1 through 10, in which the intercepted API call increments a reference count and further returns a shared file descriptor that points to a memory backing previously allocated for the resource. Example 12: A computer implementation described in any one of Examples 1 through 11, in which the application instance uses a shared file descriptor when accessing a specific resource. Example 13: A system comprising at least one physical processing unit and physical memory containing a computer executable instruction, wherein, when executed by the physical processing unit, the computer executable instruction causes the physical processing unit to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances, the memory management process receiving a request from at least one application instance indicating that content associated with a specific resource implemented by the application instance should be stored in shared memory, determining that the content specified in the request has been previously stored at a specific location in shared memory, identifying a time span during which the specified content stored in shared memory will be immutable, and instructing the requesting application instance to access the specified content from the specific location in shared memory during the specified time span. Example 14: The system described in Example 13, where the memory management process preloads one or more resources for the application instance that sent the request. Example 15: The system described in Example 13 or 14, in which the memory management process determines that a particular resource is no longer being used by a large number of application instances, and then stores that resource in shared memory for at least a certain length of time. Example 16: A system as described in any one of Examples 13 to 15, in which a specific resource is stored in shared memory for a certain length of time based on a determined amount of churn related to that specific resource. Example 17: The system described in any one of Examples 13 through 16, in which content identified as mutable is shared in a pool among many different application instances through memory management processes. Example 18: The system described in any one of Examples 13 through 17, in which content identified as mutable is associated with a fence synchronization object used to track when an asynchronous task performed using the content is completed. Example 19: The system described in any one of Examples 13 through 18, in which the memory management process redistributes content identified as mutable to a pool once an asynchronous task is completed. Example 20: A non-temporary computer-readable medium comprising one or more computer-executable instructions, the instructions, when executed by at least one processing unit of a computer, cause the computer to instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by a number of different application instances, the memory management process receiving a request from at least one of the application instances to store content associated with a specific resource implemented by the application instance in shared memory, determining that the content specified in the request has been previously stored at a specific location in shared memory, determining a time span during which the specified content stored in shared memory will be immutable, and instructing the requesting application instance to access the specified content from the specific location in shared memory during the specified time span.

[0089] As described in detail herein, the computing devices and systems described and / or shown herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions, such as those housed within the modules described herein. In its most basic configuration, each of these computing devices may include at least one memory device and at least one physical processing unit.

[0090] In some instances, the term “memory device” generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device may store, load, and / or maintain one or more of the modules described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drives (HDDs), solid-state drives (SSDs), optical disk drives, caches, one or more variations or combinations thereof, or any other suitable storage memory.

[0091] In some instances, the term “physical processing unit” generally refers to a processing unit using any type or form of hardware capable of interpreting and / or executing computer-readable instructions. In one example, a physical processing unit may access and / or modify one or more modules stored in the memory device. Examples of physical processing units include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) with soft-core processors, application-specific integrated circuits (ASICs), one or more parts thereof, one or more variations or combinations thereof, or any other suitable physical processing unit.

[0092] Although shown as separate elements, the modules described and / or shown herein may represent a single module or part of an application. Furthermore, in some embodiments, one or more of these modules may represent one or more software applications or programs that, when executed by a computer, cause the computer to perform one or more tasks. For example, one or more of the modules described and / or shown herein may represent a module configured to be stored and executed in one or more of the computer or systems described and / or shown herein. One or more of these modules may also represent all or part of one or more dedicated computers configured to perform one or more tasks.

[0093] Furthermore, one or more of the modules described herein may convert data, physical devices, and / or representations of physical devices from one form to another. In addition, or instead, one or more of the modules described herein may convert processing units, volatile memory, non-volatile memory, and / or any other parts of a physical computer from one form to another by executing on a computer, storing data on a computer, and / or interacting with a computer in other ways.

[0094] In some embodiments, the term “computer-readable medium” generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable mediums include, but are not limited to, transmission media such as carrier waves, and non-transient media such as magnetic storage media (e.g., hard disk drives, tape drives, and floppy disks), optical storage media (e.g., compact discs (CDs), digital video discs (DVDs), and Blu-ray discs), and electronic storage media (e.g., solid-state drives and flash media), and other distributed systems.

[0095] The processing parameters and process sequences described and / or shown herein are illustrative and can be modified as desired. For example, the processes shown and / or described herein may be shown or described in a specific order, but they do not necessarily have to be performed in the order shown or described. Various exemplary methods described and / or shown herein may omit one or more of the processes described or shown herein, or include additional steps to those described processes.

[0096] The foregoing description is provided so that those skilled in the art can best utilize various aspects of the exemplary embodiments of the disclosure herein. This exemplary description is not intended to be exhaustive or to limit to any specific form disclosed. Numerous modifications and variations are possible without departing from the spirit and scope of this disclosure. The embodiments disclosed herein should be considered in all respects to be illustrative and not limiting. To specify the scope of this disclosure, refer to the appended claims and their equivalents.

[0097] Unless otherwise stated, the terms “connected” and “linked” (and their derivatives) as used herein and in the claims should be interpreted as referring to both direct and indirect connections (i.e., connections through other elements or components). Furthermore, the indefinite article used in the original English specification and claims should be interpreted as meaning “at least one.” Finally, the terms “include” and “have” (and their derivatives) as used herein and in the claims are, for ease of use, interchangeable and have the same meaning as the term “equipped with.” [Explanation of symbols]

[0098] 612, 622 Physical Processing Unit 614, 624 memory 620 Gaming Client 840 Storage device

Claims

1. In computer implementation methods, The process of instantiating a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by multiple different application instances. Including, the memory management process is The application receives a request from at least one of the application instances indicating that it wants to store content associated with a specific resource implemented by that application instance in the shared memory. It is determined that the content identified in the request has been previously stored in a specific location in the shared memory, Identify the time span during which the identified content stored in the shared memory remains unchanged. A computer implementation method that instructs the requesting application instance to access the specified content from the specified location in the shared memory during the specified time span.

2. The computer implementation method according to claim 1, wherein the memory management process instructs the requesting application instance to access the identified content from the specific location in the shared memory during the specified time span, instead of storing the identified content associated with the specific resource in the shared memory.

3. The computer implementation method according to claim 1, wherein the identified content stored in the shared memory is monitored by the memory management process.

4. The computer implementation method according to claim 1, wherein the memory management process is incorporated into an application processing package file associated with the application instance that sent the request.

5. The computer implementation method according to claim 4, wherein the memory management process determines whether the specific resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from the application processing package file.

6. The computer implementation method according to claim 4, wherein the application processing package file includes a game engine for video games.

7. The computer implementation method according to claim 1, wherein the memory management process is dynamically loaded together with the application instance that sent the request.

8. The computer implementation method according to claim 7, wherein the memory management process is dynamically loaded using a Vulkan layer that enables dynamic interception of graphics application programming interface (API) calls.

9. The computer implementation method according to claim 8, wherein the memory management process determines whether the specific resource is already managed by the memory management process using one or more resource identifiers or resource characteristics obtained from at least one intercepted API call.

10. The computer implementation method according to claim 9, wherein at least one of the resource identifiers obtained from the intercepted API call includes a universally unique identifier (UUID).

11. The computer implementation method according to claim 9, wherein the intercepted API call increments a reference count and further returns a shared file descriptor pointing to a memory backing previously allocated for the resource.

12. The computer implementation method according to claim 11, wherein the application instance uses the shared file descriptor when accessing the specific resource.

13. In the system, At least one physical processing unit, Physical memory containing computer executable instructions and Includes, When the aforementioned computer executable instruction is executed by the physical processing unit, the physical processing unit will, This instantiates a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by multiple different application instances. The aforementioned memory management process is: The application receives a request from at least one of the application instances indicating that it wants to store content associated with a specific resource implemented by that application instance in the shared memory. It is determined that the content identified in the request has been previously stored in a specific location in the shared memory, Identify the time span during which the identified content stored in the shared memory remains unchanged. A system that instructs the requesting application instance to access the specified content from the specified location in the shared memory during the specified time span.

14. The system according to claim 13, wherein the memory management process preloads one or more resources for the application instance that sent the request.

15. The system according to claim 13, wherein the memory management process determines that the particular resource is no longer being used by the numerous application instances, and then stores the particular resource in the shared memory for at least a certain length of time.

16. The system according to claim 15, wherein the specific resource is stored in the shared memory for a specific length of time based on the determined churn amount related to the specific resource.

17. The system according to claim 13, wherein content identified as variable is shared in a pool among the many different application instances by the memory management process.

18. The system according to claim 17, wherein the content identified as variable is associated with a fence synchronization object used to track when an asynchronous task performed using the content has been completed.

19. The system according to claim 18, wherein the memory management process, upon completion of the asynchronous task, redistributes the content identified as variable to the pool.

20. In a non-temporary computer-readable medium, The instructions include one or more computer executable instructions, and when the instructions are executed by at least one processing unit of the computer, the computer will be able to: Instantiate a memory management process configured to communicate with one or more graphics processing hardware elements to control the use of shared memory by multiple different application instances, The aforementioned memory management process is: The application receives a request from at least one of the application instances indicating that it wants to store content associated with a specific resource implemented by that application instance in the shared memory. It is determined that the content identified in the request has been previously stored in a specific location in the shared memory, Identify the time span during which the identified content stored in the shared memory remains unchanged. A non-temporary computer-readable medium that instructs the requesting application instance to access the specified content from the specified location in the shared memory during the specified time span.

Images