Resource sharing of content delivery systems and applications

Abstract

The invention relates to resource sharing of content delivery systems and applications. In various examples, static resources (or physical memory locations storing the static resources) may be shared between application instances running in a distributed environment. For example, the disclosed systems and methods may determine whether application resources are sharable (e.g., static or dynamic) by evaluating metadata associated with the resources. In some examples, a system may allocate a portion (e.g., a range) of virtual memory associated with an application instance as a binding target for static resources. The portion of virtual memory may then be mapped to a physical memory allocation that stores static resources. In this manner, multiple virtual memory portions of multiple instances of an application may be mapped to the same physical memory allocation, and static resources may be shared between different application instances.

Description

Background Technology

[0001] Various techniques enable multiple application instances running on the same server or server group to share certain resources stored in memory. For example, some of these techniques can help certain applications using graphical application programming interfaces (APIs), such as game applications or other interactive applications, achieve higher density. However, while these techniques can be applied to and used for some graphics APIs, applying the same or similar techniques to other graphics APIs (such as modern and / or low-level graphics APIs) has proven challenging.

[0002] For example, in some graphics APIs (such as traditional graphics APIs), system drivers can be used to manage the association between memory and application resources (such as textures, buffers, render targets, neural networks, or model weights). However, in other graphics APIs (such as modern graphics APIs), the application itself can control these associations between memory and resources. Therefore, memory that might otherwise be shared among application instances can be effectively eliminated. Furthermore, in graphics API systems where resources may be managed and populated by the application itself, the system driver may not be able to access the resource content to determine whether multiple instances of the same resource are stored in memory. Summary of the Invention

[0003] Embodiments of this disclosure relate to resource sharing in content delivery systems and applications. Systems and methods are disclosed that enable different instances of an application running in a distributed environment (e.g., multiple instances of a game application or any other application running on a single server or a set of servers) to share certain resources.

[0004] For example, the systems and methods of this disclosure can determine whether an application resource is shareable (e.g., static or dynamic) by evaluating supplementary information (e.g., metadata) associated with the resource. In some examples, the system can allocate a portion of virtual memory (e.g., range, region, etc.) associated with an application instance as a binding target for a static resource. This portion of virtual memory can be mapped to a physical memory allocation storing the static resource. In this way, multiple portions of virtual memory for multiple instances of the application can be mapped to the same physical memory location, and the static resource can be shared between different application instances. Furthermore, the systems and methods of this disclosure can use a graphics processing unit (GPU) to compute an identifier (e.g., a hash value) corresponding to the shareable resource. These identifiers can be used to search for duplicates of the shareable resource in allocated physical memory. In this way, duplicates of the shareable resource can be identified and merged, and redundant portions of physical memory can be freed up.

[0005] Compared to conventional systems, the systems disclosed herein, in some embodiments, are capable of transparently sharing resources in a multi-application environment, where a graphical API is used to control memory allocation, resource creation, and resource binding. For example, by examining attributes in the metadata used to create application resources, the systems disclosed herein are able to identify which application resources can be shared and allocate virtual memory accordingly for these shareable application resources. Furthermore, compared to conventional systems, the systems disclosed herein are able to map the virtual memory allocation of shareable resources to dedicated physical memory locations that store application resources. Because the system can allocate dedicated physical memory to store application resources, rather than sharing memory with multiple resources bound together, the systems disclosed herein can share physical memory with a single resource binding among multiple application instances.

[0006] Furthermore, compared to conventional systems, the systems of this disclosure in some embodiments can use a graphics processing unit (GPU) to compute resource identifiers. This allows the systems of this disclosure to use these identifiers to identify duplicate shareable resources in multi-application environments where graphics APIs are used to control memory allocation, resource creation, and resource binding. By identifying shareable resources that have been duplicated or otherwise stored multiple times in multiple locations of physical memory, the system can consolidate instances of shareable resources into a single instance (or fewer instances) stored in a single location of physical memory, freeing up physical memory portions previously used to store duplicate resources. This improves memory utilization and, by sharing resources and reducing or even eliminating potentially unnecessary redundant copies of resources, enables the system to achieve higher density and host more application instances per device / system. Attached Figure Description

[0007] The system and method for resource sharing between the content delivery system and applications will now be described in detail with reference to the accompanying drawings, wherein:

[0008] Figure 1 This is a data flow diagram illustrating an example of a resource sharing process for a content delivery system and application according to some embodiments of the present disclosure;

[0009] Figure 2 Examples of determining a classification associated with a resource according to some embodiments of this disclosure are shown;

[0010] Figure 3 Examples of determining the type of memory to allocate for application resources according to some embodiments of this disclosure are shown;

[0011] Figure 4 Examples of determining a binding process for binding resources to memory are shown according to some embodiments of the present disclosure;

[0012] Figure 5 Examples of merging duplicate resources according to some embodiments of this disclosure are shown;

[0013] Figure 6A A hierarchical view of shareable resources and their corresponding physical and virtual memory allocations according to some embodiments of this disclosure is shown;

[0014] Figure 6B A hierarchical view of an example of memory aliasing according to some embodiments of the present disclosure is shown;

[0015] Figure 7 This is a flowchart illustrating example methods performed in association with shared resources in a multi-application environment that uses a graphical API to control memory allocation, resource creation, and / or resource binding, according to some embodiments of the present disclosure;

[0016] Figure 8 This is a flowchart illustrating an example method for remapping virtual memory from a first physical memory allocation to a second physical memory allocation according to some embodiments of the present disclosure;

[0017] Figure 9 This is a flowchart illustrating an example method for merging duplicate resources and releasing physical memory allocations according to some embodiments of this disclosure;

[0018] Figure 10 Example parallel processing units suitable for implementing at least some embodiments of this disclosure are shown;

[0019] Figure 11A The present invention illustrates at least some embodiments suitable for implementing this disclosure. Figure 10 An example of a general-purpose processing cluster within a parallel processing unit;

[0020] Figure 11B The present invention illustrates at least some embodiments suitable for implementing this disclosure. Figure 10 Example memory partitioning unit of a parallel processing unit;

[0021] Figure 12A The present invention illustrates at least some embodiments suitable for implementing this disclosure. Figure 11A An example of streaming multiprocessors;

[0022] Figure 12B Is using Figure 10 An example conceptual diagram of a processing system implemented by a PPU is provided, which is applicable to implementing at least some embodiments of this disclosure;

[0023] Figure 12C Example systems are shown that can implement various architectures and / or functions of various embodiments;

[0024] Figure 13 An example ray tracing pipeline suitable for implementing at least some embodiments of this disclosure is shown;

[0025] Figure 14 Example acceleration structures suitable for implementing at least some embodiments of this disclosure are shown;

[0026] Figure 15 Example shader records suitable for implementing at least some embodiments of this disclosure are shown;

[0027] Figure 16 This is a block diagram of an example computing device applicable to implementing some embodiments of the present disclosure; and

[0028] Figure 17 This is a block diagram of an example data center applicable to implementing some embodiments of this disclosure. Detailed Implementation

[0029] Systems and methods related to resource sharing in content delivery systems and applications are disclosed. For example, to share storage and / or resources (such as textures, shader code, mesh data, machine learning model weights, or any other application resources), the system can determine whether resources created on behalf of application instances are suitable for sharing. That is, the system can determine whether application resources are shareable or non-shareable resources. In some examples, shareable resources may include or otherwise correspond to static resources associated with the application, such as images, textures, shader code, mesh data, machine learning model weights, or any other static resources. On the other hand, non-shareable resources may include or correspond to dynamic resources associated with the application, such as render target resources or any other dynamic resources.

[0030] As described herein, in certain circumstances, applications may include games or game streaming applications, video streaming applications, machine control applications, machine motion applications, machine driving applications, synthetic data generation applications, model training applications, perception applications, augmented reality applications, virtual reality applications, mixed reality applications, robotics applications, security and supervision applications, autonomous or semi-autonomous machine applications, deep learning applications, environment simulation applications, applications for performing machine simulations, data center processing applications, generative artificial intelligence applications, applications using (large) language models, conversational AI applications, optical transport simulation applications (e.g., ray tracing, path tracing, etc.), collaborative content creation applications for 3D assets, digital twin system applications, cloud computing applications, and / or other types of applications or services.

[0031] In some examples, to determine whether an application resource is shareable or not, the system can retrieve the metadata associated with that resource. For instance, the system can determine whether a resource is likely to be shared by examining attributes in the metadata used to create the resource. In some cases, if the metadata contains any attributes that indicate the resource's content may be dynamic, the resource may be identified as a non-shareable resource. For example, a rendering target resource might be non-shareable because its content is likely to change frequently. In some examples, the system can evaluate attributes in the metadata to determine categories associated with the resource that can indicate whether the resource is a shareable static resource or a non-shareable dynamic resource.

[0032] As described in this document, in some cases, the system can allocate a portion of virtual memory to bind to a shared resource. Virtual memory can serve as a binding target specifically for shared resources, and in some cases, all shared resources can only be bound to virtual memory. During memory allocation for application resources, the system can determine whether to allocate virtual memory or physical memory. For example, an application instance can submit a request to the memory allocation API to allocate memory for a resource. The system can determine whether the request is for shared or non-shared memory. If the request is for non-shared memory, the system can allocate physical memory for the requested memory. However, if the request is for shared memory, the system can allocate virtual memory for the requested memory. In some examples, the system may not allocate physical memory pages during virtual memory allocation.

[0033] In some examples, the system can bind application resources to memory resources. The system can determine which category or type of resource to bind and perform a specific resource binding process / procedure based on the resource classification. For example, for non-shareable / dynamic resources, the system can bind non-shareable resources to physical memory allocations. For shareable / static resources, the system can bind shareable resources to virtual memory allocations for those resources and map the virtual memory allocations to physical memory allocations.

[0034] For example (but not limited to), after allocating virtual memory to be bound to a shared resource, the system can allocate dedicated physical memory to store the shared resource. Once allocated, the system can map the physical memory allocation used to store the shared resource to virtual memory allocations already bound (or about to be bound) to the shared resource. In some examples, the system can maintain a single physical memory allocation for the shared resource, which can be mapped to multiple different virtual memory allocations bound to the shared resource for different instances of the application. For example, a first instance of the application may have a first virtual memory allocation for the shared resource, a second instance of the application may have a second virtual memory allocation for the shared resource, and so on, and the first virtual memory allocation, the second virtual memory allocation, etc., can be mapped to physical memory allocations storing the shared resource.

[0035] In some examples, once the memory mapping between virtual and physical memory is complete, application instances can perform read and / or write operations on the virtual memory as usual. That is, because the virtual memory has been mapped to physical memory pages, the application can begin data transfer with the virtual memory. Furthermore, because shareable resources have dedicated physical memory allocations associated with them, sharing resources is possible, and the system can share physical memory with only one resource bound, rather than sharing memory bound to multiple resources. Such dedicated allocations may be transparently applied to application instances.

[0036] In some cases, the system may determine that one or more portions of physical memory have been allocated to store shared resources that are duplicates of each other. That is, the system may determine whether the same shared resource has been stored multiple times in physical memory. Furthermore, in such cases, the system may perform one or more operations or procedures to merge the duplicated shared resources into a single resource and a single physical memory allocation.

[0037] For example, for a shareable resource, the system can calculate an identifier corresponding to that shareable resource. This identifier may include a hash value corresponding to the shareable resource, and the system can use one or more hash algorithms to calculate the hash identifier. In some cases, the identifier can be calculated based on the content of the shareable resource. As an example, if the shareable resource is a 2D image corresponding to a texture associated with an application, the system can calculate the identifier based on the appearance of the 2D image. Thus, if the same 2D image is already stored in physical memory, the identifier can be looked up (e.g., in a database, key-value store, etc.), and the system can determine whether the shareable resource is a duplicate. The method described above for calculating hash values ​​based on file content is intended as an illustrative example. Other methods for calculating hash values ​​are also envisioned, such as generating hash values ​​from file metadata or using a combination of content and metadata, among others.

[0038] In some examples, the system may use one or more graphics processing units (GPUs) to compute resource identifiers. As described above and herein, in systems where resources and memory are managed within a driver, identifiers can be easily computed based on resource content to identify copies of the same resource. However, in systems where applications manage and populate resources, the system driver may not be able to access resource content to generate hash values ​​from the host (e.g., CPU). Therefore, in some cases, the systems of this disclosure can compute resource identifiers by offloading the operation to the GPU. In some examples, the computation of identifiers using the GPU can be performed after the application submits a command to the GPU to transfer data to shared memory.

[0039] In some examples, the system may use one or more databases to store the association between resource identifiers and physical memory allocations. For example, for an application resource stored in physical memory, the system may store in the database an identifier corresponding to the application resource and data in the database indicating the portion of physical memory allocated to store the application resource (e.g., location, address, etc.). Therefore, to determine whether at least one instance of an application resource is stored in physical memory, the system can query the database using the application identifier of that resource. If the application identifier appears multiple times in the database, and / or multiple physical memory allocations are listed as being bound to the application resource corresponding to that application identifier, the system can determine that multiple copies of the shareable resource exist.

[0040] As described herein, the system of this disclosure can merge multiple instances of duplicate shareable resources and / or their corresponding physical memory allocations. For example, if the system determines that multiple allocations of physical memory have been reserved for the same shareable resource, the system can migrate or otherwise remap all virtual memory allocations of the shareable resource to the same physical memory allocation for the shareable resource. After the migration and / or remapping is complete, the system can release any redundant or excess physical allocations of the copy of the shareable resource.

[0041] For example, but not limited to, a first instance of the application may submit a request to create a resource and allocate memory for storing that resource. Based on this request, the system of this disclosure may use the techniques described herein to determine that the requested resource is a shareable resource, allocate virtual memory to be bound to the shareable resource, and map the virtual memory allocation to a first portion of physical memory already allocated for storing the new shareable resource. After these operations are completed, as described further in detail herein, the system may compute an identifier for the shareable resource and use that identifier to query a database to determine whether a second portion of physical memory has been allocated to store the shareable resource (e.g., a copy of the requested resource). If the system determines based on the query that the new resource is actually a duplicate of a previous resource already stored in the second portion of physical memory, the system may remap the virtual memory allocation associated with the first instance of the application from the first portion of physical memory to the second portion of physical memory and release the first portion of physical memory so that it can be used / reused to store other data.

[0042] In some examples, if the system determines that a newly created / stored resource is not a duplicate, it can update the database to indicate the allocation of a portion of physical storage for storing that resource. For example, the system can store data in the database indicating the association between an identifier corresponding to the resource and a portion of physical storage that has been allocated and / or is storing that resource. This allows the system to later query the database when creating a new resource to determine whether the new resource is a duplicate of another resource already stored in physical storage.

[0043] In at least one embodiment, the system can detect memory aliasing (e.g., API-level aliasing) and / or dynamic changes to the contents of physical memory, and in response, avoid sharing (or stop sharing) these memory resources. As described herein, if a resource is already shared and is determined to be experiencing memory aliasing, the system of this disclosure can release the resource from sharing. Furthermore, or alternatively, the system can detect whether shared memory has been written to and / or modified, resulting in changes to the contents stored in physical memory. In this case, the system can stop sharing the resource, avoid sharing the resource, or otherwise release the resource so that it is not shared. In some cases, to stop sharing a resource, the system can transparently convert the resource to an instance-local allocation and copy the currently associated shared content to it.

[0044] The systems and methods described herein can be used for a variety of purposes, such as, but not limited to, machine control, machine motion, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, resource sharing between applications and / or services hosted on data center infrastructure, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation of 3D assets, cloud computing, generative AI, (large-scale) language models, and / or any other suitable application.

[0045] The disclosed embodiments can be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, marine systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for application resource sharing, systems implemented using edge devices, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations, systems for performing optical transmission simulations, systems for performing collaborative content creation for 3D assets, systems implemented at least partially using cloud computing resources, systems for performing generative AI operations, systems for performing operations using large language models, and / or other types of systems.

[0046] refer to Figure 1 , Figure 1This is a data flow diagram illustrating an example of a resource sharing process 100 for a content delivery system and application according to certain embodiments of this disclosure. It should be understood that this and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to the arrangements and elements shown, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or combined with other components, and implemented in any suitable combination and location. The various functions performed by the entities described herein can be performed by hardware, firmware, and / or software. For example, various functions can be implemented by a processor executing instructions stored in memory.

[0047] In addition to additional or alternative components, process 100 can be implemented using application 102, graphical application programming interface (API) 104, and resource manager 106. The resource manager may include classifier 108, memory type checker 110, memory allocator 112, binding process determiner 114, resource binder 116, mapper 118, resource identifier (ID) generator 120, duplicate resource detector 122, and resource consolidator 124. Furthermore, graphical API 104 may include resource creation API 126, memory allocation API 128, resource binding API 130, and resource population API 132.

[0048] In general, process 100 may include: application 102 requesting the creation of a resource using resource creation API 126, and resource manager 106 classifier 108 determining the category associated with the resource. Application 102 may also request memory allocation for the resource using memory allocation API 128. Memory type checker 110 determines whether the requested memory is shareable or non-shareable, and memory allocator 112 generates allocation command 134, allocating one or more portions of physical memory 138 if the requested memory is non-shareable, and allocating one or more portions of virtual memory 136 if the requested memory is shareable. Application 102 may also request binding the resource to memory using resource binding API 130. Binding process determiner 114 determines the binding process to use based on whether the resource / memory is shareable or non-shareable. Resource binder 116 executes the selected binding process to bind the resource to memory. For virtual memory, mapper 118 can generate mapping data 140 to map virtual memory allocation 136 to physical memory allocation 138 storing the resource. Application 102 can also use resource population API 132 to request resource population. Resource ID generator 120 can calculate an identifier for the populated resource, which duplicate resource detector 122 can use to determine whether a duplicate of the resource is stored in physical memory. If a duplicate exists, resource consolidator 124 can consolidate physical memory allocations / duplicate resources and send deallocation command 142 to release one or more portions of physical memory 138 allocated for storing the resource, while mapper 118 updates mapping data 140.

[0049] In one or more embodiments, application 102 may represent multiple application instances running on a virtual machine. Application 102 may include games, video streaming applications, machine control applications, machine motion applications, machine driving applications, synthetic data generation applications, model training applications, perception applications, augmented reality applications, virtual reality applications, mixed reality applications, robotics applications, security and monitoring applications, autonomous or semi-autonomous machine applications, deep learning applications, environment simulation applications, data center processing applications, generative AI applications, applications using (large) language models, conversational AI applications, optical transport simulation applications (e.g., ray tracing, path tracing, etc.), collaborative content creation applications for 3D assets, digital twin system applications, cloud computing applications, and / or other types of applications or services.

[0050] Application 102 may include mobile applications, computer applications, console applications, tablet applications, and / or other types of applications. Application 102 may include instructions that, when executed by a processor (e.g., Figure 16 When executed by the CPU 1606 and / or GPU 1608 described in the example, these instructions cause the processor (but not limited to) to configure, modify, update, transfer, process, and / or manipulate GPU state data, receive input data representing user input to one or more input devices, retrieve at least a portion of application data from memory, receive at least a portion of application data from a server, and / or cause data (e.g., image and / or video data) corresponding to the GPU state data to be displayed on one or more displays. In one or more embodiments, application 102 may be used as a facilitator to enable interaction with an application instance hosted on an application server using a client device and to view the output of the application instance.

[0051] In some embodiments, application 102 can be used to perform simulations in a simulated environment (e.g., NVIDIA's DriveSIM) using simulated data (e.g., simulated sensor data from simulated sensors of a virtual machine or simulated machine). These simulations can be used to test the performance of algorithms, systems, and / or processes before deploying them to real-world scenarios. In some cases, application 102 can be used to generate synthetic training data for optimizing one or more models (e.g., machine learning models, neural networks, etc.). In some embodiments, application 102 can be a 3D content collaboration application (e.g., NVIDIA's OMNIVERSE) for industrial digitization, generative physics AI, and / or other use cases, applications, or services. For example, the content collaboration application or system may include systems that use or develop generic scene descriptors (USD) (e.g., OpenUSD) data to manage objects, features, scenes, etc., in simulated environments, digital environments, etc. The application may include real-world physics simulations, such as using NVIDIA's PhysX SDK, to simulate real-world physics and physical interactions with simulations hosted by the application. This application can integrate OpenUSD with ray tracing / path tracing / light transport simulations (such as NVIDIA's RTX rendering technology) into software tools and simulation workflows for building, training, deploying, or testing AI systems, such as systems for testing, validating, training (e.g., machine learning models, neural networks, etc.) and / or other tasks related to automobiles, robots, machines, or other applications.

[0052] In various examples, for the purpose of sharing memory and / or resources (such as textures, shader code, mesh data, or any other application resources), the classifier 108 of the resource manager 106 can be configured to determine whether resources created on behalf of an application 102 instance are suitable for sharing. That is, the classifier 108 can determine whether the resources of application 102 are shareable or non-shareable resources. In some examples, shareable resources may include or otherwise correspond to static resources associated with the application, such as images, textures, shader code, mesh data, or any other static resources. On the other hand, non-shareable resources may include or correspond to dynamic resources associated with the application, such as render target resources or any other dynamic resources. Therefore, the classifier 108 can determine whether a resource is a static or dynamic resource.

[0053] In some examples, to determine whether a resource is shareable or not, classifier 108 can retrieve the metadata associated with the resource. For example, classifier 108 can determine whether the resource is likely to be shared by examining attributes in the metadata used by the resource creation API 126 to create the resource. In some cases, if the metadata contains any attributes indicating that the resource content may be dynamic, classifier 108 can identify the resource as a non-shareable resource. As an example, a rendering target resource might be non-shareable because its content is likely to change frequently. In some examples, classifier 108 can evaluate attributes in the metadata to determine a classification associated with the resource, and these classifications can indicate whether the resource is a shareable static resource or a non-shareable dynamic resource.

[0054] For example, Figure 2An example process 200 for determining a classification associated with a resource according to some embodiments of this disclosure is illustrated. As shown, application 102 (which may represent an instance of application 102 running on one or a set of servers) may invoke resource creation API 126 to create a resource. Classifier 108 may obtain resource metadata 202 associated with resource creation. In some examples, classifier 108 may evaluate the attributes contained in the resource metadata 202 to determine whether the created resource is shareable (e.g., static) or non-shareable (e.g., dynamic). For example, if resource metadata 202 contains one or more attributes (e.g., more than a threshold number of attributes) indicating that the resource content may be dynamic, classifier 108 may identify the resource as a non-shareable resource. For example, if resource metadata 202 includes attributes that are generally texture-related (e.g., size, format, type, etc.), classifier 108 may determine that the resource is a shareable resource. In some examples, classifier 108 may generate classification data 204 associated with the resource. Categorical data 204 may indicate the category of a resource, the confidence level associated with the category (e.g., the confidence level that the resource is shareable or non-shareable), or any other information associated with the resource. In some examples, memory type checker 110 and / or memory allocator 112 may use categorical data when determining what type of memory to allocate to a resource, as described further in detail herein.

[0055] review Figure 1 For example, process 100 may include application 102 requesting the allocation of memory for storing resources. For instance, application 102 may submit a request to the memory allocation API 128 of the graphics API 104 for allocating memory for the resource. In some examples, memory type checker 110 may determine the type of memory to be allocated. For example, memory type checker 110 may determine whether to allocate physical memory or virtual memory for the resource. In some cases, memory type checker 110 may determine the type of memory to allocate based on the resource classification determined by classifier 108 and / or based on the memory type requested by application 102. For example, application 102 may request the allocation of shareable or non-shareable memory for the resource. Furthermore, or alternatively, classification data 204 may indicate whether the resource is shareable or non-shareable, and memory type checker 110 may determine whether to allocate only physical memory or allocate virtual memory based on the resource's classification.

[0056] Memory allocator 112 can determine, based on memory type checker 110, whether the allocation is for physical memory 138 or virtual memory 136, and submit allocation command 134 to allocate portions of the corresponding type of memory. As described herein, in some cases, memory allocator 112 of resource manager 106 can initially allocate portions of virtual memory 136 to bind to a shared resource. Virtual memory 136 can be used as a binding target specifically for shared resources, and in some cases, all shared resources can only be bound to virtual memory 136. That is, if the request is for non-shareable memory, memory allocator 112 can submit allocation command 134 to physical memory 138 to allocate portions of physical memory 138. However, if the request is for shared memory, memory allocator 112 can submit allocation command 134 to virtual memory 136 to allocate portions of virtual memory 136.

[0057] For example, Figure 3 Example process 300 for determining the type of memory to be allocated for application resources, according to some embodiments of this disclosure, is illustrated. Process 300 may include application 102 requesting memory using a memory allocation API. Memory type checker 110 may determine the type of memory to be allocated based on the type of memory requested by application 102. Alternatively, memory type checker 110 may determine the type of memory to be allocated based at least on classification data 204. If memory allocator 112 determines that shared memory needs to be allocated, memory allocator 112 may submit allocation command 134A to virtual memory 136 to allocate one or more portions of virtual memory 136. However, if memory allocator 112 determines that non-shared memory needs to be allocated, memory allocator 112 may submit allocation command 134B to physical memory 138 to allocate one or more portions of physical memory 138.

[0058] review Figure 1For example, process 100 may include application 102 using resource binding API 130 to bind created resources to allocated memory. In some examples, binding process determiner 114 may determine what category or type of resource to bind and, based on the resource category / memory type, cause resource binder 116 to execute a specific resource binding process / procedure. For example, if binding process determiner 114 determines that the resource to be bound contains a non-shareable / dynamic resource, resource binder 116 may execute a regular resource binding process to bind the non-shareable resource to the allocation in physical memory 138. Conversely, if binding process determiner 114 determines that the resource to be bound contains a shareable / static resource, resource binder 116 may bind newly created shareable resources to the allocation in virtual memory 136 for those resources. Then, for the shareable resource, mapper 118 may generate mapping data 140 to map the allocation in virtual memory 136 to the allocation in physical memory 138.

[0059] By way of example (but not limited to), after allocating one or more portions of virtual memory 136 to be bound to a shared resource, memory allocator 112 may also allocate dedicated physical memory 138 for storing the shared resource. Once the allocation is complete, mapper 118 may map the physical memory 138 allocation for storing the shared resource to the virtual memory 136 allocation already bound (or about to be bound) to the shared resource. In some examples, resource manager 106 may maintain a single physical memory 138 allocation for the shared resource, which may be mapped to multiple different virtual memory 136 allocations bound to the shared resource for different instances of application 102. For example, a first instance of application 102 may have a first virtual memory 136 with a shared resource allocation, a second instance of application 102 may have a second virtual memory 136 with a shared resource allocation, and so on, and the allocation of the first virtual memory 136, the allocation of the second virtual memory 136, etc., may be mapped to the allocation of physical memory 138 for storing the shared resource.

[0060] For example, Figure 4An example of process 400 for determining a binding process to bind a resource to memory according to certain embodiments of the present disclosure is shown. As shown, application 102 may call resource binding API 130 to bind a resource to memory, and binding process determiner 114 may determine whether a first binding process 402A or a second binding process 402B should be used to bind the resource to memory. For example, binding process determiner 114 may determine whether the resource to be bound is shareable or non-shareable based on classification data 204, based on the requested memory for the resource (e.g., whether the requested memory is shareable or non-shareable), based on the allocated memory for the resource (e.g., whether the allocated memory for the resource is physical memory 138 or virtual memory 136), etc. If binding process determiner 114 determines that the resource to be bound is non-shareable, resource binder 116 may execute the first binding process 402A, which may be a conventional binding process to bind the resource to physical memory. However, if binding process determiner 114 determines that the resource to be bound is shareable, resource binder 116 may execute the second binding process 402B.

[0061] As part of the second binding process 402B, memory allocator 112 may submit allocation command 134 to allocate one or more portions of physical memory 138 for storing the shared resource. For example, as described above, memory allocator 112 may initially allocate one or more portions of virtual memory 136 for binding to the shared resource; therefore, in the second binding process 402B, memory allocator 112 may allocate one or more portions of physical memory 138 for actually storing the shared resource. Mapper 118 may then generate mapping data 140 and map the allocated portions of virtual memory 136 to the allocated portions of physical memory 138.

[0062] In some examples, once memory mapping is completed between virtual memory 136 and physical memory 138, application 102 can perform read and / or write operations on virtual memory 136 as usual. That is, since virtual memory 136 has mapped one or more pages of physical memory 138, application 102 can begin data transfer with virtual memory 136. Furthermore, since the shareable resource has a dedicated physical memory allocation associated with it, the resource can be shared, and the systems of this disclosure can share physical memory 138 with only one resource binding, rather than sharing physical memory 138 with multiple resources bound.

[0063] review Figure 1For example, in some cases, resource manager 106 can determine that one or more portions of physical memory 138 have been allocated to store shared resources that are duplicates of each other. That is, resource manager 106 can determine whether the same shared resource has been stored multiple times in physical memory 138. Furthermore, in this case, resource manager 106 can perform one or more operations or procedures to merge duplicate shared resources into a single resource and a single physical memory allocation.

[0064] For example, for a shareable resource, resource ID generator 120 can calculate an identifier corresponding to the shareable resource. This identifier may include a hash value corresponding to the shareable resource, and resource ID generator 120 may use one or more hash algorithms to calculate the hash identifier. In some cases, the identifier may be calculated based on the content of the shareable resource. For example, if the shareable resource is a 2D image corresponding to a texture associated with an application, resource ID generator 120 can calculate the identifier based on the appearance of the 2D image. Thus, if the same 2D image is already stored in physical memory 138, duplicate resource detector 122 can use the identifier to query the identifier in physical memory 138 and / or a database to determine if the shareable resource is a duplicate.

[0065] In some examples, one or more graphics processing units (GPUs) may be used to compute resource identifiers. As described above and herein, in systems where resources and memory are managed by a driver, identifiers can be easily computed based on resource content to identify copies of the same resource. However, in systems where resources are managed and populated by application 102, the system driver may not be able to access resource content to generate hash values ​​from the host (e.g., CPU). Therefore, resource manager 106 may compute resource identifiers by offloading the operation to the GPU. In some examples, the computation of identifiers using the GPU may be performed after the application submits a resource population command to the GPU to transfer data to shared memory.

[0066] As described herein, resource consolidator 124 can consolidate multiple instances of duplicate shareable resources and / or their corresponding physical memory allocations. For example, if duplicate resource detector 122 determines that multiple allocations of physical memory 138 have been reserved for the same shareable resource, resource consolidator 124 can initiate a migration or otherwise cause all virtual memory allocations of the shareable resource to be remapped to the same physical memory allocation for that shareable resource. After the migration and / or remapping is complete, resource consolidator 124 can submit a deallocation command 142 to cause physical memory 138 to release redundant or redundant physical allocations for copies of the shareable resource.

[0067] For example, Figure 5 An example of a process 500 for consolidating duplicate resources according to certain embodiments of the present disclosure is shown. Figure 5 As shown in the example, application 102 can use the resource population API 132 to populate a newly created shareable resource. Resource ID generator 120 can evaluate resource data 502 corresponding to the resource and calculate identification data 504 associated with the resource. For example, if the resource is an image corresponding to a texture, resource ID generator 120 (which may correspond to or be executed using a graphics processing unit) can use a hash algorithm to calculate a hash identifier for the image. In some examples, this might involve reading the resource file, converting the resource to a format suitable for hashing, and feeding that data to the hash algorithm.

[0068] The duplicate resource detector 122 can query 506 one or more databases 508 using identification data 504 to obtain identifiers. For example, database 508 may store associations between resource identifiers (e.g., hash values) and locations or portions of physical memory 138 allocated to the storage of the resource corresponding to that identifier. If the result of query 506 is that the identifier is not in database 508, the identifier can be added to database 508 and associated with its physical memory allocation, thereby avoiding resource duplication. On the other hand, if the result of query 506 is that the identifier is included in database 508, resource consolidator 124 can initiate resource / memory consolidation. In some examples, the duplicate resource detector 122 may query physical memory 138 instead of database 508.

[0069] To consolidate resources / memory, resource consolidator 124 can submit a deallocation command 142 to physical memory 138, indicating that portions of physical memory 138 can be released and later reallocated for storing other non-duplicate resources. In some examples, mapper 118 can update mapping data 140 to remap the allocation of virtual memory 136 to a single (one or more) allocation of physical memory 138 storing that resource.

[0070] As described above, if the duplicate resource detector 122 determines that the newly created / stored resource is not a duplicate, process 500 may include updating database 508 to indicate the portion of physical memory 138 (not shown) allocated for storing the resource. For example, the duplicate resource detector 122 (and / or other components) may store data in database 508 indicating the association between an identifier corresponding to the resource and the portion of physical memory 138 that has been allocated and / or is storing the resource. In this way, the duplicate resource detector 122 may later query database 508 506 when a new resource is created to determine whether the new resource is a duplicate of another resource already stored in physical memory 138.

[0071] review Figure 1 Examples, in various cases, Figure 1 The examples described may include one or more machine learning models, or may be implemented using one or more machine learning models. Machine learning models may include any type of machine learning model, such as those using linear regression, logistic regression, decision trees, support vector machines (SVM), Naive Bayes, k-nearest neighbors (Knn), K-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., autoencoders, convolutions, recursion, perceptrons, long / short-term memory (LSTM), Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial, liquid machines, etc.), and / or other types of machine learning models.

[0072] Now for reference Figure 6A , Figure 6A A hierarchical view of shareable resources and their corresponding physical and virtual memory allocations according to certain embodiments of the present disclosure is shown. For example, a first physical memory allocation 604(1) may be used to store a first shareable resource 602(1), which has a first mapping range 606(1) pointing to a first portion of virtual memory 608. Similarly, a second physical memory allocation 604(2) may be used to store a second shareable resource 602(1), which has a second mapping range 606(2) pointing to a second portion of virtual memory 608. Furthermore, although not shown, the first physical memory allocation 604(1) and the second physical memory allocation 604(2) may be mapped to one or more other portions of one or more other virtual memories, respectively. For example, virtual memory 608 may be associated with a first instance of an application (e.g., application 102), while other virtual memories may be associated with one or more other instances of the application.

[0073] Now for reference Figure 6B , Figure 6BA hierarchical view of an example of memory aliasing according to some embodiments of this disclosure is shown. Memory aliasing may occur when a first resource 610(1) is mapped to a first portion of memory 612 and shares an overlap 614 of memory 612 with a second resource 610(2). That is, the first resource 610(1) is mapped to or uses the first portion of memory 612 for storage, while the second resource is mapped to or uses the second portion of memory 612 for storage, and the first and second portions of memory 612 at least partially overlap each other. In this case, memory aliasing occurs when an application binds multiple resources to the same or “overlapping range” 614 of memory 612. This may indicate that the application may update the contents of resource 610 in the future, making it a non-static resource. Therefore, these resources 610 may not be shareable. As described herein, if any resource is already in a shared state and is determined to be subject to memory aliasing, the system of this disclosure may release the resource so that it is no longer shared. The system driver can transparently convert resources into instance-local allocations and copy currently associated shared content into them.

[0074] Now for reference Figures 7 to 9 Each block of methods 700, 800, and 900 described herein contains a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be executed by a processor executing instructions stored in memory. These methods can also be embodied as computer-usable instructions stored on a computer storage medium. These methods can be provided by standalone applications, services, or managed services (standalone or in combination with other managed services) or plug-ins of other products, to name a few. Furthermore, methods 700, 800, and 900 are combined with... Figure 1 The systems described herein are given by way of example. However, these methods may be additionally or alternatively implemented by any single system or any combination of systems, including but not limited to the systems described herein.

[0075] Figure 7 This is a flowchart illustrating an example method 700 according to some embodiments of the present disclosure, which can be performed in association with sharing resources in a multi-application environment using a graphics API to control memory allocation, resource creation, and / or resource binding. Method 700 may include, at block B702, creating a shareable resource. For example, a resource creation API 126 of graphics API 104 may create a shareable resource based at least on a request associated with an instance of application 102. In some examples, a classifier 108 of resource manager 106 may determine whether a created resource is shareable based at least on evaluating attributes contained in the metadata used to create the resource.

[0076] Method 700 at block B704 may include: allocating virtual memory. For example, memory allocator 112 may allocate one or more portions of virtual memory 136 for binding to a shared resource. In some examples, memory allocator 112 may allocate virtual memory based at least on memory type checker 110 determining that application 102 requests allocation of shared type memory. Additionally, or alternatively, memory allocator 112 may allocate virtual memory based at least on classifier 108 determining a classification associated with the created resource. This classification may indicate whether the resource is a static resource or otherwise shareable. In some examples, virtual memory 136 may be used as a binding target specifically for shared resources, and in some cases, all shared resources may be bound to virtual memory 136 only. In some examples, shared resources may be bound to portions of virtual memory 136 allocated for binding to shared resources. For shared / static resources, resource binder 116 may bind the shared resource to a virtual memory allocation and map the virtual memory allocation to a physical memory allocation.

[0077] Therefore, method 700 at block B706 may include: allocating physical memory. For example, memory allocator 112 may allocate one or more portions of physical memory 138 for the shared resource. That is, after allocating a portion of virtual memory 136 to be bound to the shared resource, resource manager 106 may allocate a portion of dedicated physical memory 138 for storing the shared resource. Then, at block B708, method 700 may include: mapping virtual memory to physical memory. For example, mapper 118 may map a portion of virtual memory 136 bound to the shared resource to a portion of physical memory 138 allocated for storing the shared resource.

[0078] In block B710, method 700 may include: populating a shareable resource and calculating an identifier corresponding to the shareable resource. For example, application 102 may submit a request or command to resource population API 132 of graphics API 104 to populate a newly created shareable resource. In some examples, once the memory mapping between virtual memory 136 and physical memory 138 is complete, application 102 can perform read and / or write operations on virtual memory 136 as usual. That is, since the virtual memory has mapped pages of physical memory 138 to virtual memory 136, application 102 can begin data transfer with virtual memory 136. Furthermore, resource ID generator 120 may calculate an identifier corresponding to the shareable resource, at least based on the shareable resource being populated. This identifier may include a hash value corresponding to the shareable resource, and resource ID generator 120 may use one or more hash algorithms to calculate the hash identifier.

[0079] In some examples, the resource ID generator 120 may be executed using one or more graphics processing units (GPUs) to compute identifiers for shareable resources. As described above and herein, in systems where resources and memory are managed by a driver, identifiers can be easily computed based on resource content to identify copies of the same resource. However, in systems where resources are managed and populated by an application, the system driver may not be able to access resource content to generate hash values ​​from the host (e.g., CPU). Therefore, in some cases, the systems of this disclosure can compute resource identifiers by offloading the operation to the GPU. In some examples, the resource ID generator 120 may generate identifiers as a side effect of moving memory. For example, identifiers may be maintained in a database associated with physical memory, and the value of the identifier is computed each time a memory transfer affects the content of that memory. In some cases, this may be performed automatically by the transfer mechanism in the GPU, mechanisms used by the GPU to facilitate the transfer, and / or another system component (e.g., running a checksum calculator that automatically fills a table with checksums for each 64k / 2MB / etc. portion of memory transferred).

[0080] Method 700 at block B712 may include querying a database for duplicate resources using an identifier. For example, duplicate resource detector 122 may use the identifier to query a database for duplicate resources. In some examples, the database may be used to store at least data indicating the association between a resource identifier and a portion of physical memory 138 allocated to a shareable resource. For example, for a resource, the database may store data indicating an identifier corresponding to that resource and data in physical memory 138 allocated for storing that resource (e.g., location, address, etc.). Therefore, to determine whether at least one instance of a shareable resource is stored in physical memory 138, duplicate resource detector 122 may use the identifier to query a database for a shareable resource.

[0081] Method 700 at block B714 may include determining whether a duplicate resource exists. For example, duplicate resource detector 122 may determine, based on a query, whether the newly created shareable resource is a duplicate (e.g., a copy) of another resource already stored in physical memory 138. In some examples, if an identifier appears once or more in the database, and / or one or more portions of physical memory 138 are listed as bound to a shareable resource corresponding to the identifier queried, then duplicate resource detector 122 may determine that one or more copies of the shareable resource exist. If it is determined at block B714 that the newly created shareable resource is the original resource (e.g., no other copies or duplicates are stored in physical memory 138), then method 700 may proceed to block B716. On the other hand, if it is determined that the newly created shareable resource is a duplicate resource, then method 700 may proceed to step B718.

[0082] Method 700 may include updating a database at step B716. For example, resource manager 106 may update the database to include data indicating the association between identifiers of shared resources and portions of physical memory 138 allocated for storing shared resources. Thus, duplicate resource detector 122 may later query the database when a new resource is created to determine whether the new resource is a duplicate of one or more other resources already stored in physical memory 138.

[0083] Method 700 may include, at step B718, remapping virtual memory. For example, virtual memory may be remapped to an existing physical memory allocation used for duplicated resources. In some cases, mapper 118 may remap a portion of virtual memory 136 bound to a newly created shareable resource to an allocated portion of physical memory 138 storing the shareable resource. In other words, virtual memory 136 may be remapped from a portion of physical memory 138 allocated in block B706 to one or more portions of physical memory 138 that already stored copies of the shareable resource prior to block B706.

[0084] Method 700 at block B720 may include: releasing physical memory. For example, once remapping is complete, resource integrator 124 may cause a portion of physical memory 138 allocated in block B706 to be released. In some examples, resource integrator 124 may submit deallocation command 142 to release one or more portions of physical memory 138. In this way, the released portions of physical memory 138 can be reused or reallocated to store other resources or data, thereby allowing the server to achieve higher density (e.g., running more instances of application 102 on a single server or group of servers) and higher utilization of computing resources.

[0085] Figure 8 This is a flowchart illustrating an example method 800 for remapping virtual memory from a first physical memory allocation to a second physical memory allocation according to some embodiments of the present disclosure. Method 800 may include, at block B802, determining one or more categories of one or more resources associated with a first instance of an application running on one or more servers. For example, classifier 108 may determine the category of resources associated with a first instance of an application 102 that may be running on a server. In some examples, classifier 108 may determine the category based at least on metadata corresponding to the resource. As an example, classifier 108 may evaluate attributes contained in the metadata used to create the resource. In some examples, if at least one of the attributes indicates that the resource is a dynamic resource, classifier 108 may classify the resource as a non-shareable resource or a dynamic resource. Otherwise, if none of the attributes indicate that the resource is a dynamic resource, classifier 108 may classify the resource as a shareable resource or a static resource.

[0086] Method 800 at block B804 may include: allocating one or more portions of virtual memory to bind to at least one resource within the resource. For example, memory allocator 112 may allocate one or more portions of virtual memory 136 to bind to the at least one resource. In some examples, the allocation of one or more portions of virtual memory may be based at least on classification. For example, if the classification indicates that the resource is a shareable resource, one or more portions of virtual memory may be allocated. Otherwise, if the resource is a non-shareable resource, physical memory 138 may be allocated. In some examples, the allocation of virtual memory may be based at least on the type of memory requested by the application or the type of memory requested by the graphics API. For example, if the requested memory is a shareable type of memory, virtual memory may be allocated.

[0087] Method 800 at block B806 may include mapping one or more portions of virtual memory to one or more first portions of physical memory allocated for storing at least one resource. For example, mapper 118 may map one or more portions of virtual memory 136 to one or more first portions of physical memory 138. In some examples, one or more first portions of physical memory 138 may be allocated at least partially in response to the allocation of one or more portions of virtual memory 136. In various examples, once the mapping is complete, the application may begin transferring data between the virtual memory and the virtual memory.

[0088] Method 800 at block B808 may include determining that the at least one resource is a duplicate resource of at least a second resource associated with one or more second instances of an application running on the server. For example, a duplicate resource detector 122 may determine that the at least one resource is a duplicate resource of at least a second resource associated with one or more second instances of an application 102 running on the server. In some examples, an identifier corresponding to the at least one resource may be calculated and used to query a database and / or physical storage 138 to determine whether the at least one resource is a duplicate resource.

[0089] Method 800 at block B810 may include: remapping one or more portions of virtual memory to one or more second portions of physical memory allocated for storing the at least second resource. For example, mapper 118 may remap one or more portions of virtual memory 136 to one or more second portions of physical memory 138 allocated for storing at least the second resource. In some examples, the remapping may be performed at least based on the fact that at least one resource is a duplicate resource. For example, because the first resource is a duplicate of the second resource (e.g., a copy of the second resource, identical to the second resource, etc.), the system may remap virtual memory to one or more second portions of physical memory that already store the second resource. Furthermore, the system may cause one or more first portions of physical memory to be released at least based on the remapping.

[0090] Figure 9 This is a flowchart illustrating an example method 900 for consolidating duplicate resources and freeing up physical memory allocations according to some embodiments of the present disclosure. Method 900 may include, at block B902, calculating one or more identifiers for one or more first resources created based on one or more first requests corresponding to at least one or more first instances of an application. For example, a resource ID generator 120 may calculate the identifier of the first resource. The identifier may include a hash value corresponding to the first resource, and the resource ID generator 120 may use one or more hash algorithms to calculate the hash identifier. In some examples, the calculation of the identifier may be based at least on one or more commands submitted by one or more first instances of the application to populate the first resource. Furthermore, in some examples, the resource ID generator 120 may use one or more graphics processing units (GPUs) to perform the calculation of the identifier.

[0091] Method 900 at block B904 may include querying one or more databases using an identifier. For example, the duplicate resource detector 122 may use an identifier to query a database. In some examples, querying a database may include searching for the existence of the identifier in the database.

[0092] Method 900 at block B906 may include determining, at least based on a query, that one or more first portions of memory have been allocated to store one or more second resources that duplicate the first resource. For example, duplicate resource detector 122 may determine that one or more first portions of memory have been allocated to store one or more second resources that duplicate the first resource. In some examples, this memory may correspond to physical memory 138. Furthermore, in some examples, determining that one or more first portions of memory have been allocated to store one or more second resources may be based on a query returning results indicating that an identifier is stored in a database and associated with one or more first portions of memory.

[0093] Method 900 at block B908 may include: releasing one or more second portions of memory allocated for storing one or more first resources. For example, resource integrator 124 may cause the allocation of memory for storing one or more second portions of one or more first resources to be released. In some examples, resource integrator 124 may submit deallocation command 142 to release one or more second portions of physical memory 138. In this way, the released portions of physical memory 138 can be reused or reallocated to store other resources or data, thereby allowing the server to achieve higher density (e.g., running more instances of application 102 on a single server or group of servers) and higher utilization of computing resources.

[0094] Example parallel processing architecture

[0095] Figure 10 Exemplary parallel processing unit (PPU) 1000 suitable for implementing at least some embodiments of the present disclosure is shown. In at least one embodiment, PPU 1000 is a multi-threaded processor implemented on one or more integrated circuit devices. PPU 1000 may have a latency-hidden architecture designed for parallel processing of many threads. A thread (e.g., an execution thread) may refer to an instance of a set of instructions configured to be executed by PPU 1000. In at least one embodiment, PPU 1000 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device, such as a liquid crystal display (LCD) device. In one or more embodiments, PPU 1000 may be used to perform general-purpose computing. Although a parallel processor is provided herein for illustrative purposes, it should be noted that this processor is illustrated for illustrative purposes only, and any processor may be used to supplement and / or replace this processor.

[0096] By way of illustration rather than limitation, one or more PPU 1000s can be configured to accelerate thousands of high-performance computing (HPC), data center, and machine learning applications. PPU 1000s can be configured to accelerate a wide range of deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analytics, molecular simulation, drug discovery, disease diagnosis, weather forecasting, big data analytics, optical transmission simulation, astronomy, molecular dynamics simulation, financial modeling, robotics, digital twins, synthetic data generation, factory automation, real-time language translation, online search optimization, personalized user recommendations, and more.

[0097] like Figure 10 As shown, PPU 1000 includes an input / output (I / O) unit 1005, a front-end unit 1015, a scheduler unit 1020, a job allocation unit 1025, a hub 1030, a crossbar (Xbar) 1070, one or more general purpose processing clusters (GPCs) 1050, and one or more partitioning units 1080. PPU 1000 can be connected to a host processor or other PPU 1000 via one or more high-speed NVLink 1010 interconnects. PPU 1000 can be connected to a host processor or other peripheral devices via interconnect 1002. PPU 1000 can also be connected to local memory including multiple memory devices 1004. In at least one embodiment, the local memory may include multiple dynamic random access memory (DRAM) devices. The DRAM devices may be configured as a high-bandwidth memory (HBM) subsystem, wherein multiple DRAM dies are stacked within each device.

[0098] The NVLink 1010 interconnect enables the system to expand and include one or more PPU 1000s coupled with one or more CPUs, supporting cache coherency between the PPU 1000 and the CPU, as well as CPU master control. Data and / or commands can be sent from or from the NVLink 1010 to other units of the PPU 1000 via hub 1030, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown).

[0099] I / O unit 1005 can be configured to send and receive communications (e.g., commands, data, etc.) from a host processor (not shown) via interconnect 1002. I / O unit 1005 can communicate directly with the host processor via interconnect 1002, or via one or more intermediate devices (such as memory bridges). In at least one embodiment, I / O unit 1005 can communicate with one or more other processors (e.g., one or more PPUs 1000) via interconnect 1002. In at least one embodiment, I / O unit 1005 implements a Peripheral Component Interconnect High Speed ​​(PCIe) interface for communication via a PCIe bus, and interconnect 1002 is a PCIe bus. In at least one embodiment, I / O unit 1005 can implement other types of known interfaces for communication with external devices.

[0100] I / O unit 1005 decodes data packets received via interconnect 1002. In at least one embodiment, the data packets represent commands configured to cause PPU 1000 to perform various operations. I / O unit 1005 sends the decoded commands to various other units of PPU 1000 as specified by the commands. For example, some commands may be sent to front-end unit 1015. Other commands may be sent to hub 1030 or other units of PPU 1000, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, I / O unit 1005 may be configured to route communication between and among the various logical units of PPU 1000.

[0101] In at least one embodiment, a program executed by the host processor encodes a command stream in a buffer that provides workload to the PPU 1000 for processing. The workload may include a number of instructions and data to be processed by those instructions. The buffer may be an area in memory accessible (e.g., read / write) by both the host processor and the PPU 1000. For example, I / O unit 1005 may be configured to access a buffer in system memory connected to interconnect 1002 via a memory request transmitted through interconnect 1002. In at least one embodiment, the host processor writes the command stream to the buffer and then sends a pointer to the start of the command stream to the PPU 1000. Front-end unit 1015 receives pointers to one or more command streams. Front-end unit 1015 manages one or more streams, reads commands from the streams, and forwards the commands to the respective units of the PPU 1000.

[0102] Front-end unit 1015 is coupled to scheduler unit 1020, which configures various GPCs 1050 to process tasks defined by one or more flows. Scheduler unit 1020 is configured to track status information associated with the various tasks managed by scheduler unit 1020. Status can indicate which GPC 1050 a task is assigned to, whether the task is active or inactive, the priority associated with the task, etc. Scheduler unit 1020 manages the execution of multiple tasks on one or more GPCs 1050.

[0103] Scheduler unit 1020 is coupled to job allocation unit 1025, which is configured to assign tasks to be executed on GPC 1050. Job allocation unit 1025 can track a number of scheduled tasks received from scheduler unit 1020. In at least one embodiment, job allocation unit 1025 manages a pending task pool and an active task pool for each GPC 1050. The pending task pool may include a number of time slots (e.g., 32 time slots) containing tasks assigned to be processed by a particular GPC 1050. The active task pool may include a number of time slots (e.g., 4 time slots) for tasks that are being actively processed by GPC 1050. When GPC 1050 completes the execution of a task, the task is evicted from the active task pool of GPC 1050, and one of the other tasks from the pending task pool is selected and scheduled to be executed on GPC 1050. If an active task on the GPC 1050 is idle, for example while waiting for data dependencies to be resolved, the active task can be evicted from the GPC 1050 and returned to the task pool, while another task in the task pool is selected and scheduled to be executed on the GPC 1050.

[0104] The work distribution unit 1025 communicates with one or more GPCs 1050 via an XBar (crossbar switch) 1070. The XBar 1070 is an interconnect network that couples a plurality of units of the PPU 1000 to other units of the PPU 1000. For example, the XBar 1070 can be configured to couple the work distribution unit 1025 to a specific GPC 1050. Although not explicitly shown, one or more other units of the PPU 1000 can also be connected to the XBar 1070 via a hub 1030.

[0105] Tasks are managed by scheduler unit 1020 and dispatched to GPC 1050 by work allocation unit 1025. GPC 1050 is configured to process tasks and generate results. Results may be consumed by other tasks within GPC 1050, routed to different GPCs 1050 via XBar 1070, or stored in memory 1004. Results may be written to memory 1004 via partitioning unit 1080, which implements a memory interface for reading data from and writing data to memory 1004. Results may be sent to another PPU 1004 or CPU via NVLink 1010. In at least one embodiment, PPU 1000 includes U partitioning units 1080, which is equal to the number of independent and distinct memory devices 1004 coupled to PPU 1000.

[0106] In at least one embodiment, the host processor executes a driver kernel that implements an application programming interface (API), enabling the execution of one or more applications on the host processor to schedule operations for execution on the PPU 1000. In at least one embodiment, multiple computing applications are executed concurrently by the PPU 1000, and the PPU 1000 provides isolation, Quality of Service (QoS), and independent address spaces for the multiple computing applications. Applications can generate instructions (e.g., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 1000. The driver kernel can output tasks to one or more streams being processed by the PPU 1000. Each task can include one or more associated thread groups, which may be referred to herein as a warp. In at least one embodiment, a warp includes 32 associated threads that can execute in parallel. Cooperative threads can refer to multiple threads that include instructions for performing tasks and can exchange data via shared memory.

[0107] Figure 11A At least some embodiments suitable for implementing this disclosure are shown. Figure 10 An example GPC 1050 of the PPU 1000. (e.g.) Figure 11A As shown, each GPC 1050 may include multiple hardware units for processing tasks. In at least one embodiment, each GPC 1050 includes a pipeline manager 1110, a pre-raster operation unit (PROP) 1115, a raster engine 1125, a work assignment crossbar switch (WDX) 1180, a memory management unit (MMU) 1190, and one or more data processing clusters (DPCs) 1120. It should be understood that... Figure 11A The GPC 1050 may include replacements Figure 11A Other hardware units of the unit shown or excluding Figure 11AOther hardware units besides the unit shown.

[0108] In at least one embodiment, the operation of GPC 1050 is controlled by pipeline manager 1110. Pipeline manager 1110 manages the configuration of one or more DPCs 1120 for processing tasks assigned to GPC 1050. In at least one embodiment, pipeline manager 1110 may configure at least one of one or more DPCs 1120 to implement at least a portion of the graphics rendering pipeline. For example, DPC 1120 may be configured to execute vertex shaders on programmable streaming multiprocessor (SM) 1140. Pipeline manager 1110 may also be configured to route packets received from job allocation unit 1125 to appropriate logic units within GPC 1050. For example, some packets may be routed to fixed-function hardware units in PROP 1115 and / or raster engine 1125, while other packets may be routed to DPC 1120 for processing by primitive engine 1135 or SM 1140. In at least one embodiment, pipeline manager 1110 may configure at least one of one or more DPCs 1120 to implement neural network models and / or computation pipelines.

[0109] PROP unit 1115 can be configured to route data generated by raster engine 1125 and DPC 1120 to raster operation (ROP) units. PROP unit 1115 can also be configured to perform color mixing optimization, organize pixel data, perform address translation, etc.

[0110] Raster engine 1125 may include several fixed-function hardware units configured to perform various raster operations. In at least one embodiment, raster engine 1125 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a block aggregation engine. The setup engine receives transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices. The plane equations are sent to the coarse raster engine to generate coverage information for primitives (e.g., x, y coverage masks for blocks). The output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and to the clipping engine, where fragments located outside the view frustum are clipped. Those fragments remaining after clipping and culling can be passed to the fine raster engine to generate properties of pixel fragments based on the plane equations generated by the setup engine. The output of raster engine 1125 includes fragments, for example, to be processed by a fragment shader implemented in DPC 1120.

[0111] This includes a GPC 1050, a DPC 1120 controller (MPC) 1130, a primitive engine 1135, and one or more SMs 1140. The MPC 1130 controls the operation of the DPC 1120, routing packets received from the pipeline manager 1110 to the appropriate units within the DPC 1120. For example, packets associated with vertices can be routed to the primitive engine 1135, which is configured to retrieve vertex attributes associated with the vertex from memory 1004. Conversely, packets associated with shaders can be sent to the SMs 1140.

[0112] The SM 1140 includes a programmable streaming processor configured to process tasks represented by multiple threads. Each SM 1140 is multithreaded and configured to execute multiple threads (e.g., 32 threads) from a specific thread group concurrently. In at least one embodiment, the SM 1140 implements a SIMD (Single Instruction, Multiple Data) architecture, where each thread in a thread group (e.g., a thread bundle) is configured to process a different dataset based on the same instruction set. All threads in the thread group execute the same instructions. In at least one embodiment, the SM 1140 implements a SIMT (Single Instruction, Multiple Threads) architecture, where each thread in a thread group is configured to process a different dataset based on the same instruction set, but where individual threads in the thread group are allowed to diverge during execution. In at least one embodiment, a program counter, call stack, and execution state are maintained for each thread bundle, enabling concurrency between the thread bundle and serial execution within the thread bundle when threads within the thread bundle diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, thereby achieving equal concurrency among all threads within and between thread bundles. When the execution state is maintained for each individual thread, threads executing the same instructions can be converged and executed in parallel to achieve maximum efficiency.

[0113] MMU 1190 can provide an interface between GPC 1050 and partition unit 1080. MMU 1190 can provide virtual address to physical address translation, memory protection, and memory request arbitration. In at least one embodiment, MMU 1190 provides one or more translation back buffers (TLBs) for performing translations from virtual addresses to physical addresses in memory 1004.

[0114] Figure 11B At least some embodiments suitable for implementing this disclosure are shown. Figure 10 An exemplary memory partition unit 1080 of the PPU 1000. For example... Figure 11BAs shown, the memory partition unit 1080 includes a raster operation (ROP) unit 1150, a secondary (L2) cache 1160, and a memory interface 1170. The memory interface 1170 can be coupled to the memory 1004. The memory interface 1170 can implement 32, 64, 128, or 1024-bit data buses for high-speed data transfer. In at least one embodiment, the PPU 1000 incorporates U memory interfaces 1170, one memory interface 1170 for each pair of partition units 1080, wherein each pair of partition units 1080 is connected to a corresponding memory device 1004. For example, the PPU 1000 can be connected to up to Y memory devices 1004, such as high-bandwidth memory stacks or synchronous dynamic random access memory of Graphics Dual Data Rate version 5, or other types of persistent memory.

[0115] In at least one embodiment, the memory interface 1170 implements the HBM2 memory interface, and Y is equal to half of U. In at least one embodiment, the HBM2 memory stack and PPU 1000 reside on the same physical package, significantly saving power and area compared to conventional GDDR5 SDRAM systems. In at least one embodiment, each HBM2 stack contains four memory dies, and Y is equal to 4, wherein each HBM2 stack die contains two 128-bit channels, for a total of eight channels, and the data bus width is 1024 bits.

[0116] In at least one embodiment, the memory 1004 supports single error correction double error detection (SECDED) error correction codes (ECC) to protect the data. ECC provides high reliability for computing applications sensitive to data corruption. Reliability is particularly important in large-scale cluster computing environments where the PPU 1000 needs to handle very large datasets and / or long-running applications.

[0117] In at least one embodiment, the PPU 1000 implements a multi-level memory hierarchy. In at least one embodiment, the memory partitioning unit 1080 supports unified memory, providing a single unified virtual address space for the CPU and PPU 1000 memory, thereby enabling data sharing between virtual memory systems. In at least one embodiment, the frequency of PPU 1000 accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 1000 that accesses pages more frequently. In at least one embodiment, the NVLink 1010 supports address translation services, allowing the PPU 1000 to directly access the CPU's page tables and providing the PPU 1000 with full access to the CPU's memory.

[0118] In at least one embodiment, the replication engine transfers data between multiple PPUs 1000 or between a PPU 1000 and a CPU. The replication engine can generate page faults for addresses not mapped to a page table. The memory partitioning unit 1080 can then process these page faults, mapping the addresses to a page table, after which the replication engine can perform the transfer. With the aid of hardware page faults, addresses can be passed to the replication engine without concern for whether memory pages reside, and the replication process is transparent.

[0119] Memory partition unit 1080 can retrieve data from memory 1004 or other system memory and store it in L2 cache 1160, which is located on-chip and shared by the various GPCs 1050. As shown, each memory partition unit 1080 includes a portion of the L2 cache 1160 associated with the corresponding memory device 1004. Lower-level caches can then be implemented in individual units within the GPC 1050. For example, each SM 1140 can implement a Level 1 (L1) cache. The L1 cache is a private memory dedicated to a specific SM 1140. Data from the L2 cache 1160 can be retrieved and stored in each L1 cache for processing in the functional units of the SM 1140. The L2 cache 1160 is coupled to memory interface 1170 and XBar 1070.

[0120] ROP unit 1150 performs graphic raster operations related to pixel color, such as color compression and pixel blending. ROP unit 1150 also performs depth testing in conjunction with raster engine 1125, receiving the depth of sample locations associated with pixel fragments from the culling engine of raster engine 1125. This depth is tested against the corresponding depth of the sample locations associated with the fragment in the depth buffer. If the fragment passes the depth test at the sample location, ROP unit 1150 updates the depth buffer and transmits the depth test result to raster engine 1125. It is understood that the number of partition units 1080 may differ from the number of GPCs 1050, thus each ROP unit 1150 can be coupled to each GPC 1050. ROP unit 1150 can track data packets received from different GPCs 1050 and determine which GPC 1050 the result generated by ROP unit 1150 is routed to via Xbar 1070. Although in Figure 11B ROP unit 1150 is contained within memory partition unit 1080, but in other examples, ROP unit 1150 may be located outside memory partition unit 1080. For example, ROP unit 1150 may reside in GPC 1050 or other units.

[0121] Figure 12A At least some embodiments suitable for implementing this disclosure are shown. Figure 11A An example of a streaming multiprocessor 1140. (e.g.) Figure 12A As shown, SM 1140 includes instruction cache 1205, one or more scheduler units 1210(K), register file 1220, one or more processing cores 1250, one or more special function units (SFU) 1252, one or more load / store units (LSU) 1254, interconnect network 1280, and shared memory / L1 cache 1270.

[0122] As described herein, the work allocation unit 1025 schedules tasks to execute on the GPC 1050 of the PPU 1000. Tasks can be assigned to a specific DPC 1120 within the GPC 1050, and if the task is associated with a shader program, it can be assigned to an SM 1140. The scheduler unit 1212 can receive tasks from the work allocation unit 1025 and manage the instruction scheduling of one or more thread blocks assigned to the SM 1140. The scheduler unit 1212 can schedule thread blocks to execute as thread bundles of parallel threads, wherein each thread block is assigned at least one thread bundle. In at least one embodiment, each thread bundle executes 32 threads. The scheduler unit 1212 can manage multiple different thread blocks, assign thread bundles to different thread blocks, and then dispatch instructions from multiple different cooperative groups to various functional units (e.g., core 1250, SFU 1252, and LSU 1254) during each clock cycle.

[0123] A cooperative group can be a programming model used to organize groups of communicating threads. It allows developers to express the granularity at which threads are communicating, enabling richer and more efficient parallel decompositions. Cooperative startup APIs can support synchronization between blocks of threads to execute parallel algorithms. Conventional programming models provide a single, simple structure for synchronizing cooperative threads: a barrier across all threads in a block (e.g., the `syncthreads()` function). However, programmers often want to define thread groups at a granularity smaller than that of thread blocks and synchronize within those groups, enabling higher performance, design flexibility, and software reuse through a collective group-wide function interface.

[0124] Collaboration groups enable programmers to explicitly define thread groups at both the sub-block (e.g., as small as a single thread) and multi-block granularity levels and perform collective operations, such as synchronization on threads within a collaboration group. The programming model supports clean composition across software boundaries, allowing libraries and utility functions to be safely synchronized in their local environments without making assumptions about convergence. Collaboration group primitives enable new modes of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire mesh of thread blocks.

[0125] Dispatch unit 1215 can be configured to send instructions to one or more functional units. In at least one embodiment, scheduler unit 1212 includes two dispatch units 1215, which enable scheduling of two different instructions from the same thread bundle during each clock cycle. In at least one embodiment, each scheduler unit 1212 may include a single dispatch unit 1215 or additional dispatch units 1215.

[0126] Each SM 1140 may include a register file 1220, which provides a set of registers for functional units of the SM 1140. In at least one embodiment, the register file 1220 is partitioned between each functional unit such that each functional unit is allocated a dedicated portion of the register file 1220. In at least one embodiment, the register file 1220 is partitioned between different thread bundles of rows of SM 1140. The register file 1220 provides temporary memory for operands of data paths connected to the functional units.

[0127] Each SM 1140 may include L processing cores 1250. In at least one embodiment, the SM 1140 includes a large number (e.g., 128, etc.) of different processing cores 1250. Each core 1250 may include fully pipelined, single-precision, double-precision, and / or mixed-precision processing units, which include floating-point arithmetic logic units and integer arithmetic logic units. In at least one embodiment, the floating-point arithmetic logic units implement the IEEE 754-2008 standard for floating-point arithmetic. In at least one embodiment, core 1250 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0128] Tensor cores are configured to perform matrix operations, and in at least one embodiment, one or more tensor cores are included in core 1250. Specifically, tensor cores may be configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In at least one embodiment, each tensor core operates on a 4×4 matrix and performs matrix multiplication and accumulation operations D = A×B + C, where A, B, C, and D are 4×4 matrices.

[0129] Training complex neural networks requires significant parallel computing power, including floating-point multiplication and addition supported by the PPU 1000. Inference is less computationally intensive than training; it is a latency-sensitive process where a trained neural network is applied to new inputs it has never seen before, to classify images, translate speech, and infer new information.

[0130] Neural networks heavily rely on matrix mathematical operations, and complex multi-layered networks require significant floating-point performance and bandwidth to improve efficiency and speed. The PPU 1000 boasts thousands of processing cores optimized for matrix mathematical operations and delivers tens to hundreds of TFLOPS of performance, enabling it to form a computing platform capable of providing the performance required for deep neural network-based artificial intelligence and machine learning applications.

[0131] In at least one embodiment, the matrix multiplication inputs A and B are 16-bit floating-point matrices, while the accumulation matrices C and D can be either 16-bit or 32-bit floating-point matrices. The Tensor Core performs operations on the 16-bit floating-point input data and the 32-bit floating-point accumulation. The 16-bit floating-point multiplication requires 64 operations to produce a full-precision product, which is then accumulated by adding the 32-bit floating-point products to other intermediate products of the 4×4×4 matrix multiplication. In practice, the Tensor Core can be used to perform larger two-dimensional or higher-dimensional matrix operations built from these smaller elements. APIs (such as the CUDA 9 C++ API) expose specialized matrix loading, matrix multiplication and accumulation, and matrix storage operations to efficiently utilize the Tensor Core from CUDA-C++ programs. At the CUDA level, the thread bundle-level interface assumes that the 16×16-size matrix spans all 32 threads of the thread bundle.

[0132] Each SM 1140 may also include M SFUs 1252 that perform special functions (e.g., attribute evaluation, inverse square root, etc.). In at least one embodiment, an SFU 1252 may include a tree traversal unit configured to traverse a hierarchical tree data structure. In at least one embodiment, an SFU 1252 may include a texture unit configured to perform texture map filtering operations. In at least one embodiment, a texture unit is configured to load a texture map (e.g., a 2D array of texture pixels) from memory 1004 and sample the texture map to produce sampled texture values ​​for use in a shader program executed by the SM 1140. In at least one embodiment, the texture map is stored in shared memory / L1 cache 1170. The texture unit performs texture operations, such as filtering operations using mip maps (e.g., texture maps at different levels of detail). In at least one embodiment, each SM 1140 includes two texture units.

[0133] Each SM 1140 may also include N LSUs 1254, which implement load and store operations between the shared memory / L1 cache 1270 and the register file 1220. Each SM 1140 may include an interconnect network 1280 connecting each functional unit to the register file 1220 and connecting the LSUs 1254 to the register file 1220 and the shared memory / L1 cache 1270. In at least one embodiment, the interconnect network 1280 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 1220 and to connect the LSUs 1254 to memory locations in the register file and the shared memory / L1 cache 1270.

[0134] The shared memory / L1 cache 1270 may include an on-chip memory array that allows data storage and communication between the SM1140 and the primitive engine 1135, as well as between threads within the SM1140. In at least one embodiment, the shared memory / L1 cache 1270 includes a storage capacity of 128KB and is located on the path from the SM1140 to the partition unit 1080. The shared memory / L1 cache 1270 can be used for cache reads and writes. One or more of the shared memory / L1 cache 1270, the L2 cache 1160, and the memory 1004 may be backup storage.

[0135] Combining data caching and shared memory functionality into a single memory block provides optimal overall performance for both types of memory access. This capacity can be used by the program as a cache that does not utilize shared memory. For example, if shared memory is configured to use half its capacity, texture and load / store operations can use the remaining capacity. Integration within the shared memory / L1 cache 1270 allows it to function as a high-throughput pipeline for streaming data, while simultaneously providing high-bandwidth and low-latency access to frequently reused data.

[0136] When configured for general-purpose parallel computing, a simpler configuration can be used compared to graphics processing. Specifically, Figure 10The fixed-function graphics processing unit shown can be bypassed, creating a simpler programming model. In a general-purpose parallel computing configuration, the work allocation unit 1025 can directly assign and allocate thread blocks to the DPC 1120. Threads within a block can execute the same program, using a unique thread ID in the computation to ensure each thread produces a unique result, using the SM 1140 to execute the program and perform the computation, using the shared memory / L1 cache 1270 for communication between threads, and using the LSU 1254 to read and write global memory via the shared memory / L1 cache 1270 and the memory partitioning unit 1080. When configured for general-purpose parallel computing, the SM 1140 can also write commands to the scheduler unit 1020 to start new jobs on the DPC 1120.

[0137] The PPU 1000 can be included in desktop computers, laptop computers, tablet computers, servers, supercomputers, smartphones (e.g., wireless, handheld devices), personal digital assistants (PDAs), digital cameras, vehicles, head-mounted displays, handheld electronic devices, etc. In at least one embodiment, the PPU 1000 is contained on a single semiconductor substrate. In at least one embodiment, the PPU 1000 is included on a system-on-a-chip (SoC) along with one or more other devices, such as an additional PPU 1000, memory, a Reduced Instruction Set Computer (RISC) CPU, a Memory Management Unit (MMU), a Digital-to-Analog Converter (DAC), etc.

[0138] In at least one embodiment, the PPU 1000 may be included on a graphics card, which includes one or more memory devices 1004. The graphics card may be configured to interface with a PCIe slot on a desktop computer's motherboard. In at least one embodiment, the PPU 1000 may be an integrated graphics processing unit (iGPU) or a parallel processor included in a chipset of the motherboard.

[0139] Examples of computing systems

[0140] Systems with multiple GPUs and CPUs are being used across various industries as developers expose to and leverage greater parallelism in applications such as artificial intelligence computing. High-performance GPU-accelerated systems with tens to thousands or more compute nodes are being deployed in data centers, research institutions, and supercomputers to tackle larger problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to be scaled to support this increased bandwidth.

[0141] Figure 12B This is suitable for use in implementing at least some embodiments of the present disclosure. Figure 10An exemplary conceptual diagram of a processing system 1200 implementing PPU 1000 is provided. The processing system 1200 includes a CPU 1230, a switch 1210, and each of a plurality of PPUs 1000, along with a corresponding memory 1004. An NVLink 1010 provides a high-speed communication link between each PPU 1000. Although... Figure 12B A specific number of NVLink 1010 and interconnect 1002 connections are shown, but the number of connections to each PPU 1000 and CPU 1230 can vary. A switch 1210 interfaces between the interconnect 1002 and the CPU 1230. The PPU 1000, memory 1004, and NVLink 1010 can reside on a single semiconductor platform to form a parallel processing system 1225. In at least one embodiment, the switch 1210 supports two or more protocols that interface between various different connections and / or links.

[0142] In at least one embodiment (not shown), NVLink 1010 provides one or more high-speed communication links between each PPU 1000 and CPU 1230, and switch 1210 interfaces between interconnect 1002 and each PPU 1000. PPU 1000, memory 1004, and interconnect 1002 may reside on a single semiconductor platform to form parallel processing module 1225. In at least one embodiment (not shown), interconnect 1002 provides one or more communication links between each PPU 1000 and CPU 1230, and switch 1210 uses NVLink 1010 to interface between each PPU 1000 to provide one or more high-speed communication links between PPUs 1000. In at least one embodiment (not shown), NVLink 1010 provides one or more high-speed communication links between PPU 1000 and CPU 1230 via switch 1210. In at least one embodiment (not shown), interconnect 1002 provides one or more communication links directly between each PPU 1000. One or more NVLink 1010 high-speed communication links can be implemented as physical NVLink interconnects or on-chip or die-on interconnects using the same protocol as NVLink 1010.

[0143] In the context of this specification, a single semiconductor platform can refer to a unique, single semiconductor-based integrated circuit fabricated on a bare die or chip. The term "single semiconductor platform" can also refer to a multi-chip module with increased connectivity, simulating on-chip operation and substantially improved by utilizing conventional bus implementations. Of course, various circuits or devices can also be placed separately or in various combinations of semiconductor platforms, depending on the user's needs. Optionally, the parallel processing module 1225 can be implemented as a circuit board substrate, and each of the PPU 1000 and / or memory 1004 can be a packaged device. In at least one embodiment, the CPU 1230, switch 1210, and parallel processing module 1225 reside on a single semiconductor platform.

[0144] In at least one embodiment, the signaling rate of each NVLink 1010 is 20 to 25 gigabits per second, and each PPU 1000 includes six NVLink 1010 interfaces (e.g., Figure 12B As shown, each PPU 1000 includes five NVLink 1010 interfaces. Each NVLink 1010 can provide a data transfer rate of 25 gigabits per second in each direction, with six links providing 1000 gigabits per second. When the CPU 1230 also includes one or more NVLink 1010 interfaces, the NVLink 1010 can be specifically used for applications such as... Figure 12B The example shows PPU-to-PPU communication, or a combination of PPU-to-PPU and PPU-to-CPU communication.

[0145] In at least one embodiment, NVLink 1010 allows direct load / store / atomic access from CPU 1230 to memory 1004 of each PPU 1000. In one embodiment, NVLink 1010 supports coherent operation, allowing data read from memory 1004 to be stored in the cache hierarchy of CPU 1230, reducing cache access latency of CPU 1230. In at least one embodiment, NVLink 1010 includes support for Address Translation Service (ATS), allowing PPU 1000 to directly access page tables within CPU 1230. One or more NVLink 1010s can also be configured to operate in a low-power mode.

[0146] Figure 12C An exemplary system 1265 suitable for implementing at least some embodiments of the present disclosure is shown, in which various architectures and / or functions of various prior embodiments can be implemented.

[0147] As shown in the figure, a system 1265 is provided, which includes at least one central processing unit (CPU) 1230 connected to a communication bus 1275. The communication bus 1275 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or one or more point-to-point communication protocols. The system 1265 also includes a main memory 1240. Control logic (software) and data are stored in the main memory 1240, which may take the form of random access memory (RAM).

[0148] System 1265 also includes input device 1260, parallel processing system 1225, and display device 1245, such as conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light emission diode), plasma display, etc. User input can be received from input device 1260 (e.g., keyboard, mouse, touchpad, microphone, etc.). Each of the aforementioned modules and / or devices can even reside on a single semiconductor platform to form system 1265. Optionally, depending on user needs, the individual modules can also be placed separately or in various combinations of semiconductor platforms.

[0149] In addition, system 1265 can be coupled to a network (e.g., telecommunications network, local area network (LAN), wireless network, wide area network (WAN) (such as the Internet), peer-to-peer network, cable network, etc.) via network interface 1235 for communication purposes.

[0150] System 1265 may also include secondary storage (not shown). Secondary storage 610 may include, for example, hard disk drives and / or removable storage drives, such as floppy disk drives, tape drives, optical disk drives, digital versatile disc (DVD) drives, recording devices, or Universal Serial Bus (USB) flash memory. The removable storage drive can read from and / or write to the removable storage unit.

[0151] Computer programs or computer control logic algorithms may be stored in main memory 1240 and / or secondary storage. When these computer programs are executed, they enable system 1265 to perform various functions. Memory 1240, storage, and / or any other storage are possible examples of computer-readable media.

[0152] The architectures and / or functions of the various prior drawings can be implemented in the context of general-purpose computer systems, circuit board systems, game console systems for entertainment purposes, dedicated systems, and / or any other desired systems. For example, system 1265 can take the form of a desktop computer, laptop computer, tablet computer, server, supercomputer, smartphone (e.g., wireless, handheld device), personal digital assistant (PDA), digital camera, vehicle, head-mounted display, handheld electronic device, mobile phone device, television, workstation, game console, embedded system, and / or any other type of logic.

[0153] Ray tracing pipeline

[0154] In at least one embodiment, the PPU 1000 includes a graphics processing unit (GPU). The PPU 1000 can be configured to receive commands specifying a shader program for processing graphics data. Graphics data can be defined as a set of primitives, such as points, lines, triangles, quadrilaterals, triangular strips, etc. A primitive can contain data specifying the number of primitive vertices (e.g., in model space coordinates) and attributes associated with each vertex of the primitive. The PPU 1000 can be configured to process the graphics primitives to generate framebuffers (e.g., pixel data for each pixel of a display).

[0155] Applications can write scene model data (e.g., a collection of vertices and attributes) to memory (e.g., system memory or memory 1004). This model data can define each object visible on the display. The application can then issue API calls to the driver kernel, requesting that the model data be rendered and displayed. The driver kernel can read the model data and write commands to one or more streams to perform operations for processing the model data. These commands can reference different shader programs to be implemented on the SM 1140 of the PPU 1000. For example, different SM 1140s can be configured to execute different shader programs.

[0156] In at least one embodiment, model data can be processed to perform one or more ray tracing operations (e.g., real-time ray tracing), thereby rendering the model data to a frame buffer. The contents of the frame buffer can be transferred to a display controller for display on a display device. Ray tracing can refer to various techniques used to model or simulate light transport and / or other aspects of an environment, such as for generating digital images or otherwise simulating an environment. Therefore, while some embodiments may be described with respect to light transport simulation, they can be applied to any aspect of simulating, modeling, and / or measuring an environment. Non-limiting examples of ray tracing include ray casting, recursive ray tracing, distributed ray tracing, photon mapping, and path tracing.

[0157] Ray tracing can be used to simulate various optical effects, such as shadows, reflections, refractions, scattering, ambient occlusion, global illumination, or chromatic aberration. Ray tracing can involve sampling the illumination and / or other environmental conditions of pixels by projecting rays into a virtual environment, thereby generating ray-traced samples. These ray-traced samples can be combined and used to determine the pixel colors of an image. In at least one embodiment, to conserve computational resources, lighting conditions may be sparsely sampled, resulting in noisy rendering data. Temporal accumulation can be used to increase the effective number of samples by leveraging information from previous frames. To produce a final render that approximates a fully sampled scene, one or more denoising filters can be applied to the noisy rendering data to reduce noise.

[0158] Many ray tracing algorithms project or emit light rays from a virtual camera or eye, traversing the 2D view plane (e.g., the pixel plane) and into a 3D scene that may contain one or more light sources. Some rays may arrive directly at the view plane from the light source, some may be occluded by objects in the scene to create shadows, and some may be reflected or refracted by objects before reaching the view plane. When rays intersect with objects, the color and lighting information at the intersection points on the object surfaces can affect the various pixel colors and illumination levels of pixels on the view plane. Different objects may have different surface properties, which may cause them to reflect, refract, or absorb light in different ways, which can be taken into account in ray tracing. Rays may reflect off objects and hit other objects, or propagate through the surface of transparent objects before reaching the light source, and the color and lighting information from all intersecting objects can affect the final pixel color.

[0159] Figure 13 An example ray tracing pipeline 1300 suitable for implementing at least some embodiments of the present disclosure is shown. By way of example (and not limitation), according to at least one embodiment, the ray tracing pipeline 1300 can be... Figure 10 The PPU 1000 is implemented in the ray tracing pipeline 1300. The ray tracing pipeline 1300 may include processing steps implemented to generate 2D computer-generated images from 3D geometric data using one or more ray tracing techniques.

[0160] In at least one embodiment, the ray tracing pipeline 1300 may be constructed using one or more ray generation shaders 1302, one or more arbitrary hit shaders 1304, one or more intersection shaders 1306, one or more miss shaders 1308 and / or one or more recently hit shaders 1310.

[0161] The ray tracing pipeline 1300 can be implemented by an application executed by a host processor (e.g., a CPU). In at least one embodiment, the device driver can implement an application programming interface (API) that defines various functions that the application can use to generate graphics data for display. The device driver can refer to a software program containing instructions that control the operation of the PPU 1000 or other PPUs used to implement the ray tracing pipeline 1300. The API can provide an abstraction for the programmer, enabling them to generate graphics data using dedicated graphics hardware (e.g., the PPU 1000) without requiring the programmer to use the specific instruction set of the PPU 1000. The application can contain API calls that are routed to the device driver of the PPU 1000. The device driver can interpret the API calls and perform various operations in response to them. In at least one embodiment, the device driver performs operations by executing instructions on the CPU. In at least one embodiment, the device driver performs operations at least in part by initiating operations on the PPU 1000 using an input / output interface between the CPU and the PPU 1000. In at least one embodiment, the device driver is configured to implement the ray tracing pipeline 1300 using the hardware of the PPU 1000.

[0162] Various programs can be executed within the PPU 1000 to implement different stages of the ray tracing pipeline 1300. For example, a device driver can boot a kernel on the PPU 1000 to execute the stage of implementing the ray generation shader 1302 on the SM 1140 (or multiple SM 1140s). The device driver (or the initial kernel executed by the PPU 1000) can also boot other kernels on the PPU 1000 to execute other stages of the ray tracing pipeline 1300.

[0163] The ray-generating shader 1302 may be the first shader involved in ray tracing dispatch. The ray-generating shader 1302 may call a High-Level Shading Language (HLSL) function named TraceRay(). This TraceRay() function projects a single ray into the scene to search for intersections, which may trigger other shaders in the process. In at least one embodiment, the ray-generating shader 1302 may call TraceRay() an arbitrary number of times.

[0164] Whenever TraceRay() detects a potential intersection between a ray and the scene, the arbitrary hit shader 1304 and the intersection shader 1306 can be invoked. The intersection shader 1306 determines whether the ray intersects with a single geometric primitive, such as a sphere, subdivision surface, triangle, or other form of primitive. Once an intersection is found, the arbitrary hit shader 1304 can be used to further process the intersection or may discard it. The arbitrary hit shader 1304 can use an alpha test (e.g., but not limited to) by performing a texture lookup and deciding whether to discard the intersection based on the texel value.

[0165] After TraceRay() has completed its search for intersections between rays and the scene, it can invoke either the missed shader 1308 or the most recently hit shader 1310 based on the search results. The most recently hit shader 1310 can perform most shading operations, such as material evaluation and texture lookup. For example, the missed shader 1308 can be used to perform environment lookup. In at least one embodiment, one or more of the most recently hit shader 1310 or the missed shader 1308 can recursively trace rays by calling TraceRay() itself.

[0166] The ray tracing pipeline 1300, constructed from the various shaders described herein, can define a single-ray programming model. In at least one embodiment, each thread of the PPU 1000 and / or other PPUs used to implement the ray tracing pipeline 1300 can process one ray at a time. In at least one embodiment, each thread cannot communicate with other threads or see other rays currently being processed. This simplifies shader code while allowing for vendor-specific optimizations using the API.

[0167] In at least one embodiment, different shaders and / or shader types can communicate with each other using ray payloads. A ray payload can be referenced as a user-defined structure passed as an INOUT parameter to TraceRay(). For example, any hit shader 1304, the most recently hit shader 1310, and / or the missed shader 1308 can read from and / or write to the ray payload, and thus return its computed results to the caller of TraceRay().

[0168] In at least one embodiment, the ray generation shader 1302 can trace master rays, which may include rays sent from a virtual camera into the scene. However, the ray generation shader 1302 is not limited to this function. In at least one embodiment, the ray generation shader 1302 can generate rays based on rasterized g-buffer data (e.g., for tracing reflections). Using this approach, ray tracing can be used to complement rasterization, rather than replace it.

[0169] When using traditional rasterization, only the shaders required for the currently drawn object must be active on the PPU. This allows for a relatively small rasterization pipeline object, containing only a set of vertex shaders, pixel shaders, etc. In contrast, the ray tracing pipeline 1300 can be used to shoot rays into the scene arbitrarily. This could mean that rays can hit any or multiple objects in the scene. Therefore, it's possible that all shaders for all objects could potentially be hit, and thus it might be desirable for all shaders to reside on the PPU and be ready to execute.

[0170] In at least one embodiment, state objects can be used to group shaders together for execution. At a high level, a state object for the ray tracing pipeline 1300 can be viewed as a binary executable generated by the linking step performed on all shaders compiled for the scene. Relationships between different shaders can be specified when the state object is created. For example, a triplet of intersecting shader 1306, any hit shader 1304, and / or most recently hit shader 1310 can be bundled into a hit group. The application can specify the state object of the ray tracing pipeline 1300 to be executed when calling the DispatchRays() function on the command list. The DispatchRays() function can invoke the ray generation shader 1302 for each pixel of the image. In at least one embodiment, the application can create any number of state objects for the ray tracing pipeline 1300 and can reuse pre-compiled shaders for this purpose.

[0171] Now for reference Figure 14 , Figure 14 An example acceleration structure 1400 suitable for implementing at least some embodiments of the present disclosure is shown. The acceleration structure 1400 includes one or more top-level acceleration structures (e.g., top-level acceleration structure 1402) and one or more bottom-level acceleration structures (e.g., bottom-level acceleration structures 1404A, 1404B, and 1404C).

[0172] Acceleration structure 1400 may include a spatial search data structure for the ray tracing pipeline 1300, which is used for acceleration structure traversal 1320 to efficiently calculate the intersection points of rays and scene geometry. In at least one embodiment, the application can explicitly construct acceleration structure 1400 using the command list method BuildRaytracingAccelerationStructure(). In at least one embodiment, the application can optimize acceleration structure 1400 for different types of content, such as static content and animated content.

[0173] The top-level acceleration structure 1402 may be constructed from one or more references to one or more lower-level acceleration structures 1404A, 1404B, and / or 1404C. These references may be referred to as instance descriptors. Each instance descriptor may contain a transformation matrix for locating the instance descriptor in the scene, and an offset to the shader table 1410 (which may also be referred to as the “shader binding table”) for locating material information. In at least one embodiment, the top-level acceleration structure 1402 may be used as scene parameters provided to TraceRay() in the ray generation shader 1302, and may represent the entry point for intersection searches.

[0174] The ray tracing pipeline 1300 specifies the shaders present in the scene, while the acceleration structure 1400 specifies the geometry of the scene. The shader table 1410 can refer to a data structure used to bind geometry to shaders. For example, the shader table 1410 can define which shader is associated with which object in the scene. Furthermore, the shader table 1410 can also store information about the resources accessed by each shader, such as textures, buffers, and constants.

[0175] Shader table 1410 may contain a PPU memory that can be managed by the application. The application can be responsible for allocating resources, populating shader table 1410 with valid data, transferring it to the PPU, and correctly synchronizing shader table 1410 with ray tracing dispatch. The application can also maintain multiple shader tables 1410, for example, multi-buffering them so that one copy is updated while another copy is being rendered.

[0176] Shader table 1410 may include an array of shader records of equal size. Each shader record can associate a shader (or hit group) with a set of resources. In at least one embodiment, there may be one record for each geometry object in the scene, and shader table 1410 may contain thousands or more entries.

[0177] Now for reference Figure 15 , Figure 15 An example shader record 1500 suitable for implementing at least some embodiments of this disclosure is shown. Shader record 1500 is something that can be included... Figure 14 An example of a shader record in shader table 1410. Shader record 1500 includes shader identifier 1502 and root table 1504.

[0178] In at least one embodiment, shader identifier 1502 may be represented in memory in the beginning portion of shader record 1500. Shader identifier 1502 may be an opaque identifier, which the application obtains by querying shader identifier 1502 from the compiled shader. Root table 1504 may contain shader resources. The layout of root table 1504 may be defined by the shader's local root signature. The root signature may contain any combination of constants, descriptor tables, and root descriptors. For ray tracing, the application can directly access root table 1504 in memory (e.g., instead of using a "setter" method), which enables efficient updates. In at least one embodiment, shader table 1410 may be updated from the PPU shader.

[0179] As described in this article, shader table offsets can be used when constructing the top-level acceleration structure 1402 from the instance descriptor. Whenever TraceRay() finds an intersection, the system can use these offsets to locate the correct shader record 1500. The system can then bind the resources defined in shader record 1500 and execute the appropriate shader for the intersecting geometry.

[0180] Example computing device

[0181] Figure 16 This is a block diagram of an example computing device 1600 suitable for implementing at least some embodiments of the present disclosure. The computing device 1600 may include an interconnect system 1602 directly or indirectly coupled to the following devices: memory 1604, one or more central processing units (CPUs) 1606, one or more graphics processing units (GPUs) 1608, a communication interface 1610, input / output (I / O) ports 1612, input / output components 1614, a power supply 1616, one or more presentation components 1618 (e.g., displays), and one or more logic units 1620. In at least one embodiment, the computing device 1600 may include one or more virtual machines (VMs), and / or any component thereof may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 1608 may include one or more PUPs, one or more CPUs 1606 may include one or more vCPUs, and / or one or more logic units 1620 may include one or more virtual logic units. Therefore, computing device 1600 may include discrete components (e.g., a complete GPU dedicated to computing device 1600), virtual components (e.g., a portion of the GPU dedicated to computing device 1600), or a combination thereof.

[0182] although Figure 16The various boxes are shown connected via an interconnect system 1602 with wiring, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, a presentation component 1618, such as a display device, may be considered an I / O component 1614 (e.g., if the display is a touchscreen). As another example, the CPU 1606 and / or GPU 1608 may include memory (e.g., memory 1604 may represent a storage device other than the memory of the GPU 1608, CPU 1606, and / or other components). In other words, Figure 16 The computing devices mentioned are merely illustrative. No distinction is made between categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all of these are considered within the same category. Figure 16 Within the scope of computing devices.

[0183] Interconnect system 1602 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 1602 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Fast (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. For example, CPU 1606 may be directly connected to memory 1604. Furthermore, CPU 1606 may be directly connected to GPU 1608. Where there is a direct or point-to-point connection between components, interconnect system 1602 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required in computing device 1600.

[0184] The memory 1604 may include any of a wide variety of computer-readable media. Computer-readable media can be any available medium that can be accessed by the computing device 1600. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. For example and without limitation, computer-readable media may include computer storage media and communication media.

[0185] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media, implemented in any way or by any method or technique for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 1604 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by computing device 1600. As used herein, computer storage media does not include the signal itself.

[0186] Computer storage media may include computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. The term "modulated data signal" may refer to a signal whose characteristics are set or altered in a manner that encodes information into that signal. For example and without limitation, computer storage media may include wired media such as wired networks or direct wired connections, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0187] CPU 1606 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. Each of CPU 1606 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads simultaneously. CPU 1606 may include any type of processor and may include different types of processors depending on the type of computing device 1600 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 1600, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors such as math coprocessors, computing device 1600 may also include one or more CPUs 1606.

[0188] In addition to or as a replacement for CPU 1606, one or more GPUs 1608 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. One or more GPUs 1608 may be integrated GPUs (e.g., having one or more CPUs 1606) and / or one or more GPUs 1608 may be discrete GPUs. In embodiments, one or more GPUs 1608 may be coprocessors of one or more CPUs 1606. Computing device 1600 may use GPUs 1608 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, one or more GPUs 1608 may be used for general-purpose computing on a GPU (GPGPU). One or more GPUs 1608 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. GPUs 1608 may generate pixel data for outputting an image in response to rendering commands (e.g., rendering commands received from CPU 1606 via a host interface). GPU 1608 may include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. Display memory may be included as part of memory 1604. One or more GPUs 1608 may include two or more GPUs operating in parallel (e.g., via a link). The link may be directly connected to the GPUs (e.g., using NVLINK) or connected via a switch (e.g., using NVSwitch). When combined, each GPU 1608 may generate pixel data or GPGPU data for different portions of the output or different outputs (e.g., the first GPU for the first image, the second GPU for the second image). Each GPU may include its own memory or may share memory with other GPUs.

[0189] In addition to or as an alternative to CPU 1606 and / or GPU 1608, logic unit 1620 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. In embodiments, CPU 1606, GPU 1608, and / or logic unit 1620 may execute any combination of methods, processes, and / or portions thereof, discretely or jointly. One or more logic units 1620 may be part of and / or integrated into one or more of CPU 1606 and / or GPU 1608, and / or one or more logic units 1620 may be discrete components or otherwise separate from CPU 1606 and / or GPU 1608. In embodiments, one or more logic units 1620 may be coprocessors of one or more CPUs 1606 and / or one or more GPUs 1608.

[0190] Examples of logic unit 1620 include one or more processing cores and / or components thereof, such as data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree traversal unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), arithmetic logic unit (ALU), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or peripheral component interconnect fast (PCIe) element, etc.

[0191] Communication interface 1610 may include one or more receivers, transmitters, and / or transceivers that enable computing device 1600 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 1610 may include components and functions that enable communication via any of several different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication via Ethernet or InfiniBand), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 1620 and / or communication interface 1610 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 1602 to one or more GPUs 1608 (e.g., their memory).

[0192] I / O port 1612 enables computing device 1600 to be logically coupled to other devices, including I / O component 1614, presentation component 1618, and / or other components, some of which may be built into (e.g., integrated into) computing device 1600. Illustrative I / O component 1614 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, and so on. I / O component 1614 can provide a Natural User Interface (NUI) for processing user-generated air gestures, voice, or other physiological input. In some instances, the input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 1600 (described in more detail below). Computing device 1600 may include depth cameras such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. In addition, the computing device 1600 may include an accelerometer or gyroscope that enables motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by the computing device 1600 to render immersive augmented reality or virtual reality.

[0193] Power supply 1616 may include hard-wired power supply, battery power supply, or a combination thereof. Power supply 1616 may supply power to computing device 1600 so that components of computing device 1600 can operate.

[0194] The presentation component 1618 may include a display (such as a monitor, touch screen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 1618 may receive data from other components (such as GPU 1608, CPU 1606, DPU, etc.) and output that data (such as as images, videos, sounds, etc.).

[0195] Example Data Center

[0196] Figure 17 An example data center 1700 that may be used in at least one embodiment of this disclosure is shown. The data center 1700 may include a data center infrastructure layer 1710, a framework layer 1720, a software layer 1730, and / or an application layer 1740.

[0197] like Figure 17As shown, the data center infrastructure layer 1710 may include a resource coordinator 1712, grouped computing resources 1714, and node computing resources (“nodes CR”) 1716(1)-1716(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 1716(1)-1716(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CR 1716(1)-1716(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CR1716(1)-17161(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more nodes CRs1916(1)-1916(N) may correspond to virtual machines (VMs).

[0198] In at least one embodiment, the grouped computing resources 1714 may include individual groups of nodes CR1716 housed within one or more racks (not shown), or multiple racks housed within a data center at different geographical locations (also not shown). Individual groups of nodes CR1716 within the grouped computing resources 1714 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several nodes CR1716, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0199] Resource coordinator 1712 may be configured or otherwise control one or more nodes CR1716(1)-1716(N) and / or grouped computing resources 1714. In at least one embodiment, resource coordinator 1712 may include a Software Design Infrastructure (“SDI”) management entity for data center 1700. Resource coordinator 1712 may include hardware, software, or some combination thereof.

[0200] In at least one embodiment, such as Figure 17As shown, framework layer 1720 may include job scheduler 1728, configuration manager 1734, resource manager 1736, and / or distributed file system 1738. Framework layer 1720 may include a framework of software 1732 supporting software layer 1730 and / or one or more applications 1742 supporting application layer 1740. Software 1732 or application 1742 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1720 may be, but is not limited to, a free and open-source software web application framework (such as Apache Spark™ (hereinafter “Spark”)) that can leverage distributed file system 1738 for large-scale data processing (e.g., “big data”). In at least one embodiment, job scheduler 1728 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 1700. Configuration manager 1734 may be able to configure different layers, such as software layer 1730 and framework layer 1720 (which includes Spark and distributed file system 1738 for supporting large-scale data processing). Resource manager 1736 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 1778 and job scheduler 1728 to support distributed file system 1738 and job scheduler 1728. In at least one embodiment, clustered or grouped computing resources may include grouped computing resources 1714 in data center infrastructure layer 1710. Resource manager 1736 may coordinate with resource coordinator 1712 to manage these mapped or allocated computing resources.

[0201] In at least one embodiment, the software 1732 included in software layer 1730 may include software used in at least a portion of nodes CR1716(1)-1716(N), grouped computing resources 1714, and / or the distributed file system 1738 of framework layer 1720. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0202] In at least one embodiment, the application 1742 included in the application layer 1740 may include one or more types of applications used at least in part by nodes CR1716(1)-1716(N), grouped computing resources 1714, and / or the distributed file system 1738 of the framework layer 1720. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0203] In at least one embodiment, any of the configuration manager 1734, resource manager 1736, and resource coordinator 1712 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can free the data center operator of data center 1700 from making potentially poor configuration decisions and potentially avoid underutilization and / or poor performance of the data center.

[0204] According to one or more embodiments described herein, data center 1700 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models may be trained by using the software and / or computing resources described above with respect to data center 1700 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to data center 1700 by using weight parameters computed through one or more training techniques, such as, but not limited to, those described herein.

[0205] In at least one embodiment, the data center 1700 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0206] Example network environment

[0207] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 16 This is implemented on one or more instances of computing devices 1600—for example, each device may include similar components, features, and / or functions of one or more computing devices 1600. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 1700, examples of which are described in this document. Figure 17 To describe in more detail.

[0208] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. Where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0209] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0210] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0211] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). The core server may assign at least a portion of the functionality to the edge server if the connection to the user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0212] One or more client devices may include the information described in this article. Figure 16 At least some of the components, features, and functions of one or more example computing devices 1600 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming equipment or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0213] This disclosure can be described in the general context of machine-usable instructions or computer code, including computer-executable instructions such as program modules, which are executed by a computer or other machine such as a personal digital assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. This disclosure can also be practiced in distributed computing environments in which tasks are performed by remote processing devices linked via a communication network.

[0214] As used herein, the phrase “and / or” relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, “element A, element B, and / or element C” can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Additionally, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Furthermore, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0215] The subject matter of this disclosure has been specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways to include different steps or combinations of steps similar to those described herein, combined with other current or future techniques. Furthermore, although the terms “step” and / or “block” may be used herein to denote different elements of the method employed, these terms should not be construed as implying any particular order between the various steps disclosed herein, unless the order of the various steps is explicitly described.

[0216] Example paragraph

[0217] A. A method comprising: determining one or more categories associated with the one or more resources based at least on information corresponding to one or more resources associated with a first instance of an application running on one or more servers; allocating one or more regions of virtual memory to bind to the one or more resources based at least on the one or more categories; mapping the one or more regions of the virtual memory to one or more first regions of physical memory allocated for storing the one or more resources; determining that at least one of the one or more resources is a duplicate resource of at least a second resource associated with one or more second instances of the application running on the one or more servers; and remapping the one or more regions of the virtual memory to one or more second regions of physical memory allocated for storing the at least second resource based at least on the fact that the at least one resource is the duplicate resource.

[0218] B. The method as described in any one of paragraphs 1, further comprising: at least based on remapping the one or more regions of the virtual memory to the one or more second regions of the physical memory, such that the one or more first regions of the physical memory are released.

[0219] C. The method as described in any one of paragraphs 1, further comprising: processing a request for access to the at least one resource associated with the first instance of the application using the one or more regions of the virtual memory mapped to the one or more second regions of the physical memory.

[0220] D. The method as described in any one of paragraphs 1, wherein determining the one or more categories associated with the one or more resources is based at least on evaluating one or more attributes contained in the information associated with the first instance of the application that requested to generate the one or more resources.

[0221] E. The method as described in any one of paragraphs 1, wherein at least one of the one or more categories is associated with at least one of the one or more resources, the at least one category indicating that the at least one resource is a static resource that can be shared between different instances of the application, and wherein the allocation of the one or more regions of the virtual memory is based at least on the at least one resource being the static resource.

[0222] F. The method as described in any one of paragraphs 1, further comprising: allocating one or more third regions of the physical memory for storing the one or more resources associated with the first instance of the application, the one or more third regions of the physical memory including the one or more regions of the physical memory, the one or more resources including at least one or more shareable resources and one or more non-shareable resources, wherein the mapping is based at least on the allocation of the one or more third regions of the physical memory.

[0223] G. The method as described in any one of paragraphs 1, further comprising: allocating one or more second regions of the virtual memory for binding to at least a subset of the one or more resources; mapping the one or more second regions of the virtual memory to one or more third regions of the physical memory; determining that at least the subset of the one or more resources includes one or more original resources associated with the application running on the one or more servers; and storing in one or more databases data indicating the use of the one or more third regions of the physical memory to store at least the subset of the one or more resources.

[0224] H. A system comprising: one or more processors configured to: calculate one or more identifiers of one or more first resources associated with one or more first application instances; determine, at least based on querying one or more data sources using the one or more identifiers, that one or more first portions of at least one memory have been allocated for storing one or more second resources that duplicate the one or more first resources, the one or more second resources being associated with one or more second application instances; and at least based on the determination, release the one or more second portions of the at least one memory allocated for storing the one or more first resources.

[0225] I. The system as described in any one of paragraphs 8, wherein the one or more processors are further configured to determine, at least based on evaluating one or more attributes contained in the metadata associated with the one or more first resources, one or more categories corresponding to the one or more first application instances, the one or more categories indicating that the one or more first resources can be shared among application instances running on one or more servers.

[0226] J. The system as described in any one of paragraphs 9, wherein the one or more classifications indicate that the one or more first resources are static resources, the static resources comprising at least one of: texture resources associated with the one or more first application instances; mesh data associated with the one or more first application instances; or shader code associated with the one or more first application instances.

[0227] K. The system as described in any one of paragraphs 8, wherein the one or more processors are further configured to: allocate one or more portions of virtual memory for the one or more first application instances; and map the one or more portions of the virtual memory to the one or more second portions of the at least one memory, wherein the at least one memory is physical memory.

[0228] L. The system as described in any one of paragraphs 11, wherein the one or more processors are further configured to, at least based on the determination, update the mapping of the one or more portions of the virtual memory from the one or more second portions mapped to the at least one memory to the one or more first portions mapped to the at least one memory.

[0229] M. The system as described in any one of paragraphs 12, wherein the one or more processors are further configured to process requests for access to the one or more first resources associated with the one or more first application instances using the one or more portions of the virtual memory mapped to the one or more first portions of the at least one memory.

[0230] N. The system as described in any one of paragraphs 8, wherein the one or more processors are further configured to: query the one or more data sources using the one or more identifiers; and, based at least on the query, determine the presence of one or more second identifiers in the one or more data sources, the one or more second identifiers being duplicates of the one or more identifiers; wherein determining that the one or more first portions of the at least one memory have been allocated for storing the one or more second resources is based at least on the one or more data sources containing the one or more second identifiers.

[0231] O. The system as described in any one of paragraphs 8, wherein calculating the one or more identifiers of the one or more first resources comprises: calculating one or more hash values ​​of the one or more first resources using one or more graphics processing units (GPUs).

[0232] P. A system as described in any one of paragraph 8, wherein the system comprises at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more visual language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system comprising one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

[0233] Q. At least one processor, comprising: processing circuitry, for updating a mapping between an allocation of virtual memory for a resource and a first portion of physical memory allocated for storing the resource, such that the allocation of the virtual memory is mapped to a second portion of the physical memory allocated for storing duplicate resources of the resource, and for releasing the first portion of the physical memory at least based on the update of the mapping.

[0234] R. At least one processor as described in any one of paragraphs 17, the processing circuitry is further configured to determine, at least based on querying a database using an identifier calculated for the resource, that the second portion of the physical memory has been allocated for storing the duplicate resource, wherein the mapping is updated at least based on the determination.

[0235] S. At least one processor as described in any one of paragraphs 17, the processing circuitry is further configured to determine a classification associated with the resource based at least on evaluating one or more attributes contained in the metadata associated with the resource, wherein the allocation of the virtual memory for the resource is based at least on the classification associated with the resource indicating that the resource can be shared among one or more instances of an application.

[0236] T. A processor as described in any one of paragraph 17, wherein the processor is included in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more visual language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system comprising one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

Claims

1. A method comprising: Based at least on information corresponding to one or more resources associated with a first instance of an application running on one or more servers, determine one or more categories associated with said one or more resources; Based on at least one or more of the categories, allocate one or more regions of virtual memory to bind to one or more resources; Mapping the one or more regions of the virtual memory to one or more first regions of the physical memory allocated for storing the one or more resources; Determine that at least one of the one or more resources is a duplicate resource of at least a second resource associated with one or more second instances of the application running on the one or more servers; as well as Based at least on the fact that the at least one resource is the duplicate resource, the one or more regions of the virtual memory are remapped to one or more second regions of the physical memory allocated for storing the at least second resource.

2. The method according to claim 1, further comprising: At least based on remapping one or more regions of the virtual memory to one or more second regions of the physical memory, such that one or more first regions of the physical memory are released.

3. The method according to claim 1, further comprising: The virtual memory, which is mapped to one or more second regions of the physical memory, is used to process requests for access to the at least one resource associated with the first instance of the application.

4. The method according to claim 1, wherein, Determining the one or more categories associated with the one or more resources is based at least on evaluating one or more attributes contained in the information associated with the first instance of the application that requested to generate the one or more resources.

5. The method according to claim 1, wherein, At least one of the one or more categories is associated with at least one of the one or more resources, the at least one category indicating that the at least one resource is a static resource that can be shared between different instances of the application, and wherein the allocation of the one or more regions of the virtual memory is based at least on the at least one resource being the static resource.

6. The method according to claim 1, further comprising: Allocate one or more third regions of the physical memory for storing the one or more resources associated with the first instance of the application, wherein the one or more third regions of the physical memory include the one or more regions of the physical memory, and the one or more resources include at least one or more shareable resources and one or more non-shareable resources, wherein the mapping is based at least on the allocation of the one or more third regions of the physical memory.

7. The method according to claim 1, further comprising: Allocate one or more second regions of the virtual memory for binding to at least a subset of the one or more resources; Map one or more second regions of the virtual memory to one or more third regions of the physical memory; Determining at least a subset of the one or more resources includes one or more original resources associated with the application running on the one or more servers; as well as Data indicating the use of the one or more third regions of the physical memory to store at least a subset of the one or more resources is stored in one or more databases.

8. A system comprising: One or more processors are used for: Calculate one or more identifiers of one or more first resources associated with one or more first application instances; Based at least on querying one or more data sources using the one or more identifiers, it is determined that one or more first portions of at least one memory have been allocated to store one or more second resources that are duplicates of the one or more first resources, and the one or more second resources are associated with one or more second application instances; as well as Based at least on the determination, the at least one memory allocated for storing one or more second portions of the first resource is released.

9. The system according to claim 8, wherein, The one or more processors are further configured to determine, at least based on evaluating one or more attributes contained in the metadata associated with the one or more first resources, one or more categories corresponding to the one or more first application instances, the one or more categories indicating that the one or more first resources can be shared among application instances running on one or more servers.

10. The system according to claim 9, wherein, The one or more classifications indicate that the one or more first resources are static resources, and the static resources include at least one of the following: Texture resources associated with the one or more first application instances; Grid data associated with the one or more first application instances; or Shader code associated with the one or more first application instances.

11. The system according to claim 8, wherein, The one or more processors are also used for: Allocate one or more portions of virtual memory for the one or more instances of the first application; as well as Mapping one or more portions of the virtual memory to one or more second portions of the at least one memory, wherein the at least one memory is physical memory.

12. The system according to claim 11, wherein, The one or more processors are further configured to, at least based on the determination, update the mapping of the one or more portions of the virtual memory from the one or more second portions mapped to the at least one memory to the one or more first portions mapped to the at least one memory.

13. The system according to claim 12, wherein, The one or more processors are also configured to process requests for access to the one or more first resources associated with the one or more first application instances, using the one or more portions of the virtual memory mapped to the one or more first portions of the at least one memory.

14. The system according to claim 8, wherein, The one or more processors are also used for: Use one or more of the identifiers to query one or more data sources; as well as Based at least on the query, determine the existence of one or more second identifiers from the one or more data sources, wherein the one or more second identifiers are duplicates of the one or more identifiers; Wherein, determining that one or more first portions of the at least one memory have been allocated for storing the one or more second resources is based at least on the fact that the one or more data sources contain the one or more second identifiers.

15. The system according to claim 8, wherein, Calculating the one or more identifiers of the one or more first resources includes: calculating one or more hash values ​​of the one or more first resources using one or more graphics processing units (GPUs).

16. The system according to claim 8, wherein, The system is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system for performing one or more simulation operations; A system for performing one or more digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more visual language models (VLMs); A system that performs operations using one or more multimodal language models; A system for performing one or more conversational AI operations; A system for generating synthetic data; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

17. At least one processor, comprising: A processing circuit is configured to update the mapping between a virtual memory allocation for a resource and a first portion of physical memory allocated for storing the resource, such that the virtual memory allocation is mapped to a second portion of the physical memory allocated for storing duplicate resources of the resource, and to release the first portion of the physical memory at least based on the update of the mapping.

18. The at least one processor of claim 17, wherein the processing circuitry is further configured to determine, at least based on querying a database using an identifier calculated for the resource, that the second portion of the physical memory has been allocated for storing the duplicate resource, wherein, The mapping is updated at least based on the determination.

19. The at least one processor of claim 17, wherein the processing circuitry is further configured to determine a classification associated with the resource based at least on evaluating one or more attributes contained in the metadata associated with the resource, wherein, The allocation of the virtual memory for the resource is based at least on the classification associated with the resource, indicating that the resource can be shared among one or more instances of the application.

20. The processor of claim 17, wherein, The processor is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system for performing one or more simulation operations; A system for performing one or more digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more visual language models (VLMs); A system that performs operations using one or more multimodal language models; A system for performing one or more conversational AI operations; A system for generating synthetic data; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

Images