Remote Promise and Remote Future for Subsequent Components to Update Pre-existing States

Abstract

Technologies are described that enable downstream components to update upstream states in streaming pipelines. A procedure on a first computer receives a remote promise object, associated with a first serialized object, from a second computer in the data center via a network structure. The remote promise object uniquely identifies a first contiguous block of the first serialized object, which is stored in memory allocated to the second computer. The procedure receives the contents of the first contiguous block and sends the contents of a second serialized object back to the second computer to release the remote promise object.

Description

TECHNICAL FIELD

[0001] At least one embodiment relates to processing resources used to perform and enable network communication. For example, at least one embodiment relates to remote direct memory access technology, and more particularly, to enabling remote direct memory access (RDMA) transport of serialized objects in streaming pipelines. For example, at least one embodiment relates to processors or computer systems used to provide remote descriptors to enable RDMA transport of serialized objects, according to various novel techniques described herein.

[0002] At least one embodiment relates to processing resources used to perform and enable network communication. For example, at least one embodiment relates to remote direct memory access technology, and more particularly, to stack remote descriptors to enable RDMA transport of serialized objects in streaming pipelines. For example, at least one embodiment relates to processors or computer systems used to provide a streaming stack from multiple remote descriptors, where at least some contents of a serialized object are obtained from a first memory for storage in a second memory before the streaming stack is complete, according to various novel techniques described herein.

[0003] At least one embodiment relates to processing resources used to perform and enable network communication. For example, at least one embodiment relates to RDMA technology such that downstream nodes receive data from upstream memory, and downstream nodes can PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is complete. For example, at least one embodiment relates to processors or computer systems used to provide a remote Promise object that uniquely identifies a first serialized object and is released when a second streaming object is returned, according to various novel techniques described herein. SHORT SUMMARY

[0004] The invention is defined by the claims. To illustrate the invention, aspects and embodiments are described herein, which may or may not fall within the scope of the claims.

[0005] Technologies are described that enable downstream components to update upstream state in streaming pipelines. A method of a first computing device receives a remote promise object associated with a first serialized object from a second computing device in the data center over a network fabric. The remote promise object uniquely identifies a first contiguous block of the first serialized object stored in memory associated with the second computing device. The method obtains the contents of the first contiguous block and sends the contents of a second serialized object back to the second computing device to release the remote promise object.

[0006] Any feature of one aspect or embodiment may be applied to other aspects or embodiments in any suitable combination. In particular, any feature of a method aspect or embodiment may be applied to a device aspect or embodiment, and vice versa. BACKGROUND

[0007] Remote Direct Memory Access (RDMA) technology enables network adapters to transfer data directly to (or from) the memory of a remote device over a network, without storing data in the data buffers of the remote device's operating system. The advantages of RDMA include reduced computation and caching by processing devices, such as central processing units (CPUs), eliminating the need to copy data between different network layers, convenient discretization of the transferred data, and so on.RDMA transactions are supported by several communication protocols, including RDMA over Converged Ethernet (RoCE), which enables RDMA operations using the traditional standard Ethernet infrastructure; Internet Wide Area RDMA Protocol (iWARP), which enables RDMA operations using the Transmission Control Protocol (TCP); and InfiniBand™, which provides native support for RDMA operations. RDMA transactions are particularly useful in cloud computing applications and numerous applications that require high data rates and low latency.

[0008] Within a streaming data pipeline, data can be collected from one or more sensors and sent to the streaming data pipeline for processing. A streaming data pipeline can have multiple stages. Some streaming data pipelines can implement all stages on the same physical machine using a single computing device and local memory. Other streaming data pipelines implement different stages on two or more different physical machines. However, when implementing the streaming data pipeline on multiple physical machines, the raw data must be transferred in bulk across machine boundaries over a network connection. The data objects can be stored in the local memory of a first machine and serialized for transport to be stored in the local memory of a second machine.In the context of streaming data pipelines, moving this raw data across the network connection at each physical compute boundary can incur some measurable throughput costs, particularly due to the potential size of the data objects being transferred.

[0009] US 9,934,173 B1 discloses a method for exchanging data between a remote host and a target system, comprising receiving at least one remote descriptor from the remote host via a front-end structure at a controller, wherein the at least one remote descriptor specifies a remote buffer in a remote memory of the remote host that is larger than a page size. US 5,713,044 A discloses computer systems with DMA (Direct Memory Access) units, and in particular to chain descriptors used by a DMA unit. US 2020 / 012534 A1 discloses a gateway comprising a streaming engine with a data mover engine and a memory management engine.US 2019 / 188181 A1 discloses serializing and deserializing an object by transmitting metadata and addresses of data elements in a byte stream over a data network, receiving the byte stream from the data network, defining a container for the object, retrieving the addresses of the data elements in the first memory from the input byte stream, applying direct memory access (DMA) or remote direct memory access (RDMA) to read the data elements using the retrieved addresses, and writing the data elements to the container to create a new instance of the object. BRIEF DESCRIPTION OF THE DIFFERENT VIEWS OF THE DRAWINGS

[0010] Various embodiments according to the present disclosure will be described with reference to the drawings in which: Fig. 1 is a block diagram of an exemplary system architecture according to at least one embodiment. Fig. 2 is a block diagram of a memory storing a remote descriptor and a contiguous block, according to at least one embodiment. Fig. 3A shows a remote descriptor generated by a first physical computer and sent to a second physical computer in a data center, according to at least one embodiment. Fig. 3B shows the second physical computer performing an RDMA GET operation with the remote descriptor from Fig. 3A. Fig. 3C shows the second physical machine sending a message to release the remote descriptor from Fig. 3A, according to at least one embodiment. Fig. 4 shows the division of a remote descriptor into multiple descriptors in a data center according to at least one embodiment. Fig. 5 is a flowchart of a method for generating and transmitting a remote descriptor according to at least one embodiment. Fig. 6 is a flowchart of a method for using a remote descriptor to transfer a serialized object between computing devices, according to at least one embodiment. Fig. 7A is a block diagram of a first computing device in a data center for generating a streaming batch in a first instance, according to at least one embodiment. Fig. 7B is a block diagram of a downstream consumer in a data center for processing the RD (unique item A) in a second instance while the batch window is still open, according to at least one embodiment. Fig. 8 is a flowchart of a method for generating a streaming stack according to at least one embodiment. Fig. 9 is a flowchart of a method for using a streaming stack to transfer multiple streaming objects, according to at least one embodiment. Fig. 10 shows a remote promise object created by a first physical computer and sent to a second physical computer in a data center, according to at least one embodiment. Fig. 11 shows a remote promise future object created by a first physical machine and sent to a second physical machine in a data center, according to at least one embodiment. Fig. 12 shows a remote promise future object created by a first physical computer and sent to a second physical computer in a data center, according to at least one embodiment. Fig. 13 is a flowchart of a method for creating and sending a remote promise object according to at least one embodiment. Fig. 14 shows an example of a data center system according to at least one embodiment. Fig. 15 shows a computer system according to at least one embodiment. Fig. 16 shows a computer system according to at least one embodiment. Fig. 17 shows at least parts of a graphics processor according to one or more embodiments. Fig. 18 shows at least parts of a graphics processor according to one or more embodiments. DETAILED DESCRIPTION

[0011] As described above, when implementing streaming data pipelines across multiple physical machines, there can be measurable throughput costs associated with moving raw data across machine boundaries when using traditional approaches.

[0012] Aspects and embodiments of the present disclosure address the above and other deficiencies by providing a remote descriptor that enables RDMA transport of a serialized object. The remote descriptor is a language construct or control structure that can uniquely identify a location of the serialized object in a data center. Instead of moving the actual data to each pipeline stage, the remote descriptor can be moved to each stage of the pipeline. The remote descriptor can be a set of one or more remote descriptor types, for example, a memory descriptor (MemoryDescriptor), a metadata descriptor (MetaDataDescriptor), a packed descriptor (PackedDescriptor), a data descriptor (EagerDataDescriptor), or the like.One type of remote descriptor is the memory descriptor, which refers to an object that uniquely identifies a starting address of a contiguous block in local memory, a size of the contiguous block, a physical machine identifier corresponding to the local memory, an RDMA access key, and a reference counter token value representing one or more ownership shares of a serialized object. The memory descriptor can contain, at a minimum, a starting address, a size, and any remote access keys required for RDMA. The metadata descriptor can contain a user-defined message that can be serialized into a small block of contiguous memory. The data descriptor can be a serialized blob of bytes.The packed descriptor can allow multiple small object blocks of data to be packed into a single contiguous block, with the packed descriptor describing the offsets within the contiguous region. These and other types of remote descriptors can be used by a user to define the representation of the object, as demonstrated in the following two examples.

[0013] In a first example, a tensor of floating-point numbers might have dimension 3, a shape { 3, 256, 256}, and an optional stride similar in format to the shape. A remote descriptor of this tensor might use i) a single memory descriptor with the starting address of the contiguous block of floating-point numbers and a size that can be larger than 3*256*256*sizeof(float) if the strides are not compact, and ii) a single specialized TensorMetaData descriptor containing the data_type, the shape, and optionally the stride.

[0014] In a second example, a ComplexTensor containing complex floating-point numbers of the same form as above could be allocated in memory through two separate memory allocations (separate contiguous memory blocks), so that the real and imaginary parts require separate memory descriptors. In this case, there are two memory descriptors and a TensorMetaData descriptor to define the object's representation.

[0015] The remote descriptor can provide a language-level construct, similar to std::future in the RDMA library, that enables RDMA transport of the serialized object. Remote descriptors are often smaller than 600 bytes (e.g., 64 bytes), while the serialized objects they point to are typically several orders of magnitude larger. Remote descriptors can significantly reduce data transport across machine boundaries in streaming data pipelines by eliminating the need for bulk data transfers at each boundary and shifting to an on-demand model where requests for bulk data can be made only when and where they are needed. The on-demand model can allow remote descriptors, rather than the serialized object, to flow through the various stages of the pipeline without incurring the object's transfer costs.Only when needed, the requesting node in the on-demand model can enable asynchronous PULL operations to retrieve the object from the local memory of a remote node (another physical machine) and transfer it to the local memory of the requesting node. In the context of streaming pipelines, moving the remote descriptors through the pipeline instead of the raw data can increase data throughput.

[0016] One approach to managing network communication is to group individual work packets into a batch. Conventional batching algorithms group individual work packets into a collection in three ways: 1) collecting a fixed number of work packets, regardless of time; 2) collecting work packets over a time window; or 3) a combined number and time window approach. Conventional batching algorithms include blocking in the algorithm in case the data for a batch is not met. Conventional batchers are typically external entities to the data plane and require that the entire data payload be collected and forwarded, resulting in at least a doubling of data movement across compute boundaries.

[0017] Aspects and embodiments of the present disclosure address the above and other deficiencies by providing a streaming stack composed of multiple remote descriptors to enable RDMA transport of serialized objects in streaming pipelines. Streaming stack refers to a set of one or more slots, each of which can be associated with a remote descriptor. The streaming stack completes when i) all slots of the streaming stack are allocated or ii) the stacking window ends (e.g., expiration of a timer corresponding to the streaming stack), whichever occurs first. As described herein, at least some contents of a serialized object may be retrieved from a first memory for storage in a second memory before the streaming stack completes.Aspects and embodiments of a streaming stacker can solve the problem of traditional stackers by 1) collecting work packets (batch items) as remote descriptors, reducing the need to move large amounts of data during batching, and 2) immediately forwarding the remote descriptors to their downstream consumers (also called downstream destinations) before a time slot (also called a batching window) completes. Streaming stacker refers to processing logic, which can be hardware, firmware, software, or any combination thereof, that can create, manage, and send a streaming batch containing one or more remote descriptors, where each remote descriptor uniquely identifies a location of a serialized object in a data center. Batching window refers to a period of time allotted for the completion of a streaming batch.The streaming stack completes when i) all slots of the streaming stack are allocated or ii) the stacking window ends (e.g., expiration of a timer corresponding to the streaming stack), whichever occurs first. By stacking remote descriptors, the streaming stacker helps reduce the cost of duplicate large data transfers by logically stacking the remote descriptors rather than the actual data (serialized objects). By forwarding the incoming work packets (stack items) to the intended downstream destination immediately upon arrival, the downstream destination can instantiate a PULL operation to obtain the remote data identified in the remote descriptor while the stacking window is still open (i.e., not completed).Aspects and embodiments of the present disclosure can reduce data movement by orders of magnitude by batching and transmitting remote descriptors instead of serialized objects. Aspects and embodiments of the present disclosure can reduce overall latency by applying overlap in retrieving remote data with windowing of the streaming batcher.

[0018] Stateful streaming pipelines can be challenging when downstream consumers need to update their upstream state. Without the concept of a remote Promise object and a remote Promise Future object, these stateful algorithms must either be restructured to be stateless or create a serialization bottleneck that can impact performance.

[0019] Aspects and embodiments of the present disclosure address the above and other deficiencies by providing a remote promise object and a remote promise future object to enable downstream nodes to receive or retrieve data (e.g., using a pull operation or a get operation) from upstream storage, and enable downstream nodes to send data (e.g., using a push operation or a put operation) to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the upstream push operation completes. A remote promise object is an object that uniquely identifies a contiguous block of memory in which a serialized object is stored.The remote promise object identifies a starting address of a contiguous block in local memory, a size of the contiguous block, a physical machine identifier corresponding to the local memory, an RDMA access key, and a value of a reference counter token that represents unique ownership of the serialized object because the remote promise object is unique. The remote promise object is a language construct or control structure that can be exchanged with another computing device, which can complete the remote promise object by sending a message back to the device sending the remote promise object.

[0020] Aspects and embodiments of the present disclosure may provide a remote promise object that uniquely identifies a first serialized object and is released when a second streaming object is returned. The remote promise object / remote promise future object constructs may provide language-level constructs similar to std::promise and std::future that allow upstream push / put operations to send the data and the indication that an upstream node can proceed. The remote promise object / remote promise future object constructs are the conceptual counterpart to the remote descriptor. The remote descriptors may allow the downstream node to obtain data (e.g., pull data) from the upstream storage, and the remote promise object / remote promise future object constructs allow downstream nodes to obtain data (e.g.,PUSH / PUT operation) to upstream nodes, while these upstream nodes relinquish their execution context and do not consume any resources until the downstream PUSH / PUT operation is completed. REMOTE DESCRIPTORS

[0021] The following section describes embodiments related to generating, sending, and processing remote descriptors to enable RDMA transport of a serialized object in streaming pipelines implemented with multiple computing devices in a data center.

[0022] Fig. 1 is a block diagram of an exemplary system architecture of a computer system 100, according to at least one embodiment. A computer system refers to a system that includes multiple physical computers with one or more computing devices, one or more additional devices, such as a memory device, a peripheral device, a storage device, a network device, or any combination thereof. A computing device refers to any processing device, such as a processor, a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a programmable device, or the like.

[0023] Computer system 100 may be a data center with two or more physical computers, including a first physical computer 112 and a physical computer B 110. A data center refers to a facility having one or more physical computers with one or more computing devices, network devices, memory and storage devices, and the like. A data center may provide systems for data acquisition, data processing, data storage, communications, and the like. Computer system 100 is configured to implement a streaming pipeline with multiple pipeline stages. The first physical computer 112 includes a first pipeline stage 102 and a second pipeline stage 104. The physical computer B 110 includes a third pipeline stage 106 and a fourth pipeline stage 108. In other embodiments, there may be more than two physical computers and more than four pipeline stages.The computer system 100 may include additional physical computing boundaries between the pipeline stages. For example, each pipeline stage may be implemented on a separate physical computer. The first physical computer 112 includes a first computing device 122 and a first memory 114 for implementing the first pipeline stage 102. The physical computer B 110 includes a second computing device 124 and a second memory 116 for implementing the second pipeline stage 104. The first computing device 122 and the second computing device 124 may be the same underlying computing device of the first physical computer 112. The first memory 114 and the second memory 116 may be the same underlying memory of the first physical computers 112.

[0024] During operation, the first pipeline stage 102 may perform one or more operations using the first computing device 122. The first computing device 122 may store a serialized object in the first memory 114. After completing one or more operations in the first pipeline stage 102, the second pipeline stage 104 may perform one or more operations on the serialized object using the second first computing device 122. The serialized object may be stored in a second memory 116 for performance of the operations by the second pipeline stages 104. In another embodiment, the serialized object is stored in the same memory (first memory 114 or second memory 116) for both the first pipeline stage 102 and the second pipeline stage 104 without moving the data between two memories.In other embodiments, the first and second memory 114, 116 represent different regions of the same memory, with the different regions being assigned to the respective pipeline stages.

[0025] After completing the one or more operations in the second pipeline stage 104, the second pipeline stage 104 may generate remote descriptors and send them to a physical computer B 110 over a network connection 130 (e.g., a network fabric coupled between physical computers of a data center) instead of sending the serialized object. Network fabric refers to a network topology in which components exchange data via interconnected switches. A network fabric may include hubs, switches, adapters, cables, or the like to support communication between devices. The serialized object may remain in the second memory 116 (or in some cases, the first memory 114). For example, the second computing device 124 may generate a remote descriptor for a serialized object, which is an object that uniquely identifies a location of the serialized object in the second memory 116.The remote descriptor may include a starting address of a contiguous block in the second memory 116. A contiguous block refers to a set of memory locations in a memory with consecutive addresses that can be assigned or mapped to an object. The remote descriptor may include a size of the contiguous block. The remote descriptor may include a first physical host identifier identifying the first physical host 112. The first physical host identifier identifies the host on which the second memory 116 is located. The remote descriptor may include an RDMA access key. This allows a downstream consumer of the remote descriptor, i.e., the third computing device 126 of the third pipeline stage 106, to access the memory location of the serialized object when it is received.The remote descriptor may include a reference count token, also known as a lifecycle count. The reference count token refers to a parameter of the remote descriptor that specifies a value representing a share of ownership. The reference count token may specify a remaining lifecycle of the remote descriptor. The value of the reference count token may represent a number of ownership shares of the serialized object. For example, if the remote descriptor is shared with two downstream consumers, the value of the reference count token would be two. When the remote descriptors are completed, the value of the reference count token may be updated (e.g., decremented). When the value of the reference count token reaches a threshold (e.g., zero), the serialized object may be released from the second memory 116.

[0026] Once the remote descriptor is generated, the second computing device 124 may send the remote descriptor to the third computing device 126 over the network connection 130. The third computing device 126 may process the remote descriptor to determine where to obtain the serialized object. In at least one embodiment, the third computing device 126 may perform an RDMA GET operation over the network connection 130 to obtain the content.

[0027] The third computing device 126 may, using the remote descriptor, obtain the contents of the contiguous block from the second memory 116 (or the first memory 114) into the third memory 118 associated with the third computing device 126. Once the contents of the contiguous block are obtained from the second memory 116, the third computing device 126 may send a message to the second computing device 124 to release the remote descriptor. In at least one embodiment, the second computing device 124 may receive the message from the third computing device 126. In response to receiving the message, the second computing device 124 may update (e.g., decrement) a value of the reference counter token in the remote descriptor. In response to the value of the reference counter token exceeding a threshold (e.g.,zero), the second computing device 124 may release the serialized object from the second memory 116 (or the first memory 114).

[0028] As described above, a size of the remote descriptor is smaller than a size of the contiguous block of the serialized object. Thus, the actual data is not transmitted over the network connection until the third computing device 126 is ready to process the serialized object. At this time, the third computing device 126 can use the remote descriptor to obtain the contents of the serialized object over the network connection 130.

[0029] In another embodiment, the third computing device 126 may receive a second remote descriptor associated with a second serialized object via the network connection 130. The second remote descriptor may be received from the first computing device 122, the second computing device 124, or another computing device of another physical machine in the data center.The second remote descriptor is a second object that uniquely identifies a second starting address of a second contiguous block in the first memory 114 or the second memory 116 associated with the first physical computer 112, a second size of the second contiguous block, a second physical computer identifier associated with the first physical computer 112, a second RDMA access key, and a value of a second reference counter token representing one or more ownership interests in the second serialized object. Using the second remote descriptor, the third computing device 126 performs an RDMA GET operation to obtain the contents of the second contiguous block from the third memory for transfer to the first memory over the network connection 130.

[0030] In at least one embodiment, the third computing device 126 sends a second message to the upstream node (e.g., the second computing device 124) over the network connection 130 to release the second remote descriptors. In response to the second message, the value of the second reference counter token is updated. In response to the value of the second reference counter token meeting a threshold (e.g., being equal to zero), the second serialized object is released from the local memory in which the second serialized object is located.

[0031] In another embodiment, the third computing device 126 receives a second remote descriptor associated with a second serialized object over the network connection 130 (e.g., network fabric). The second remote descriptor is a second object that uniquely identifies: a second starting address of a second contiguous block in the second memory 116 associated with the second computing device 124 in the data center, a second size of the second contiguous block, a second physical machine identifier corresponding to the second computing device 124, a second RDMA access key, and a value of a second reference counter token representing one or more ownership interests in the second serialized object. The third computing device 126 may send the second remote descriptor to a fourth computing device 128.In some cases, the fourth computing device 128 is located on the same physical computer as the third computing device 126. In other cases, the fourth computing device 128 is located on a separate physical computer, and the third computing device 126 may send the second remote descriptor to the fourth computing device 128 over a second network connection (e.g., the network fabric). By sending the second remote descriptor to the fourth computing device 128, full ownership of the second serialized object is transferred to the fourth computing device 128.

[0032] In another embodiment, the third computing device 126 receives the second remote descriptor described above and generates a set of remote descriptors from the second remote descriptor. In this embodiment, the value of the second reference counter token is divided among the set of remote descriptors. For example, if the value of the second reference counter token is two and the third computing device 126 generates two remote descriptors, the value of each of the reference counter tokens of the two remote descriptors is set to one. An example of a remote descriptor is described below with reference to Fig. 2 shown.

[0033] Fig. 2 is a block diagram of a memory 202 storing a remote descriptor 204 and a contiguous block 206, according to at least one embodiment. The memory 202 may be one of the memory types described above with respect to Fig. 1, including the first memory 114, the second memory 116, the third memory 118, or the fourth memory 120. The remote descriptor 204 is an object that uniquely identifies or describes the contiguous block 206. The remote descriptor 204 may contain multiple elements to uniquely identify a location of a serialized object within a connected data center. That is, the memory 202 is accessible via a network connection, for example, via a network structure, to any computer devices in the data center subject to access control. The remote descriptor 204 contains a starting address 208 of the contiguous block 206, a size 210 of the contiguous block 206 (e.g., in bytes), a physical computer identifier 212 corresponding to the physical computer in which the memory 202 is located (e.g.,a host identifier, a device identifier), an RDMA access key 214, a reference counter token 216.

[0034] In at least one embodiment, remote descriptor 204 is generated and stored in a location identical to the contiguous block 206 it describes. That is, remote descriptor 204 is generated and stored in the same memory 202 as contiguous block 206. As mentioned above, remote descriptor 204 describes or identifies the contiguous block 206 of the serialized object. Contiguous block 206 includes a starting address 220 having the same value as starting address 208, a size 222 of contiguous block 206 having the same value as size 210, and a physical host identifier 224 having the same value as physical host identifier 212. The remote descriptor 204 has a size 218 that is typically less than 600 bytes, depending on the size of the RDMA access key 214. The size 218 may be, for example, 128-512 bytes or less than 128 bytes.In another example, size 218 may be 64 bytes. Alternatively, size 218 may also take on other values. However, the serialized object stored in contiguous block 206 has a physical size 222 (as noted by size 210 of remote descriptor 204) that may be larger, even several orders of magnitude larger (e.g., 1 gigabyte), than size 218 of remote descriptor 204. Remote descriptor 204 provides a language-level construct, similar to std::future in the RDMA library, that enables RDMA transport of the serialized object from contiguous block 206. Remote descriptor 204 reduces the transport of data across a compute boundary in a streaming data pipeline by eliminating the need for bulk data transfers at each compute boundary.The remote descriptor 204 enables the streaming data pipeline to be an on-demand model, where a request for the data is issued only when and where it is needed. The on-demand model allows the remote descriptor 204, rather than the serialized object, to traverse the various stages of the streaming pipeline without incurring the transfer costs of the serialized object. Only when needed, the on-demand model then enables asynchronous PULL operations to retrieve the serialized object from memory 202 on the other machine. In the context of streaming pipelines, moving the remote descriptor 204, rather than the raw data, through the pipeline can increase data throughput.

[0035] In at least one embodiment, once generated, the remote descriptor 204 may be forwarded to any physical host in the secure network fabric of the data center. The physical host receiving the remote descriptor 204 may perform an RDMA GET operation to transfer the contents of the contiguous block 206 (i.e., the remote memory for transfer to the receiving physical host) to the local memory of the physical host (i.e., the local memory of the receiving physical host). The remote descriptor 204 is deconstructed by the receiving physical host, and the receiving physical host may send a message to the physical host where the memory 202 is located, as described in the remote descriptor 204. The physical host where the memory 202 is located may receive the message and update the reference counter token 216.Once the reference counter token 216 goes to zero, the contiguous block 206 storing the serialized object can be released.

[0036] Fig. 3A shows a remote descriptor 304 generated by a first physical machine and sent to a second physical machine in a data center 300, according to at least one embodiment. Data center 300 includes physical machine A 302 and physical machine B 308. Physical machine A 302 generates remote descriptor 304, which uniquely identifies a serialized object stored in a contiguous block 306.

[0037] The remote descriptor 304 describes or identifies the contiguous block 306 in which a serialized object is stored. The remote descriptor 304 includes a starting address of the contiguous block 306, a size of the contiguous block 306 (e.g., in bytes), a physical host identifier corresponding to the physical host A 302 in which the contiguous block 306 is located (e.g., a host identifier, a device identifier), an RDMA access key, and a reference counter token. The contiguous block 306 includes a starting address having the same value as the starting address described in the remote descriptor 304, a size of the contiguous block 306 having the same value as the size described in the remote descriptor 304, and a physical host identifier having the same value as the physical host identifier described in the remote descriptor 304.The remote descriptor 304 has a size that is smaller than the size of the serialized object.

[0038] As in Fig. 3A, physical computer A 302 sends the remote descriptor 304 instead of the contiguous block 306 to physical computer B 308 (operation 312). Physical computer B 308 may process the remote descriptor 304 if necessary, as shown in Fig. 3B shown and described.

[0039] Fig. 3B shows the second physical computer performing an RDMA GET operation with the remote descriptor 304 from Fig. 3A. After physical computer B 308 receives remote descriptor 304, physical computer B 308 may deconstruct remote descriptor 304 to identify where the serialized object is located. Physical computer B 308 uses remote descriptor 304 to perform an RDMA GET operation 314, in which the contents of contiguous block 306 are transferred from physical computer A 302 to a contiguous block 310 on physical computer B 308 (316). Once physical computer B 308 has deconstructed remote descriptor 304, physical computer B 308 may send a message back to physical computer A 302, as in Fig. 3C shown and described.

[0040] Fig. Figure 3C shows the second physical computer sending a message to release the remote descriptor 304 from Fig. 3A, according to at least one embodiment. After physical machine B 308 deconstructs remote descriptor 304 and transfers the contents of contiguous block 306 into contiguous block 310 of physical machine B 308, physical machine B 308 may send a message 318 to physical machine A 302. Receipt of message 318 causes physical machine A 302 to update a value (e.g., decrement) of the reference counter token in the remote descriptor. In response to the value of the reference counter token meeting a threshold (e.g., zero), remote descriptor 304 may be released.

[0041] In one embodiment, the remote descriptor 304 is transferred to another physical computer B 308, as in Fig. 3A to Fig. 3C. In another embodiment, a remote descriptor may be divided into a set of remote descriptors, as shown in Fig. 4 and described below.

[0042] Fig. 4 illustrates the partitioning of a remote descriptor 402 into multiple remote descriptors 404a and 404b in a data center 400, according to at least one embodiment. In this embodiment, a computing device may receive or generate the remote descriptor 402. The remote descriptor 402 describes or identifies a contiguous block in which a serialized object is stored. The remote descriptor 402 includes a starting address of the contiguous block, a size of the contiguous block (e.g., in bytes), a physical machine identifier corresponding to a location where the contiguous block 406 is located (e.g., a host identifier, a device identifier), an RDMA access key, and a reference counter token 406. In this example, the reference counter token 406 has a value of 2, which may partition the remote descriptor 402 into two remote descriptors 404a and 404b.If the remote descriptor 402 is to be split into ten remote descriptors, the value of the reference count token 406 must be at least ten. Splitting the remote descriptor 402 into multiple remote descriptors allows the individual remote descriptors 404a and 404b to be sent to different physical machines or different stages of the streaming pipeline. 4 [sic].

[0043] As in Fig. 4, the remote descriptor 402 is split into two remote descriptors 404a and 404b. The remote descriptor 404a contains the same information describing or identifying the contiguous block in which the serialized object is stored, except that the remote descriptor 404a contains a reference count token 408 with a value of one. The remote descriptor 404b contains the same information describing or identifying the contiguous block in which the serialized object is stored, except that the remote descriptor 404b contains a reference count token 410 with a value of one. As shown in Fig. 4, the value of reference counter token 406 is divided among the set of remote descriptors 404a and 404b. In another embodiment, the value of reference counter token 406 could be greater than two and divided among the remote descriptors 404a and 404b. For example, the value could be five and divided such that reference counter token 408 has a value of three and reference counter token 410 has a value of two. The computing device can send each of the sets of remote descriptors 404a and 404b to one or more computing devices in the data center 400 over a network fabric.

[0044] As in Fig. 4, the remote descriptor 402 (and the remote descriptors 404a and 404b) may be smaller in size (i.e., fewer bytes) than the underlying serialized objects, facilitating copying and moving between nodes, segments, and hosts. The remote descriptor 402 (and the remote descriptors 404a and 404b) may allow the data to be moved only on request and to be obtained (e.g., pulled) from the source. As illustrated with the remote descriptors 404a and 404b, the remote descriptor 402 may be copied by splitting the reference count tokens. For example, if a remote descriptor has only two tokens (i.e., the value of the reference count token is two), it may only be duplicated once. The tokens may represent ownership interests, and at least one token is required for a remote descriptor.

[0045] Fig. 5 is a flowchart of a method 500 for generating and sending a remote descriptor, according to at least one embodiment. In at least one embodiment, the method 500 may be performed by the processing logic of a computing device, such as computing device 112, computing device 122, or the like. The processing logic may be a combination of hardware, firmware, software, or any combination thereof. The method 500 may be performed by one or more computing units (e.g., DPUs, CPUs, and / or GPUs) that include (or communicate with) one or more memory devices. In at least one embodiment, the first physical computer 112 executes Fig. 1 executes the method 500. In another embodiment, the physical computer B 110 of Fig. 1 executes the method 500. In at least one embodiment, the physical computer A 302 executes Fig. 3 executes the method 500. In another embodiment, the physical computer B of Fig. 3 illustrates the method 500. In at least one embodiment, the method 500 may be executed by multiple processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, processing threads implementing the method 500 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization logic). Alternatively, processing threads implementing the method 500 may be executed asynchronously with each other. Various operations of the method 500 may be executed in a different synchronization sequence than that described in Fig. 5. Some operations of the methods may be performed concurrently with other operations. In at least one embodiment, one or more of the Fig. The operations shown in Figure 5 are not always performed.

[0046] In block 502, the processing logic creates a remote descriptor associated with a serialized object. The remote descriptor is an object that uniquely identifies a first contiguous block of the serialized object stored in a first memory associated with the first computing device. In block 504, the processing logic sends the remote descriptor to a second computing device. The contents of the first contiguous block are retrieved from the first memory for storage in a second memory associated with the second computing device when the remote descriptor is processed by the second computing device.

[0047] Fig. 6 is a flowchart of a method 600 for using a remote descriptor to transfer a serialized object between computing devices, according to at least one embodiment. In at least one embodiment, the method 600 may be performed by the processing logic of a computing device, such as computing device 112, computing device 122, or the like. The processing logic may be a combination of hardware, firmware, software, or any combination thereof. The method 600 may be performed by one or more computing units (e.g., DPUs, CPUs, and / or GPUs) that include (or communicate with) one or more memory devices. In at least one embodiment, the first physical computer 112 executes Fig. 1 executes the method 600. In another embodiment, the physical computer B 110 of Fig. 1 executes the method 600. In at least one embodiment, the physical computer A 302 executes Fig. 3 executes the method 600. In another embodiment, the physical computer B of Fig. 3 illustrates the method 600. In at least one embodiment, the method 600 may be executed by multiple processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, processing threads implementing the method 600 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization logic). Alternatively, processing threads implementing the method 600 may be executed asynchronously with each other. Various operations of the method 600 may be executed in a different synchronization sequence than that described in Fig. 6. Some operations of the methods may be performed concurrently with other operations. In at least one embodiment, one or more of the Fig. The operations shown in Figure 6 are not always performed.

[0048] In block 602, the processing logic stores a serialized object in a first memory associated with the first computing device. In block 604, the processing logic generates a remote descriptor associated with the serialized object. The remote descriptor is an object that uniquely identifies a starting address of a contiguous block in the first memory, a size of the contiguous block, a physical machine identifier corresponding to the first memory, an RDMA access key, and a value of a reference counter token representing one or more ownership interests in the serialized object. In block 606, the processing logic sends the remote descriptor to a second computing device in the data center over a network fabric.Using the remote descriptor, the second computing device may retrieve the contents of the contiguous block from the first memory for storage in a second memory associated with the second computing device. In block 608, the processing logic receives a message from the second computing device to release the remote descriptor. In block 610, the processing logic updates the value of the reference counter token in response to receiving the message. In block 612, the processing logic releases the serialized object from the first memory in response to the value of the reference counter token meeting a threshold.

[0049] In another embodiment, a size of the remote descriptor is smaller than the size of the contiguous block. In another embodiment, the processing logic may receive a second remote descriptor associated with a second serialized object via the network fabric. The second remote descriptor is a second object that uniquely identifies: a second starting address of a second contiguous block in a third memory associated with a third computing device in the data center, a second size of the second contiguous block, a second physical machine identifier corresponding to the third memory, a second RDMA access key, and a value of a second reference counter token representing one or more ownership interests in the second serialized object.Using the second remote descriptor, the processing logic performs an RDMA GET operation to obtain the contents of the second contiguous block from the third memory for transfer to the first memory over the network structure.

[0050] In another embodiment, the processing logic may receive the second remote descriptor from a fourth computing device over the network fabric. In another embodiment, the processing logic may receive the second remote descriptor from another computing device in the data center.

[0051] In another embodiment, the processing logic may send a second message to the third computing device over the network structure to release the second remote descriptor. In response to the second message, the value of the second reference counter token is updated. The second serialized object is released in response to the value of the second reference counter token meeting a threshold.

[0052] In another embodiment, the processing logic may receive a second remote descriptor associated with a second serialized object over the network fabric. The second remote descriptor is a second object that uniquely identifies: a second starting address of a second contiguous block in a third memory associated with a third computing device in the data center, a second size of the second contiguous block, a second physical machine identifier corresponding to the third memory, a second RDMA access key, and a value of a second reference counter token representing one or more ownership interests in the second serialized object. The processing logic sends the second remote descriptor to the second computing device or a fourth computing device over the network fabric.Sending the second remote descriptor to the second or fourth computing device transfers full ownership of the second serialized object to the second or fourth computing device.

[0053] In another embodiment, the processing logic may receive a second remote descriptor associated with a second serialized object via the network fabric. The second remote descriptor is a second object that uniquely identifies: a second starting address of a second contiguous block in a third memory associated with a third computing device, a second size of the second contiguous block, a second physical machine identifier corresponding to the third memory, a second RDMA access key, and a value of a second reference count token representing one or more ownership interests in the second serialized object. The processing logic generates a set of remote descriptors from the second remote descriptor. A value of the second reference count token is apportioned among the set of remote descriptors.The processing logic may send each of the set of remote descriptors over the network fabric to one of a set of computing devices in the data center. The set of computing devices may include the second computing device described above. STREAMING STACKER

[0054] The following section describes embodiments related to generating, sending, and processing streaming batches with multiple remote descriptors to enable RDMA transport of a serialized object in streaming pipelines implemented with multiple computing devices in a data center. As described above, traditional batchers reside outside the data plane and require the entire data payload to be collected and forwarded, resulting in at least a doubling of data movement across compute boundaries.Aspects and embodiments of the present disclosure address the above and other deficiencies by providing a streaming stack of multiple remote descriptors that enable at least some contents of a serialized object to be obtained from a first memory for storage in a second memory before the streaming stack is complete, as described below with reference to FIG. Fig. 7A to 7B.

[0055] Fig. 7A is a block diagram of a first computing device 702 in a data center 700 for generating a streaming batch 704 in a first instance, according to at least one embodiment. The data center 700 includes a first computing device 702 with a streaming stacker. Streaming stacker refers to processing logic, which may be a combination of hardware, firmware, software, or any combination thereof, that can generate, manage, and send a streaming batch with one or more remote descriptors, where each remote descriptor uniquely identifies a location of a serialized object in a data center. The streaming stacker collects individual objects in a collection, called a streaming batch 704, by number and / or by time. The streaming batch 704 may have a fixed number of slots (e.g.,N slots for individual batch items) and a batching window 710 (also called a time window) in which a timer starts, in a first instance, upon assignment or acceptance of a first batch item. Slots are assigned to batch items, and each slot corresponds to a batch item. As shown in an example, the streaming batch 704 includes four slots, including a slot 1 712, a slot 2 714, a slot 3 716, and a slot 4 718. The streaming batcher may collect work items as remote descriptors (also called in a first instance or streaming batch items). The streaming batcher may assign a remote descriptor (RD) 708 (unique item A) to slot 1 712 as a first instance to start the timer of the batching window 710.Upon the arrival of the first stack item RD 708, a stream is initiated or opened for the downstream consumers 706 that can operate on the streaming stack 704, and each stack item (remote descriptor) is streamed to the downstream consumers 706 upon arrival. This means that after assigning the RD 708 to slot 1 712, the streaming stacker can immediately forward the RD 708 to a downstream consumer 706 (also referred to as a downstream destination) before a stacking window 710 completes. The streaming stacker has completed the streaming stack 704 when either all slots are filled or the stacking window 710 ends (i.e., the stacking window 710 timer expires), whichever occurs first. With the completion of the streaming stack 704, the stream also terminates or closes.

[0056] As in Fig. 7A, the first computing device 702 creates the streaming batch 704, allocates the RD 708 in the first instance to start the batching window 710, and immediately sends the RD 708 to the downstream consumer 706. The operations of the downstream consumer 706 are described below with reference to Fig. 7B.

[0057] Fig. 7B is a block diagram of a downstream consumer 706 in a data center 700 for processing the RD 708 in a second instance while the stack window 710 is still open, according to at least one embodiment. The downstream consumers 706 may be one or more computing devices in the data center 700. As described above, the first computing device 702 may forward the remote descriptors in the streaming stack 704 that have smaller payloads than the underlying objects (e.g., 1 kilobyte (kb) vs. 10 megabytes (mb)), and the downstream consumers 706 may begin some operations on individual stack items of the streaming stack 704 before the streaming stack 704 completes. The downstream consumer 706 may receive the individual remote descriptors as they come through the stack stream.The RD 708 can reduce the need to move large amounts of data (serialized object) to the downstream consumer 706 while batching multiple serialized objects in the batching window 710. By forwarding the incoming work packets (batch items) to the intended downstream consumer 706 immediately upon their arrival, the downstream consumer 706 can instantiate an operation (e.g., a PULL operation, an RDMA GET operation, or the like) to retrieve the remote data (serialized object) identified in the RD 708 while the batching window 710 is still open (i.e., before the streaming batch 704 completes). The downstream consumer 706 can issue one or more RDMA GET operations on the payload to obtain the remote data (local to the originating device) upon receipt of the remote descriptor. This allows an overlap of the stacking window 710 and the RDMA communication.In at least one embodiment, the downstream consumer 706 may instantiate a first RDMA GET operation to obtain payload 1 720 of a serialized object described by the RD 708, a second RDMA GET operation to obtain payload 2 722 of a serialized object described by the RD 708, a third RDMA GET operation to obtain payload 3 724 of a serialized object described by the RD 708, and a fourth RDMA GET operation to obtain payload 4 726 of a serialized object described by the RD 708. The RDMA GET operations may overlap with the batching window 710 (e.g., 9.9 ms in the timer).

[0058] By stacking remote descriptors as stack items of the streaming stack 704, the streaming stacker helps reduce the cost of duplicate large data transfers by logically stacking the remote descriptors rather than the actual data (serialized objects) described in the remote descriptors in the streaming stack 704. The streaming stacker can reduce data movement by orders of magnitude by stacking and transferring the remote descriptors rather than the serialized objects. The streaming stacker can reduce overall latency by applying overlap in retrieving the remote data with the streaming stacker's windowing.

[0059] In at least one embodiment, during the stacking window 710 (e.g., 9.9 ms remaining on the timer), the streaming stacker may allocate a second RD 728 (unique item B) to slot 2 714 of the streaming stack 704. Once the second RD 728 is allocated, the streaming stacker sends the RD 720 to the downstream consumers 706. As described above, the streaming stacker may continue allocating RDs to the remaining slots of the streaming stack 704 until all slots are filled or the streaming stack 704 timer expires.

[0060] Fig. 8 is a flowchart 800 of a method for generating a streaming batch, according to at least one embodiment. In at least one embodiment, the method 800 may be performed by the processing logic of a computing device, such as computing device 112, computing device 122, or the like. The processing logic may be a combination of hardware, firmware, software, or any combination thereof. The method 800 may be performed by one or more computing units (e.g., DPUs, CPUs, and / or GPUs) that include (or communicate with) one or more memory devices. In at least one embodiment, the first physical computer 112 executes Fig. 1 executes the method 800. In another embodiment, the physical computer B 110 of Fig. 1 executes the method 800. In at least one embodiment, the physical computer A 302 executes Fig. 3 executes the method 600. In another embodiment, the physical computer B of Fig. 3 illustrates the method 900. In at least one embodiment, the method 800 may be executed by multiple processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, processing threads implementing the method 800 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization logic). Alternatively, processing threads implementing the method 800 may be executed asynchronously with each other. Various operations of the method 800 may be executed in a different order than that described in Fig. 8. Some operations of the methods may be performed concurrently with other operations. In at least one embodiment, one or more of the Fig. The operations shown in Figure 8 are not always performed.

[0061] In block 802, the processing logic creates a streaming batch of a set of remote descriptors, each being an object that uniquely identifies a contiguous block of a serialized object stored in a first memory associated with the first computing device. In block 804, the processing logic sends at least one of the set of remote descriptors to a second computing device, for example, over a network structure, before the streaming batch is complete. At least some contents of a contiguous block are obtained from the first memory for storage in a second memory associated with the second computing device before the streaming batch is complete.

[0062] Fig. 9 is a flowchart of a method 900 for using a streaming stack to transfer multiple streaming objects, according to at least one embodiment. In at least one embodiment, the method 900 may be performed by the processing logic of a computing device, such as computing device 112, computing device 122, or the like. The processing logic may be a combination of hardware, firmware, software, or any combination thereof. The method 900 may be performed by one or more computing units (e.g., DPUs, CPUs, and / or GPUs) that include (or communicate with) one or more memory devices. In at least one embodiment, the first physical computer 112 executes Fig. 1 executes the method 900. In another embodiment, the physical computer B 110 of Fig. 1 executes the method 900. In at least one embodiment, the physical computer A 302 executes Fig. 3 executes the method 900. In another embodiment, the physical computer B of Fig. 3 illustrates the method 900. In at least one embodiment, the method 900 may be executed by multiple processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, processing threads implementing the method 900 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization logic). Alternatively, processing threads implementing the method 900 may be executed asynchronously with each other. Various operations of the method 900 may be executed in a different synchronization sequence than that described in Fig. 10. Some operations of the methods may be performed concurrently with other operations. In at least one embodiment, one or more of the Fig. 10 operations shown are not always carried out.

[0063] In block 902, the processing logic assigns a first remote descriptor to a first slot of a first streaming batch during a first time window. The first remote descriptor is an object that uniquely identifies a first contiguous block of a first serialized object stored in a first memory associated with the first computing device. In block 904, the processing logic sends the first remote descriptor over a network fabric to a second computing device in the data center before the first streaming batch completes. Sending the first remote descriptor opens a stream between the first memory and the second computing device. In block 906, the processing logic assigns a second remote descriptor to a second slot of the first streaming batch during the first time window.The second remote descriptor is an object that uniquely identifies a second contiguous block of a second serialized object stored in the first memory. In block 908, the processing logic sends the second remote descriptor over the network fabric to the second computing device before the first streaming batch completes. In block 910, the processing logic completes the first streaming batch in response to i) all slots of the first streaming batch being allocated or ii) the first time window ending. The second computing device obtains at least some contents of the first contiguous block from the first memory for storage in a second memory associated with the second computing device before the first streaming batch completes.

[0064] In another embodiment, the first remote descriptor includes a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to the first memory, an RDMA access key, and a value of a reference counter token representing one or more ownership interests in the first serialized object. A size of the first remote descriptor is smaller than the size of the first contiguous block.

[0065] In another embodiment, the processing logic may also receive a first message from the second computing device to release the first remote descriptor. The processing logic updates the value of the reference counter token in response to receiving the message. The processing logic may release the first serialized object from the first memory in response to the value of the reference counter token meeting a threshold.

[0066] In another embodiment, the processing logic may initialize the first streaming stack to contain a certain number of slots, where each slot corresponds to a single streaming stack item. The first time slot starts in response to the allocation of a first streaming stack item to the first streaming stack. The received contents of the first contiguous block are processed by the second computing device once the first streaming stack is complete.

[0067] In another embodiment, the processing logic may also receive a third remote descriptor associated with a second streaming batch during a second time window. The third remote descriptor is a second object that uniquely identifies a second contiguous block of a second serialized object stored in a third memory associated with a third computing device in the data center. Using the third remote descriptor, the processing logic performs an RDMA GET operation to obtain at least some contents of the second contiguous block from the third memory for transfer to the first memory over the network fabric.

[0068] In another embodiment, the processing logic may receive the second remote descriptor from a fourth computing device over the network structure.

[0069] The processing logic may also send a second message over the network fabric to the third computing device to release the second remote descriptor. The second remote descriptor includes a starting address of the second contiguous block, a size of the second contiguous block, a physical machine identifier corresponding to the third memory, an RDMA access key, and a value of a reference counter token representing one or more ownership interests in the second serialized object. A size of the second remote descriptor is smaller than the size of the second contiguous block. In response to the message, the value of the second reference counter token is updated. The second serialized object is released in response to the value of the reference counter token meeting a threshold. REMOTE-PROMISE-FUTURE-OBJECTS

[0070] The following section describes embodiments related to creating, sending, and processing remote promise objects and remote promise future objects to enable downstream nodes to obtain data from upstream storage and enable downstream nodes to send data (e.g., PUSH / PUT operation) to upstream nodes in streaming pipelines implemented with multiple computing devices in a data center, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH / PUT operation is completed.

[0071] Stateful streaming pipelines can be challenging when downstream consumers need to update their upstream state. Without the concept of remote Promise objects and remote Promise Future objects, these stateful algorithms must either be restructured to be stateless or create a serialization bottleneck that can impact performance.

[0072] Aspects and embodiments of the present disclosure address the above and other deficiencies by providing a remote promise object or a remote promise future object to enable downstream nodes to obtain data from an upstream storage, and enable downstream nodes to send data (e.g., push / put operation) to upstream nodes while those upstream nodes relinquish their execution context and do not consume resources until the upstream push operation completes. A remote promise object is an object that uniquely identifies a contiguous block of memory in which a serialized object is stored.The remote promise object identifies a starting address of a contiguous block in local memory, a size of the contiguous block, a physical machine identifier corresponding to the local memory, an RDMA access key, and a value of a reference counter token that represents unique ownership of the serialized object because the remote promise object is unique. The remote promise object is a language construct or control structure that can be exchanged with another computing device, which can complete the remote promise object by sending a message back to the device sending the remote promise object. A remote promise future object is an object that uniquely identifies a contiguous block in memory storing a first serialized object and specifies how the remote promise future object will be completed.The remote promise future object may be assigned to a first serialized object stored on a first physical machine in a data center and may specify that the remote promise future object complete when a second physical machine in the data center returns a second serialized object to the first physical machine. The remote promise future object is a language construct or control structure that can be exchanged with another computing device that can complete the remote promise future object by delivering an object back to the device sending the remote promise future object. The remote promise object and the remote promise future object may provide language-level constructs similar to std::promise and std::future, respectively, that enable upstream PUSH / PUT data and indication that an upstream node can proceed.The remote promise object and the remote promise future object are language constructs that represent the conceptual counterparts of the remote descriptor. Remote descriptors can allow a downstream node to retrieve data from upstream storage, and the remote promise object and remote promise future object constructs allow downstream nodes to send data (e.g., push / put operations) to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream push / put operation completes.

[0073] Fig. 10 shows a remote promise object 1004 created by a first physical computer and sent to a second physical computer in a data center 1000, according to at least one embodiment. The data center 1000 includes a physical computer A 1002 and a physical computer B 1008. The physical computer A 1002 creates the remote promise object 1004 that uniquely identifies a serialized object stored in a contiguous block 1006. The remote promise object 1004 describes or identifies the contiguous block 1006 in which a serialized object is stored. The remote promise object 1004 contains a start address of the contiguous block 1006, a size of the contiguous block 1006 (e.g., in bytes), a physical computer identifier corresponding to the physical computer A 1002 in which the contiguous block 1006 is located (e.g.,a host identifier, a device identifier), an RDMA access key, and a reference counter token. The contiguous block 1006 includes a starting address that has the same value as the starting address described in the remote promise object 1004, a size of the contiguous block 1006 that has the same value as the size described in the remote promise object 1004, and a physical host identifier that has the same value as the physical host identifier described in the remote promise object 1004. The remote promise object 1004 has a size that is smaller than the size of the serialized object.

[0074] As in Fig. 10, physical host A 1002 sends remote promise object 1004 to physical host B 1008 (operation 1010) instead of contiguous block 1006. Physical host B 1008 may process remote promise object 1004 as needed, as shown and described below.

[0075] After physical host B 1008 receives remote promise object 1004, physical host B 1008 may deconstruct remote promise object 1004 to identify where the serialized object is located. Physical host B 1008 uses remote promise object 1004 to perform an RDMA GET operation 1012, transferring the contents of contiguous block 1006 from physical host A 1002 to a contiguous block on physical host B 1008 (operation 1014). Once physical host B 1008 has deconstructed remote promise object 1004, physical host B 1008 may send a message 1016 back to physical host A 1002. Receiving message 1016 causes physical machine A 1002 to update (e.g., decrement) a value of the reference counter token in the remote promise object 1004. In response to the value of the reference counter token exceeding a threshold (e.g.,Zero), the remote promise object 1004 can be released. In this example, the remote promise object 1004 is unique, so the value of the reference count token of the remote promise object 1004 can be initially set to one and decremented to zero upon receipt of message 1016, thereby releasing the remote promise object 1004.

[0076] Fig. 11 shows a remote promise future object created by a first physical machine and sent to a second physical machine in a data center 1100, according to at least one embodiment. Data center 1100 includes physical machine A 1102 and physical machine B 110. Physical machine A 1102 creates remote promise future object 1104 that uniquely identifies a first serialized object stored in a contiguous block 1106. Remote promise object 1104 describes or identifies the contiguous block 1106 in which the first serialized object is stored and specifies that remote promise future object 1104 is complete when a second object 1110 is returned to physical machine A 1102. The remote promise future object 1104 contains a starting address of the contiguous block 1106, a size of the contiguous block 1106 (e.g.in bytes), a physical host identifier corresponding to the physical host A 1102 in which the contiguous block 1106 is located (e.g., a host identifier, a device identifier), an RDMA access key, and a reference counter token. The contiguous block 1106 includes a starting address that has the same value as the starting address described in the remote promise future object 1104, a size of the contiguous block 1106 that has the same value as the size described in the remote promise future object 1104, and a physical host identifier that has the same value as the physical host identifier described in the remote promise future object 1104. The remote promise future object 1104 has a size that is smaller than the size of the first serialized object.

[0077] As in Fig. 11, physical host A 1102 sends remote promise future object 1104 to physical host B 110 (operation 1112) instead of contiguous block 1106. Physical host B 110 may process remote promise future object 1104 as needed, as shown and described below.

[0078] After physical host B 110 receives the remote promise future object 1104, physical host B 110 may deconstruct the remote promise future object 1104 to identify where the first serialized object is located. Physical host B 110 uses the remote promise future object 1104 to perform an RDMA GET operation 1114, transferring the contents of the contiguous block 1106 from physical host A 1102 to a contiguous block on physical host B 110. The contiguous block may be the same location where the second object 1110 is stored. Physical host B 110 processes the first serialized object and creates the second object 1110. Physical host B 110 may serialize the second object 1110 and send the second object 1110 to physical host A 1102.In at least one embodiment, physical machine B 110 sends the second object 1110 by performing an RDMA PUT operation 1116 to push the second object 1110 (e.g., the serialized object) to complete the remote promise future object 1104. That is, physical machine B 110 completes the remote promise future object 1104 by sending the second object 1110 back to physical machine A 1102. In another embodiment, physical machine B 110 may send a message back to physical machine A 1102 (in . Fig. 11 not shown). Receipt of the message causes physical machine A 1102 to update (e.g., decrement) a value of the reference counter token in remote promise future object 1104. In response to the value of the reference counter token meeting a threshold (e.g., zero), remote promise future object 1104 may be freed. In this example, remote promise future object 1104 is unique, so the value of the reference counter token of remote promise future object 1104 may be initially set to one and decremented to zero upon receipt of the message, thereby freeing remote promise future object 1104.

[0079] As in Fig. 11, physical machine A 1102 can use the following sample code: auto f = remote_promise.get_future(); / / Block the execution of the network fabric on physical machine A / / awaiting the completion of the remote promise future object on a remote node auto data = f.get(); / / data is now the object that was moved from physical machine B to physical machine A / / the network fabric is now active again.

[0080] As in Fig. 11, the physical computer B 110 can use the following example code: remote_promise.set_value (std::move(my_object)

[0081] Fig. 12 shows a remote promise future object 1104 created by a first physical computer and sent to a second physical computer in the data center 1200, according to at least one embodiment. The data center 1200 of Fig. 12 is similar to the data center 1100 of Fig. 11, except that physical host B 1108 does not return a result to complete the remote promise future object 1104. Rather, physical host B 1108 returns a message 1202 to complete the remote promise future object 1104 by releasing the remote promise future object 1104. Receipt of message 1202 causes physical host A 1102 to update (e.g., decrement) a value of the reference count token in the remote promise future object 1104. In response to the value of the reference count token meeting a threshold (e.g., zero), the remote promise future object 1104 may be released, resulting in an exception being thrown.

[0082] As shown in Figure 121, physical machine A 1102 may use the following sample code: auto f = remote_promise.get_future(); / / Block the execution of the network fabric on physical machine A / / awaiting the completion of the remote promise future object on a remote node auto data = f.get(); / / data is now the object that was moved from physical machine B to physical machine A / / the network fabric is now active again.

[0083] Fig. 13 is a flowchart of a method 1300 for creating and sending a remote promise object, according to at least one embodiment. In at least one embodiment, the method 1300 may be performed by the processing logic of a computing device, such as computing device 112, computing device 122, or the like. The processing logic may be a combination of hardware, firmware, software, or any combination thereof. The method 1300 may be performed by one or more computing units (e.g., DPUs, CPUs, and / or GPUs) that include (or communicate with) one or more memory devices. In at least one embodiment, the first physical computer 112 executes Fig. 1 executes the method 1300. In another embodiment, the physical computer B 110 of Fig. 1 executes method 1400. In at least one embodiment, physical computer A 302 executes Fig. 3 performs the method 1300. In another embodiment, the physical computer B of Fig. 3 illustrates the method 1300. In at least one embodiment, the method 1300 may be executed by multiple processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, processing threads implementing the method 1300 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization logic). Alternatively, processing threads implementing the method 1300 may be executed asynchronously with each other. Various operations of the method 1300 may be executed in a different synchronization sequence than that described in Fig. 13. Some operations of the methods may be performed concurrently with other operations. In at least one embodiment, one or more of the operations in Fig. 13 operations are not always performed.

[0084] In block 1302, processing logic of a first computing device in a data center receives a remote promise object (or a remote promises future object) associated with a first serialized object from a second computing device in the data center over a network fabric. The remote promise object uniquely identifies a first contiguous block of the first serialized object stored in memory associated with the second computing device. The remote promise object is released when a second serialized object is returned to the second computing device. In block 1304, the processing logic obtains the contents of the first contiguous block from memory associated with the second computing device for storage in memory associated with the first computing device.In block 1306, the processing logic sends the contents of a second contiguous block of the second serialized object stored in the memory associated with the first computing device to the memory associated with the second computing device. In block 1308, the processing logic sends a message to the second computing device to release the remote Promise object in response to the second serialized object being returned to the second computing device.

[0085] In another embodiment, the processing logic releases the remote promise object by returning the second serialized object to the first computing device without sending the message in block 1308.

[0086] In another embodiment, the remote promise object includes a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to memory associated with the second computing device, an RDMA access key, and a value of a reference counter token representing an ownership interest in the first serialized object.

[0087] In another embodiment, the processing logic, by sending the message to the second computing device, causes the second computing device to update the value of the reference counter token. The first serialized object is released from the memory associated with the first computing device when the value of the reference counter token meets a threshold (e.g., is equal to zero).

[0088] In another embodiment, the processing logic may obtain the contents of the first contiguous block by performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the memory associated with the second computing device and transfer it over the network fabric to the memory associated with the first computing device. The processing logic may send the contents of the second contiguous block by performing an RDMA PUT operation to send the contents of the second serialized object from the memory associated with the first computing device to the memory associated with the second computing device over the network fabric.

[0089] In another embodiment, the processing logic may create a second remote promise object. The second remote promise object is an object that uniquely identifies a third contiguous block of a third serialized object stored in memory associated with the first computing device. The second remote promise object is released when a fourth serialized object is returned to the first computing device. The processing logic may send the second remote promise object over the network fabric to another computing device in the data center. The processing logic may receive a second message from the other computing device to release the second remote promise object in response to the fourth serialized object being returned to the first computing device.In another embodiment, the processing logic may update the value of the reference counter token in response to receiving the second message. The processing logic may release the remote promise object from the first memory in response to the value of the reference counter token meeting a threshold. DATA CENTER

[0090] Fig. 14 shows an example of a data center 1400 in which at least one embodiment may be used. In at least one embodiment, the data center 1400 includes a data center infrastructure layer 1402, a framework layer 1404, a software layer 1406, and an application layer 1408.

[0091] In at least one embodiment, as in Fig. 14, the data center infrastructure layer 1402 may include a resource orchestrator 1410, clustered compute resources 1412, and node compute resources ("node CRs") 1414a(1)-1414c(N), where "N" represents any integer positive number. In at least one embodiment, the node CRs 1414a(1)-1414c(N) may include, but are not limited to, any number of central processing units ("CPUs") or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or hard disk drives), network input / output devices ("NW I / O"), network switches, virtual machines ("VMs"), power modules and cooling modules, etc. In at least one embodiment, one or more Node CRs may be included among the Node CRs 1414a(1)-1414c(N) may be a server with one or more of the computing resources listed above.

[0092] In at least one embodiment, the grouped computing resources 1412 may include separate groupings of node CRs housed in one or more racks (not shown) or multiple racks located in data centers in different geographic locations (also not shown). Separate groupings of node CRs within the grouped computing resources 1412 may include grouped computing, networking, memory, or storage resources that may be configured or allocated to support one or more workloads. In at least one embodiment, multiple node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources to support one or more workloads.In at least one embodiment, one or more racks may also contain any number of power modules, cooling modules, and network switches in any combination.

[0093] In at least one embodiment, resource orchestrator 1410 may configure or otherwise control one or more node CRs 1414a(1)-1414c(N) and / or clustered computing resources 1412. In at least one embodiment, resource orchestrator 1410 may include a software design infrastructure ("SDI") management entity for data center 1400. In at least one embodiment, resource orchestrator 1410 may include hardware, software, or a combination thereof.

[0094] In at least one embodiment, as in Fig. 14, the framework layer 1404 includes a job scheduler 1416, a configuration manager 1418, a resource manager 1420, and a distributed file system 1422. In at least one embodiment, the framework layer 1404 may include a framework for supporting software 1424 of the software layer 1406 and / or one or more applications 1426 of the application layer 1408. In at least one embodiment, the supporting software 1424 or the application(s) 1426 may each include web-based service software or applications such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 1406 may be some type of free and open source software web application framework such as, but not limited to, Apache Spark™ (hereinafter "Spark"), which may utilize a distributed file system 1422 for processing large amounts of data (e.g., "Big Data").In at least one embodiment, job scheduler 1416 may include a Spark driver to facilitate scheduling workloads supported by various layers of data center 1400. In at least one embodiment, configuration manager 1418 may be capable of configuring various layers, such as software layer 1406 and framework layer 1404, including Spark and distributed file system 1422 to support processing large amounts of data. In at least one embodiment, resource manager 1420 may be capable of managing clustered or grouped compute resources allocated or assigned to support distributed file system 1422 and job scheduler 1416. In at least one embodiment, clustered or grouped compute resources may include grouped compute resources 1412 at infrastructure layer 1402 of the data center.In at least one embodiment, the resource manager 1420 may be coordinated with the resource orchestrator 1410 to manage these allocated or assigned computing resources.

[0095] In at least one embodiment, the support software 1424 included in software layer 1406 may include software used by at least portions of node CRs 1414a(1)-1414c(N), clustered computing resources 1412, and / or distributed file system 1422 of framework layer 1404. One or more types of software may include, but are not limited to, web page searching software, email virus scanning software, database software, and video content streaming software.

[0096] In at least one embodiment, the applications 1426 included in the application layer 1408 may include one or more types of applications used by at least portions of the node CRs 1414a(1)-1414c(N), the clustered compute resources 1412, and / or the distributed file system 1422 of the framework layer 1404. One or more types of applications may include, but are not limited to, any number of genomic applications, cognitive computing, and a machine learning application, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in connection with one or more embodiments.

[0097] In at least one embodiment, configuration manager 1418, resource manager 1420, and resource orchestrator 1410 may implement any number and type of self-modifying actions based on any amount and type of data collected in any technically feasible manner. In at least one embodiment, self-modifying actions may relieve a data center operator of data center 1400 from potentially making poor configuration decisions and potentially avoiding underutilized and / or malfunctioning portions of a data center.

[0098] In at least one embodiment, data center 1400 may include tools, services, software, or other resources to train one or more machine learning models or to predict or infer information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by calculating weight parameters according to a neural network architecture using software and computing resources described above with respect to data center 1400.In at least one embodiment, trained machine learning models corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to data center 1400 by using weighting parameters calculated by one or more training techniques described herein.

[0099] In at least one embodiment, data center 1400 may utilize CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above may be configured as a service to enable users to train or infer information, such as image capture, speech capture, or other artificial intelligence services.

[0100] Transfer logic 1428 may be used to perform operations related to remote descriptors, remote promise objects, remote promise future objects, streaming stacks, and RDMA operations in connection with one or more embodiments. Details of transfer logic 1428 are described below in connection with Fig. 1 to Fig. 13. In at least one embodiment, the transfer logic 1428 may be implemented in the Fig. 14 to enable RDMA transport of serialized objects in the streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 14 to enable RDMA transport of serialized objects in streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 14 to enable downstream nodes to receive data from upstream storage and to enable downstream nodes to PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is completed, as described herein. CONTROL SYSTEM

[0101] Fig. 15 is a block diagram illustrating an exemplary computer system 1500, which may be a system with interconnected devices and components, a system-on-a-chip (SOC), or a combination thereof, formed with a processor that may include execution units for executing an instruction in accordance with at least one embodiment. In at least one embodiment, the computer system 1500 may include, without limitation, a component, such as a processor 1502, for employing execution units including logic for performing algorithms for processing data in accordance with the present disclosure, as in the embodiments described herein. In at least one embodiment, the computer system 1500 may include processors, such asthe PENTIUM® processor family, Xeon™, Itanium®, XScale™ and / or StrongARM™, Intel® Core™ or Intel® Nervana™ microprocessors available from Intel Corporation of Santa Clara, California, USA, although other systems (including PCs with other microprocessors, technical workstations, set-top boxes, and the like) may be used. In at least one embodiment, computer system 1500 may run a version of the WINDOWS operating system available from Microsoft Corporation of Redmond, Washington, USA, although other operating systems (e.g., UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0102] Embodiments may be used in other devices such as handheld devices and embedded applications. Some examples of handheld devices are cellular phones, Internet Protocol devices, digital cameras, personal digital assistants ("PDAs"), and handheld PCs. In at least one embodiment, embedded applications may include a microcontroller, a digital signal processor ("DSP"), a system on a chip, network computers ("NetPCs"), set-top boxes, network hubs, wide area network ("WAN") switches, or any other system capable of executing one or more instructions according to at least one embodiment.

[0103] In at least one embodiment, computer system 1500 may include, without limitation, a processor 1502, which may include, without limitation, one or more execution units 1508 to perform operations according to the techniques described herein. In at least one embodiment, computer system 1500 is a single-processor desktop or server system, but in another embodiment, computer system 1500 may be a multiprocessor system. In at least one embodiment, processor 1502 may be, without limitation, a complex instruction set ("CISC") microprocessor, a reduced instruction set ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, a processor implementing a combination of instruction sets, or any other processor device, such as a digital signal processor.In at least one embodiment, the processor 1502 may be connected to a processor bus 1512 that may transmit data signals between the processor 1502 and other components in the computer system 1500.

[0104] In at least one embodiment, processor 1502 may include, without limitation, an internal level 1 ("L1") cache 1504. In at least one embodiment, processor 1502 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache may be external to processor 1502. Other embodiments may also include a combination of both internal and external caches, depending on the particular implementation and needs. In at least one embodiment, register file 1506 may store various types of data in various registers, including, without limitation, integer registers, floating-point registers, status registers, and instruction pointer registers.

[0105] In at least one embodiment, an execution unit 1508 is also located within processor 1502, including, without limitation, logic for performing integer and floating-point operations. In at least one embodiment, processor 1502 may also include microcode ("ucode") read-only memory ("ROM") that stores microcode for certain macroinstructions. In at least one embodiment, execution unit 1508 may include logic for processing a packaged instruction set 1510. In at least one embodiment, by including a packaged instruction set 1510 in an instruction set of a general-purpose processor 1502, along with associated instruction execution circuitry, operations used by many multimedia applications may be performed using packed data in a general-purpose processor 1502.In one or more embodiments, many multimedia applications can be accelerated and executed more efficiently by utilizing the full width of a processor's data bus to perform operations on packed data, eliminating the need to transfer smaller units of data across the processor's data bus to perform one or more operations on one data element at a time.

[0106] In at least one embodiment, execution unit 1508 may also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, computer system 1500 may include, without limitation, memory 1524. In at least one embodiment, memory 1524 may be implemented as dynamic random access memory ("DRAM"), static random access memory ("SRAM"), flash memory, or other memory. In at least one embodiment, memory 1524 may store one or more instructions 1522 and / or data 1526 represented by data signals that may be executed by processor 1502.

[0107] In at least one embodiment, the system logic chip may be connected to the processor bus 1512 and the memory 1524. In at least one embodiment, the system logic chip may include, without limitation, a memory controller hub ("MCH") 1518, and the processor 1502 may communicate with the MCH 1518 via the processor bus 1512. In at least one embodiment, the MCH 1518 may provide a high-bandwidth memory path 1520 to the memory 1524 for instruction and data storage and for the storage of graphics instructions, data, and textures. In at least one embodiment, the MCH 1518 may route data signals between the processor 1502, the memory 1524, and other components in the computer system 1500, and may bridge data signals between the processor bus 1512, the memory 1524, and a system I / O 1528.In at least one embodiment, a system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH 1518 may be coupled to the memory 1524 via a high-bandwidth memory path 1520, and the graphics / video card 1514 may be coupled to the MCH 1518 via an Accelerated Graphics Port ("AGP") connection 1516.

[0108] In at least one embodiment, computer system 1500 may use system I / O 1528, which is a proprietary hub interface bus, to connect MCH 1518 to I / O controller hub ("ICH") 1544. In at least one embodiment, ICH 1544 may provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, a local I / O bus may include, without limitation, a high-speed I / O bus for connecting peripheral devices to memory 1524, chipset, and processor 1502.Examples may include, without limitation, an audio controller 1542, a firmware hub ("Flash BIOS") 1540, a wireless transceiver 1536, a data storage 1532, a legacy I / O controller 1530 including a user input and a user input interface 1534, a serial expansion port 1538 such as the Universal Serial Bus ("USB"), and a network controller 1546. The data storage 1532 may include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage devices.

[0109] In at least one embodiment, Fig. 15 a system 1500 comprising interconnected hardware devices or “chips”, while in other embodiments Fig. 15 may illustrate an exemplary system on a chip ("SoC"). In at least one embodiment, the devices may be interconnected using proprietary interconnects, standardized interconnects (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of computer system 1500 are interconnected via Compute Express Link (CXL) interconnects.

[0110] Transfer logic 1428 may be used to perform operations related to remote descriptors, remote promise objects, remote promise future objects, streaming stacks, and RDMA operations in connection with one or more embodiments. Details of transfer logic 1428 are described below in connection with Fig. 1 - Fig. 13. In at least one embodiment, the transfer logic 1428 may be implemented in the Fig. 15 to enable RDMA transport of serialized objects in the streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 15 to enable RDMA transport of serialized objects in streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 15 to enable downstream nodes to receive data from upstream storage and enable downstream nodes to PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is completed, as described herein.

[0111] Fig. 16 is a block diagram illustrating an electronic device 1600 for using a processor 1610 according to at least one embodiment. In at least one embodiment, the electronic device 1600 may be, for example and without limitation, a notebook, a tower server, a rack server, a blade server, a laptop, a desktop, a tablet, a mobile device, a phone, an embedded computer, or any other suitable electronic device.

[0112] In at least one embodiment, the electronic system device 1600 may include, without limitation, a processor 1602 communicatively coupled to any number or type of components, peripherals, modules, or devices. In at least one embodiment, the processor 1602 is coupled via a bus or interface, such as an I2C bus, a System Management Bus ("SMBus"), a Low Pin Count (LPC) bus, a Serial Peripheral Interface ("SPI"), a High Definition Audio ("HDA") bus, a Serial Advance Technology Attachment ("SATA") bus, a Universal Serial Bus ("USB") (versions 1, 2, 3), or a Universal Asynchronous Receiver / Transmitter ("UART") bus. In at least one embodiment, Fig. 16 a system comprising interconnected hardware devices or “chips”, while in other embodiments Fig. 16 may illustrate an exemplary system on a chip (“SoC”). In at least one embodiment, the Fig. 16 may be interconnected with proprietary connections, standardized connections (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of Fig. 16 connected via Compute Express Link (CXL) connections.

[0113] In at least one embodiment, Fig. 16 a display 1610, a touchscreen 1612, a touchpad 1614, a near field communications unit (“NFC”) 1636, a sensor hub 1626, a thermal sensor 1638, an express chipset (“EC”) 1616, a trusted platform module (“TPM”) 1622, a BIOS / firmware / flash memory (“BIOS, firmware (FW) Flash”) 1608, a DSP 1652, a drive 1606 such as a solid state disk (“SSD”) or a hard disk (“HDD”), a wireless local area network unit (“WLAN”) 1640, a Bluetooth unit 1642, a wireless wide area network unit (“WWAN”) 1648, a global positioning system (GPS) 1646, a camera (“USB 3.0 camera”) 1644, such as a USB 3.0 camera, and / or a Low Power Double Data Rate ("LPDDR") memory unit ("LPDDR3") 1604, implemented, for example, in the LPDDR3 standard. These components may each be implemented in any suitable manner.

[0114] In at least one embodiment, other components may be communicatively coupled to processor 1602 via the above-mentioned components. In at least one embodiment, an accelerometer 1628, an ambient light sensor ("ALS") 1630, a compass 1632, and a gyroscope 1634 may be communicatively coupled to sensor hub 1626. In at least one embodiment, thermal sensor 1624, a fan 1620, a keyboard 1618, and a touchpad 1614 may be communicatively coupled to EC 1616. In at least one embodiment, speakers 1656, headphones 1658, and microphone ("mic") 1660 may be communicatively coupled to an audio unit ("audio codec and Class-D amplifier") 1654, which in turn may be communicatively coupled to DSP 1652. In at least one embodiment, the audio unit 1654 may include, for example and without limitation, an audio encoder / decoder ("codec") and a Class D amplifier.In at least one embodiment, the SIM card ("SIM") 1650 may be communicatively coupled to the WWAN unit 1648. In at least one embodiment, components such as the WLAN unit 1640 and the Bluetooth unit 1642, as well as the WWAN unit 1648, may be implemented in a Next Generation Form Factor ("NGFF").

[0115] Transfer logic 1428 may be used to perform operations related to remote descriptors, remote promise objects, remote promise future objects, streaming stacks, and RDMA operations in connection with one or more embodiments. Details of transfer logic 1428 are described below in connection with Fig. 1 - Fig. 13. In at least one embodiment, the transfer logic 1428 may be implemented in the Fig. 16 to enable RDMA transport of serialized objects in the streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system shown in Fig. 16 to enable RDMA transport of serialized objects in streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 16 to enable downstream nodes to receive data from upstream storage and enable downstream nodes to PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is completed, as described herein.

[0116] Fig. 17 is a block diagram of a processing system 1700 according to at least one embodiment. In at least one embodiment, the processing system 1700 includes one or more processors 1702 and one or more graphics processors 1710, and may be a single-processor desktop system, a multiprocessor workstation system, or a server system with a large number of processors 1702 or processor cores 1708. In at least one embodiment, the processing system 1700 is a processing platform integrated into a system-on-a-chip (SoC) integrated circuit for use in mobile, wearable, or embedded devices.

[0117] In at least one embodiment, processing system 1700 may include or be integrated with a server-based gaming platform, a gaming console, including a gaming and media console, a mobile gaming console, a handheld gaming console, or an online gaming console. In at least one embodiment, processing system 1700 is a mobile phone, a smartphone, a tablet computing device, or a mobile internet device. In at least one embodiment, processing system 1700 may also include, be coupled to, or integrated with a wearable device, such as a wearable device with a smart watch, smart glasses, an augmented reality device, or a virtual reality device.In at least one embodiment, the processing system 1700 is a television device or set-top box having one or more processors 1702 and a graphical interface generated by one or more graphics processors 1710.

[0118] In at least one embodiment, one or more processors 1702 each include one or more processor cores 1708 for processing one or more instructions that, when executed, perform operations for system and application software. In at least one embodiment, each of one or more processor cores 1708 is configured to process a particular instruction set 1709. In at least one embodiment, the instruction set 1712 may enable Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or Very Long Instruction Word (VLIW) computing. In at least one embodiment, the one or more processor cores 1708 may each process a different instruction set 1712, which may include instructions to facilitate emulation of other instruction sets.In at least one embodiment, the processor core(s) 1708 may also include other processing devices, such as a digital signal processor (DSP).

[0119] In at least one embodiment, the processor(s) 1702 includes a cache memory 1704. In at least one embodiment, the processor(s) 1708 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory 1704 is shared by various components of the processor(s) 1702. In at least one embodiment, the processor(s) 1702 also utilizes an external cache (e.g., a Level 3 (L3) cache or Last Level Cache (LLC)) (not shown), which may be shared by the processor core(s) 1708 using known cache coherence techniques. In at least one embodiment, the register file 1706 is additionally included in the processor(s) 1702, which may include various types of registers for storing different types of data (e.g.,Integer registers, floating-point registers, status registers, and an instruction pointer register.) In at least one embodiment, register file 1706 may include general-purpose registers or other registers.

[0120] In at least one embodiment, one or more processors 1702 are coupled to one or more interface bus lines 1714 for communicating communication signals, such as address, data, or control signals, between processor core(s) 1708 and other components in processing system 1700. In at least one embodiment, the interface bus line(s) 1714 may be a processor bus, such as a version of a Direct Media Interface (DMI) bus line. In at least one embodiment, the one or more interface bus lines 1714 are not limited to a DMI bus and may include one or more Peripheral Component Interconnect (e.g., PCI, PCI Express) bus lines, memory bus lines, or other types of interface bus lines. In at least one embodiment, the processor(s) 1702 include an integrated memory controller 1720 and a platform controller hub 1736.In at least one embodiment, memory controller 1720 facilitates communication between a memory device and other components of processing system 1700, while platform controller hub (PCH) 1736 provides connections to I / O devices via a local I / O bus.

[0121] In at least one embodiment, memory device 1722 may be dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, phase-change memory, or other memory device with suitable performance to serve as process memory. In at least one embodiment, memory device 1720 may operate as system memory for processing system 1700 to store data 1726 and instructions 1724 for use when one or more processors 1702 are executing an application or process. In at least one embodiment, memory controller 1720 is also coupled to an optional external graphics processor 1718 that can communicate with one or more graphics processors 1710 in processor(s) 1702 to perform graphics and media operations.In at least one embodiment, a display device 1716 may be connected to the processor(s) 1702. In at least one embodiment, a display device 1716 may comprise one or more of an internal display device, such as in a mobile electronic device or a laptop device, or an external display device connected via a display interface (e.g., DisplayPort, etc.). In at least one embodiment, the display device 1716 may comprise a head-mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) or augmented reality (AR) applications.

[0122] In at least one embodiment, the platform controller hub 1736 enables the connection of peripheral devices to the memory device 1722 and the processor(s) 1702 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include, among others, an audio controller 1748, a network controller 1738, a firmware interface 1734, a wireless transceiver 1732, touch sensors 1730, and a data storage device 1728 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 1728 can be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCI Express).In at least one embodiment, touch sensors 1730 may include touchscreen sensors, pressure sensors, or fingerprint sensors. In at least one embodiment, wireless transceiver 1732 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a cellular network transceiver such as a 3G, 4G, or Long Term Evolution (LTE) transceiver. In at least one embodiment, firmware interface 1734 enables communication with system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, network controller 1734 may enable network connection to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to the one or more interface bus lines 1714.In at least one embodiment, the audio controller 1748 is a multi-channel high-definition audio controller. In at least one embodiment, the processing system 1700 includes an optional legacy I / O controller 1740 for coupling legacy devices (e.g., Personal System 2 (PS / 2)) to the processing system 1700. In at least one embodiment, the platform controller hub 1736 may also be connected to one or more Universal Serial Bus (USB) controllers 1742 that connect input devices such as a keyboard and mouse combination 1744, a camera 1746, or other USB input devices.

[0123] In at least one embodiment, an instance of the memory controller 1720 and the platform controller hub 1736 may be integrated into a discrete external graphics processor, such as the external graphics processor 1718. In at least one embodiment, the platform controller hub 1736 and / or the memory controller 1720 may be external to one or more processors 1702. For example, in at least one embodiment, the processing system 1700 may include an external memory controller 1720 and a platform controller hub 1736, which may be configured as a memory controller hub and a peripheral controller hub within a system chipset in communication with the one or more processors 1702.

[0124] Transfer logic 1428 may be used to perform operations related to remote descriptors, remote promise objects, remote promise future objects, streaming stacks, and RDMA operations in connection with one or more embodiments. Details of transfer logic 1428 are described below in connection with Fig. 1 to Fig. 13. In at least one embodiment, parts or all of the transfer logic 1428 may be integrated into the one or more graphics processors 1710. In at least one embodiment, the transfer logic 1428 may be implemented in the Fig. 17 to enable RDMA transport of serialized objects in the streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system shown in Fig. 17 may be used to enable RDMA transport of serialized objects in streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig. 17 to enable downstream nodes to receive data from upstream storage and enable downstream nodes to PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is completed, as described herein.

[0125] Fig. 18 is a block diagram of a processor 1800 having one or more processor cores (1) 1802a through core (N) 1802b, an integrated memory controller 1818, and an integrated graphics processor 1808, according to at least one embodiment. In at least one embodiment, the processor 1800 may include additional cores up to and including the additional core (N) 1802b, represented by dashed, lined boxes. In at least one embodiment, each of processor core (1) 1802a through core (N) 1802b includes one or more internal cache units (1) 1804a through cache units (N) 1804b. In at least one embodiment, each processor core also has access to one or more shared cache units 1806.

[0126] In at least one embodiment, the internal cache unit(s) (1) 1804a through cache unit(s) (N) 1804b and the shared cache units 1806 represent a cache memory hierarchy within the processor 1800. In at least one embodiment, cache unit(s) (1) 1804a through cache unit(s) (N) 1804b may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as a Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, with a highest level of cache prior to external memory being classified as LLC. In at least one embodiment, the cache coherency logic maintains coherency between various cache units 1806 and cache unit(s) (1) 1804a through cache unit(s) (N) 1804b.

[0127] In at least one embodiment, the processor 1800 may also include a set of one or more bus controller units 1814 and a system agent core 1810. In at least one embodiment, one or more bus controller units 1814 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. In at least one embodiment, the system agent core 1810 provides management functions for various processor components. In at least one embodiment, the system agent core 1810 includes one or more integrated memory controllers 1818 to manage access to various external memory devices (not shown).

[0128] In at least one embodiment, one or more of processor cores (1) 1802a through core (N) 1802b include support for concurrent multithreading. In at least one embodiment, system agent core 1810 includes components for coordinating and operating processor core (1) 1802a through core (N) 1802b during multithreaded processing. In at least one embodiment, system agent core 1810 may additionally include a power control unit (PCU) that includes logic and components for regulating one or more power states of processor core (1) 1802a through core (N) 1802b and graphics processor 1808.

[0129] In at least one embodiment, processor 1800 additionally includes graphics processor 1808 for performing graphics processing operations. In at least one embodiment, graphics processor 1808 is coupled to shared cache units 1806 and a system agent core 1810 that includes one or more integrated memory controllers 1822. In at least one embodiment, system agent core 1810 also includes a display controller 1812 for controlling the output of the graphics processor to one or more coupled displays. In at least one embodiment, display controller 1812 may also be a separate module connected to graphics processor 1808 via at least one interconnect, or it may be integrated into graphics processor 1808.

[0130] In at least one embodiment, a ring-based interconnect 1822 is used to connect internal components of processor 1800. In at least one embodiment, an alternative interconnect may be used, such as a point-to-point connection, a switched connection, or other techniques. In at least one embodiment, graphics processor 1808 is connected to ring-based interconnect 1822 via an I / O connection 1816.

[0131] In at least one embodiment, I / O connection 1816 represents at least one of several types of I / O connections, including an on-package I / O connection that facilitates communication between various processor components and an embedded high-performance memory module 1820, such as an eDRAM module. In at least one embodiment, each of processor cores (1) 1802a through core (N) 1802b and graphics processor 1808 utilize embedded memory module 1820 as a shared last-level cache.

[0132] In at least one embodiment, processor core (1) 1802a through core (N) 1802b are homogeneous cores executing a common instruction set architecture. In at least one embodiment, processor core (1) 1802a through core (N) 1802b are heterogeneous in terms of instruction set architecture (ISA), where one or more processor cores (1) 1802a through core (N) 1802b execute a common instruction set, while one or more other cores of processor core (1) 1802a through core (N) 1802b execute a subset of a common instruction set or a different instruction set. In at least one embodiment, processor core (1) 1802a through core (N) 1802b are heterogeneous in terms of microarchitecture, where one or more relatively higher-power cores combine with one or more lower-power cores.In at least one embodiment, the processor 1800 may be implemented on one or more chips or as an SoC integrated circuit.

[0133] Transfer logic 1428 may be used to perform operations related to remote descriptors, remote promise objects, remote promise future objects, streaming stacks, and RDMA operations in connection with one or more embodiments. Details of transfer logic 1428 are described below in connection with Fig. 1 to Fig. 13. In at least one embodiment, parts or all of the transfer logic 1428 may be integrated into the processor 1800. For example, in at least one embodiment, the training and / or inference techniques described herein may utilize one or more arithmetic logic units (ALUs) located in the graphics processor 1808, in the processor core (1) 1802a through core (N) 1802b, or in other components in Fig. 18. In at least one embodiment, the transfer logic 1428 may be included in the Fig. 18 to enable RDMA transport of serialized objects in the streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system shown in Fig. 18 to enable RDMA transport of serialized objects in streaming pipelines described herein. In at least one embodiment, the transfer logic 1428 may be implemented in the system illustrated in Fig.18 to enable downstream nodes to receive data from upstream storage and enable downstream nodes to PUSH data to upstream nodes, while those upstream nodes relinquish their execution context and do not consume resources until the downstream PUSH operation is completed, as described herein.

[0134] The disclosure of this application also includes the following numbered sections: Clause 1. A procedure which includes: Receiving, by a first computing device in a data center, a remote promise object associated with a first serialized object from a second computing device in the data center over a network structure, the remote promise object uniquely identifying a first contiguous block of the first serialized object stored in memory associated with the second computing device, the remote promise object being released when a second serialized object is returned to the second computing device; Obtaining the contents of the first contiguous block from the memory associated with the second computing device for storage in a memory associated with the first computing device; Sending the contents of a second contiguous block of the second serialized object stored in the memory associated with the first computing device to the memory associated with the second computing device; and Sending a message to the second computing device to release the remote Promise object in response to the second serialized object being returned to the second computing device. Clause 2. The method of clause 1, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to memory associated with the second computing device, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing an ownership interest in the first serialized object. Clause 3. The method of clause 2, wherein sending the message to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from memory associated with the first computing device when the value of the reference counter token meets a threshold. Clause 4. The method of Clause 1, wherein a size of the remote Promise object is smaller than the size of the first contiguous block. Clause 5. The procedure under Clause 1, whereby: Obtaining the contents of the first contiguous block comprises performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the memory associated with the second computing device to transfer it to the memory associated with the first computing device over the network structure; and sending the contents of the second contiguous block comprises performing an RDMA put operation to send the contents of the second serialized object from the memory associated with the first computing device to the memory associated with the second computing device via the network structure. Clause 6. The procedure under Clause 1, which further includes: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in memory associated with the first computing device, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. Clause 7. The method of Clause 6, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical machine corresponding to the memory associated with the second computing device, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, the method further comprising: Updating the value of the reference counter token in response to receiving the second message; and Releasing the remote promise object from the memory associated with the first computing device when the value of the reference counter token meets a threshold. Clause 8. A computer system comprising: a first computing device; and a first memory coupled to the first computing device, the first computing device for: Receiving, by a first computing device in a data center, a remote promise object associated with a first serialized object from a second computing device in the data center over a network structure, wherein the remote promise object is an object that uniquely identifies a first contiguous block of the first serialized object stored in a second memory associated with the second computing device, wherein the remote promise object is released when a second serialized object is returned to the second computing device; Obtaining contents of the first contiguous block from the second memory for storage in a first memory associated with the first computing device; Sending the contents of a second contiguous block of the second serialized object stored in the first memory to the second memory; and Sending a message to the second computing device to release the remote Promise object in response to the second serialized object being returned to the second computing device. Clause 9. The computer system of Clause 8, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to the second memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing an ownership interest in the first serialized object. Clause 10. The computer system of Clause 9, wherein the message sent to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from the first memory in response to the value of the reference counter token meeting a threshold. Clause 11. The computer system of Clause 8, wherein a size of the remote promise object is smaller than the size of the first contiguous block. Clause 12. The computer system referred to in Clause 8, where: the first computing device is to obtain the contents of the first contiguous block by performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the second memory for transmission to the first memory over the network structure; and the first computing device is to send the contents of the second contiguous block by performing an RDMA put operation to send the contents of the second serialized object from the first memory to the second memory via the network structure. Clause 13. The computer system of Clause 8, wherein the first computer device further serves to: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in the first memory, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. Clause 14. The computer system of Clause 13, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical machine corresponding to the first memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, wherein the first computing device is further for: Updating the value of the reference counter token in response to receiving the second message; and Release the remote Promise object from the first memory in response to the value of the reference count token meeting a threshold. Clause 15. A data center comprising: a variety of computer devices a plurality of memory locations, each memory location being associated as local memory with one of the plurality of computing devices; and a network structure coupled to the plurality of computing devices, wherein a first computing device of the plurality of computing devices is for: Receiving, by a first computing device in the data center, a remote promise object associated with a first serialized object from a second computing device in the data center over the network fabric, wherein the remote promise object is an object that uniquely identifies a first contiguous block of the first serialized object stored in a second memory associated with the second computing device, wherein the remote promise object is released when a second serialized object is returned to the second computing device; Obtaining contents of the first contiguous block from the second memory for storage in a first memory associated with the first computing device; Sending the contents of a second contiguous block of the second serialized object stored in the first memory to the second memory; and Sending a message to the second computing device to release the remote Promise object in response to the second serialized object being returned to the second computing device. Clause 16. The data center of Clause 15, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to the second memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing an ownership interest in the first serialized object. Clause 17. The data center of Clause 16, wherein the message sent to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from the first memory in response to the value of the reference counter token meeting a threshold. Clause 18. The data center according to Clause 15, wherein a size of the remote promise object is smaller than the size of the first contiguous block. Clause 19. The data centre referred to in Clause 15, where: the first computing device is to obtain the contents of the first contiguous block by performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the second memory for transmission to the first memory over the network structure; and the first computing device is to send the contents of the second contiguous block by performing an RDMA put operation to send the contents of the second serialized object from the first memory to the second memory via the network structure. Clause 20. The data centre as defined in Clause 15, wherein the first computing device further serves to: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in the first memory, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. Clause 21. The data center of Clause 20, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical machine corresponding to the first memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, the first computing device further for: Updating the value of the reference counter token in response to receiving the second message; and Release the remote Promise object from the first memory in response to the value of the reference count token meeting a threshold.

[0135] Other variations are within the spirit of the present disclosure. While the disclosed techniques are susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof have been shown in the drawings and described in detail above. It should be understood, however, that the intention is not to limit the disclosure to any particular form or forms; on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure as defined by the appended claims.

[0136] The use of the terms "a" and "an" and "the," "the," "which," and similar designations in connection with the description of disclosed embodiments (particularly in connection with the following claims) is to be construed to include both the singular and the plural, unless otherwise stated herein or clearly contradicted by context, and not as a definition of any term. The terms "comprising," "with," "including," and "containing" are to be construed as open-ended terms (in the sense of "including, but not limited to") unless otherwise specified. The term "connected," when unmodified and referring to physical connections, is to be construed as partially or wholly contained, attached to, or connected thereto, even if something in between.Any enumeration of ranges of values ​​is intended merely as a shorthand method for referring individually to each value falling within the range, unless otherwise noted here, and each value is included in the specification as if individually enumerated here. The use of the term "set" (e.g., "a set of items") or "subset" is, unless otherwise noted or contradicted by the context, to a non-empty collection comprising one or more members. Further, unless otherwise noted or contradicted by the context, the term "subset" denotes a corresponding set, not necessarily a true subset of the corresponding set, but the subset and corresponding set may be the same.

[0137] Subjunctive expressions, such as sentences of the form "at least one of A, B, and C" or "at least one of A, B, and C," are, unless explicitly stated otherwise or clearly contradicted by the context, taken with the context as they are generally used to represent that an object, term, etc., can be either A or B or C, or any non-empty subset of the set of A, B, and C. In an illustrative example of a three-member sentence, the subjunctive expressions "at least one of A, B, and C" and "at least one of A, B, and C" refer to one of the following sentences: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Such a subjunctive formulation is therefore not intended to mean in general that in certain embodiments at least one of A, at least one of B and at least one of C must be present.Additionally, unless otherwise noted or contradicted by context, the term "plurality" denotes a state of pluralism (e.g., "a plurality of items" denotes multiple items). A plurality consists of at least two items, but may be more if indicated either explicitly or by context. Unless otherwise noted or evident from context, the phrase "based on" means "at least partly based on" and not "exclusively based on."

[0138] The operations of the processes described herein may be performed in any suitable order, unless otherwise specified herein or apparent from the context. In at least one embodiment, a process such as the processes described herein (or variations and / or combinations thereof) is performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that are collectively executed on one or more processors, by hardware, or combinations thereof. In at least one embodiment, the code is stored on a computer-readable storage medium, e.g., in the form of a computer program comprising a plurality of instructions executable by one or more processors.In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., a propagating transient electrical or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers for transient signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media on which executable instructions (or other memory for storing executable instructions) are stored that, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform operations described herein.In at least one embodiment, a set of non-transitory computer-readable storage media comprises a plurality of non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media of a plurality of non-transitory computer-readable storage media lacks all code, while a plurality of non-transitory computer-readable storage media collectively store all code. In at least one embodiment, executable instructions are executed such that different instructions are executed by different processors—for example, a non-transitory computer-readable storage medium stores instructions, and a central processing unit ("CPU") executes some of the instructions, while a graphics processing unit ("GPU") executes other instructions. In at least one embodiment, different components of a computer system have separate processors, and different processors execute different subsets of instructions.

[0139] Accordingly, in at least one embodiment, computer systems are configured to implement one or more services that individually or collectively perform operations of the processes described herein, and such computer systems are configured with applicable hardware and / or software that enable operations to be performed. Further, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, a distributed computer system comprising multiple devices that operate differently such that the distributed computer system performs operations described herein and a single device does not perform all operations.

[0140] The use of examples or exemplary expressions (e.g., "such as") is intended only to better illustrate embodiments of the disclosure and does not limit the scope of the disclosure unless otherwise claimed. No language in the specification should be construed to identify any unclaimed element as essential to the practice of the disclosure.

[0141] All references cited herein, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each reference were individually and expressly indicated to be incorporated by reference and reproduced herein in its entirety.

[0142] The terms "coupled" and "connected," and their derivatives, may be used in the description and claims. It should be understood that these terms are not intended to be synonymous. Rather, in certain examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" can also mean that two or more elements are not in direct contact with each other, but nevertheless cooperate or interact with each other.

[0143] Unless expressly stated otherwise, terms such as "processing", "computation", "calculating", "determining" or the like throughout this specification refer to actions and / or processes of a computer or computer system or similar electronic computing device that manipulates and / or transforms data represented as physically, such as electronic quantities in the registers and / or memory of the computer system into other data similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computer system.

[0144] Similarly, the term "processor" may refer to a device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As non-limiting examples, "processor" may be a CPU or a GPU. A "computing platform" may include one or more processors. As used herein, "software" processes may include, for example, software and / or hardware units that perform work over time, such as tasks, threads, and intelligent agents. Each process may also refer to multiple processes to execute instructions sequentially or in parallel, continuously or intermittently.The terms “system” and “process” are used interchangeably here in the sense that the system may embody one or more processes and processes may be considered as a system.

[0145] In this document, reference may be made to the acquiring, acquiring, receiving, or inputting of analog or digital data to a subsystem, a computer system, or a computer-implemented calculator. The acquiring, acquiring, receiving, or inputting of analog and digital data may be performed in a variety of ways, such as by receiving data as a parameter of a function call or a call to an application programming interface. In some implementations, the process of acquiring, acquiring, receiving, or inputting of analog or digital data may be performed by transmitting data over a serial or parallel interface. In another implementation, the process of acquiring, acquiring, receiving, or inputting of analog or digital data may be performed by transmitting data over a computer network from the providing entity to the acquiring entity.It may also refer to the provision, output, transmission, sending, or presentation of analog or digital data. In various examples, the process of providing, output, transmission, sending, or presentation of analog or digital data may be achieved by passing data as an input or output parameter of a function call, a parameter of an application programming interface, or an interprocess communication mechanism.

[0146] Although the above discussion sets forth example implementations of the described techniques, other architectures may be used to implement the described functionality and are intended to be within the scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for discussion purposes, various functions and responsibilities may be distributed and allocated in different ways depending on the circumstances.

[0147] Further, although the subject matter has been described in language referring to structural features and / or methodological acts, it is to be understood that the subject matter claimed in the appended claims is not necessarily limited to the particular features or acts described. Rather, particular features and acts are disclosed as exemplary forms of implementing the claims.

[0148] It is to be understood that the aspects and embodiments described above are purely exemplary and that changes in detail may be made within the scope of the claims.

[0149] Each apparatus, method, and feature disclosed in the description and (optionally) in the claims and drawings may be provided independently or in any suitable combination.

[0150] The reference numbers used in the claims are for illustrative purposes only and do not limit the scope of the claims.

Claims

[1] Procedure comprising: Receiving, by a first computing device (122) in a data center, a remote promise object associated with a first serialized object from a second computing device (124) in the data center over a network structure, the remote promise object uniquely identifying a first contiguous block of the first serialized object stored in a memory (116) associated with the second computing device (124), the remote promise object being released when a second serialized object is returned to the second computing device (124); Obtaining the contents of the first contiguous block from the memory (116) associated with the second computing device (124) for storage in a memory (114) associated with the first computing device (122); Sending the contents of a second contiguous block of the second serialized object stored in the memory (114) associated with the first computing device (122) to the memory (116) associated with the second computing device (124); and Sending a message to the second computing device (124) to release the remote promise object in response to the second serialized object being returned to the second computing device (124). [2] The method of claim 1, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to memory associated with the second computing device, a remote direct memory access (RDMA) key, and a value of a reference counter token representing an ownership interest in the first serialized object. [3] The method of claim 2, wherein sending the message to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from the memory associated with the first computing device when the value of the reference counter token meets a threshold. [4] The method of claim 1, 2 or 3, wherein a size of the remote promise object is smaller than the size of the first contiguous block. [5] Method according to one of the preceding claims, wherein: Obtaining the contents of the first contiguous block comprises performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the memory associated with the second computing device to transfer it to the memory associated with the first computing device over the network structure; and sending the contents of the second contiguous block comprises performing an RDMA put operation to send the contents of the second serialized object from the memory associated with the first computing device to the memory associated with the second computing device over the network structure. [6] A method according to any one of the preceding claims, further comprising: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in memory associated with the first computing device, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. [7] The method of claim 6, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical machine corresponding to the memory associated with the second computing device, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, the method further comprising: Updating the value of the reference counter token in response to receiving the second message; and Releasing the remote promise object from the memory associated with the first computing device when the value of the reference counter token meets a threshold. [8] A computer system (100) comprising: a first computer device (122); and a first memory (114) coupled to the first computer device (122), the first computer device (122) serving to: Receiving, by a first computing device (122) in a data center, a remote promise object associated with a first serialized object from a second computing device (124) in the data center over a network structure, wherein the remote promise object is an object that uniquely identifies a first contiguous block of the first serialized object stored in a second memory (116) associated with the second computing device (124), wherein the remote promise object is released when a second serialized object is returned to the second computing device (122); Obtaining contents of the first contiguous block from the second memory (116) for storage in a first memory (114) associated with the first computing device (122); Sending the contents of a second contiguous block of the second serialized object stored in the first memory (114) to the second memory (116); and Sending a message to the second computing device (124) to release the remote promise object in response to the second serialized object being returned to the second computing device (124). [9] The computer system of claim 8, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to the second memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing an ownership interest in the first serialized object. [10] The computer system of claim 9, wherein the message sent to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from the first memory in response to the value of the reference counter token meeting a threshold. [11] The computer system of claim 8, 9 or 10, wherein a size of the remote promise object is smaller than the size of the first contiguous block. [12] A computer system according to any one of claims 8 to 11, wherein: the first computing device is to obtain the contents of the first contiguous block by performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the second memory for transmission to the first memory over the network structure; and the first computing device is to send the contents of the second contiguous block by performing an RDMA put operation to send the contents of the second serialized object from the first memory to the second memory via the network structure. [13] A computer system according to any one of claims 8 to 12, wherein the first computer device further serves to: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in the first memory, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. [14] The computer system of claim 13, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical host corresponding to the first memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, the first computing device further for: Updating the value of the reference counter token in response to receiving the second message; and Release the remote Promise object from the first memory in response to the value of the reference count token meeting a threshold. [15] Data center, which includes: a variety of computer devices a plurality of memory locations, each memory location being associated as local memory with one of the plurality of computing devices; and a network structure coupled to the plurality of computing devices, wherein a first computing device of the plurality of computing devices is for: Receiving, by a first computing device (122) in the data center, a remote promise object associated with a first serialized object from a second computing device (124) in the data center over the network structure, wherein the remote promise object is an object that uniquely identifies a first contiguous block of the first serialized object stored in a second memory (116) associated with the second computing device (124), wherein the remote promise object is released when a second serialized object is returned to the second computing device (124); Obtaining contents of the first contiguous block from the second memory (116) for storage in a first memory (114) associated with the first computing device (122); Sending the contents of a second contiguous block of the second serialized object stored in the first memory (114) to the second memory (116); and Sending a message to the second computing device (124) to release the remote promise object in response to the second serialized object being returned to the second computing device (124). [16] The data center of claim 15, wherein the remote promise object comprises a starting address of the first contiguous block, a size of the first contiguous block, a physical machine identifier corresponding to the second memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing an ownership interest in the first serialized object. [17] The data center of claim 16, wherein the message sent to the second computing device causes the second computing device to update the value of the reference counter token, and wherein the first serialized object is released from the first memory in response to the value of the reference counter token meeting a threshold. [18] The data center of claim 15, 16 or 17, wherein a size of the remote promise object is smaller than the size of the first contiguous block. [19] Data center according to one of claims 15 to 18, wherein: the first computing device is to obtain the contents of the first contiguous block by performing an RDMA GET operation using the Remote Promise object to obtain the contents of the first contiguous block from the second memory for transmission to the first memory over the network structure; and the first computing device is to send the contents of the second contiguous block by performing an RDMA put operation to send the contents of the second serialized object from the first memory to the second memory via the network structure. [20] A data center according to any one of claims 15 to 19, wherein the first computing device further serves to: Creating a second remote promise object, the second remote promise object being an object that uniquely identifies a third contiguous block of a third serialized object stored in the first memory, the second remote promise object being released when a fourth serialized object is returned to the first computing device; Sending the second Remote Promise object over the network structure to another computing device in the data center; and Receiving a second message from the other computing device to release the second remote Promise object in response to the fourth serialized object being returned to the first computing device. [21] The data center of claim 20, wherein the second remote promise object is an object that uniquely identifies: a starting address of the third contiguous block, a size of the third contiguous block, a physical host corresponding to the first memory, a remote direct memory access (RDMA) access key, and a value of a reference counter token representing one or more ownership interests in the third serialized object, the first computing device further for: Updating the value of the reference counter token in response to receiving the second message; and Release the remote Promise object from the first memory in response to the value of the reference count token meeting a threshold.

Images